vinez/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-10 14:27:00 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

chore: remove committed node_modules + stray/internal markdown (repo hygiene) (#1528)

Browse source 
## Description

Repo hygiene for a public OSS project: removes committed `node_modules`,
stray/internal/draft markdown, and commercial-surface references —
keeping every real doc (the published docs site, the wiki guides, and
all component READMEs) intact. Every file was content-audited before
removal, and load-bearing files were verified against the code/CI and
kept.

Net: **1,695 files changed, +23 / −266,409** (the deletions are
dominated by a committed `node_modules` tree).

Closes # (no tracking issue)

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [x] Code refactoring (no functional changes)

## Changes Made

**Removed (verified to have no code/CI dependencies):**
- `examples/vercel-ai-sdk-pr/` — 1,649 committed `node_modules` files
(zero example source); `node_modules/` added to `.gitignore`.
- `docs/spec/` (23 draft "Living Specification" files — orphaned,
`1.0.0-draft`, drifted from the code), `docs/superpowers/` (2 agent
plans), `docs/proposals/` (2 internal/commercial memos).
- 6 orphan `docs/*.md` (auth-modes, bedrock,
claude-code-vertex-headroom, cortex-code, output-token-reduction-guide,
rtk-loop-weighting).
- `PR.md` (committed PR draft), `ENTERPRISE.md`, `.github/FUNDING.yml`.

**Content scrubs:**
- Removed unreleased "Headroom Cloud" / `api.headroom.ai` / `hr_`
references from `configuration.mdx`, `wiki/configuration.md`,
`wiki/typescript-sdk.md`, `sdk/typescript/README.md` (reworded to
neutral, accurate phrasing).
- Dropped a stale "awaiting maintainer before merge" line from
`plugins/headroom-oauth2/SPEC.md`; tidied `.gitignore` comments (kept
the protective `headroom-managed/` ignore rule).
- Fixed the now-dangling links into removed files (README
nav/`output-token-reduction` link, `scripts/README`, `wiki/vertex`).

**Explicitly KEPT (load-bearing — would orphan in-code citations if
removed):**
- `.changelog.md` — consumed by `.github/workflows/release.yml` (read as
the release-notes file).
- `REALIGNMENT/`, `docs/observability.md`, `docs/rtk-architecture.md`,
`wiki/plans/`, `TESTING-copilot-subscription.md` — referenced by the
Rust core / Python / tests as design docs.

## Testing

- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [ ] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# Docs/markdown + .gitignore only — no Python/Rust source changed, so the
# behavioral test suite is unaffected. Verified the cleanup did not orphan
# references or break the published docs site:

$ git ls-files 'docs/content/docs/*.mdx' | wc -l      # published site intact
42
$ # meta.json nav unchanged; no published page removed.

$ grep -rnI "Headroom Cloud|api.headroom.ai|'hr_" $(git ls-files '*.md' '*.mdx')
>>> none

$ # dangling refs to removed files (excl pre-existing P0/P2 spec stubs that
$ # never existed in git): none remaining.
```

## Real Behavior Proof

- Environment: macOS, local git clone of the repo (markdown/.gitignore
changes only — no runtime).
- Exact command / steps: 4 read-only content-audit agents classified
every `.md`/`.mdx` file; each removal candidate was cross-checked
against the codebase (`grep` for citations in `.rs`/`.py`/tests,
workflows, and configs); only files with no dependents were removed; the
tree was re-grepped after removal to confirm no new dangling references;
verified the published docs site page count (`git ls-files
'docs/content/docs/*.mdx' | wc -l` = 42, unchanged).
- Observed result: the 42-page published docs site and all wiki guides
are untouched; no source or workflow references a removed file;
`.changelog.md` (consumed by release.yml) and the code-cited design docs
were detected as dependencies and kept; the committed `node_modules`
tree is removed and `node_modules/` is gitignored so it can't be
re-committed; zero "Headroom Cloud"/`headroom.dev` references remain.
- Not tested: N/A — no executable code changed (only markdown, `.mdx`,
and `.gitignore`), so the behavioral test suite is unaffected.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

- This branch deletes `.github/FUNDING.yml` while PR #1526 edits it —
the two will be sequenced at merge (delete wins).
- A follow-up option (not in this PR): also remove the internal design
docs that are currently cited by the code (`REALIGNMENT/`,
`docs/observability.md`, `docs/rtk-architecture.md`, `wiki/plans/`) —
that requires scrubbing ~15–20 in-code citations so nothing dangles, so
it's deliberately deferred.
- Untracked local working files (`benchmarks/hf_pilot/`,
`tools/copilot-test/`) are intentionally left out of git (not
committed).
This commit is contained in:
Tejas Chopra 2026-06-27 23:32:54 -07:00 • committed by GitHub
parent 077e3e9b9a
commit a639540959
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1695 changed files with 23 additions and 266409 deletions
Download patch file Download diff file Expand all files Collapse all files

7
.github/FUNDING.yml vendored
View file

@ -1,7 +0,0 @@
# These are supported funding model platforms
github: [headroom-sdk]
# patreon: headroom
# open_collective: headroom
# ko_fi: headroom
# custom: ["https://headroomlabs.ai/sponsor"]

8
.gitignore vendored
View file

@ -118,6 +118,9 @@ env.bak/
venv.bak/
.python-version
# Node.js dependencies (never commit vendored deps)
node_modules/
# Secrets and API keys - NEVER commit these
*.pem
*.key
@ -229,10 +232,7 @@ CLAUDE.md
# Vitals provenance data
.vitals/
# Superpowers working files (plans, specs, brainstorming)
# docs/spec/ (lives in git - git-versioned living specification)
# Managed platform (separate private repo)
# Separate private repos — never commit here
headroom-managed/
# Local act testing (never commit test tokens)

5
ENTERPRISE.md
View file

@ -1,5 +0,0 @@
# Enterprise
## Enterprise Support
Interested in using Headroom at scale? We're happy to discuss your deployment requirements! Email us at: **hello@headroomlabs.ai**

170
PR.md
View file

@ -1,170 +0,0 @@
## Description
Implement unified CI/CD release automation with semantic versioning across all three packages:
- **Python (headroom-ai)** — pip package on PyPI
- **TypeScript SDK (headroom-ai)** — npm package on npmjs.org
- **OpenClaw plugin (headroom-openclaw)** — npm package on npmjs.org and GitHub Package Registry
Currently the three packages are independently versioned (0.5.25 / 0.1.0 / 0.1.0). This PR introduces a single-source-of-truth version in `pyproject.toml` that propagates to all packages on every release, driven by conventional commit messages.
Fixes #(issue number)
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)
## Changes Made
### New Files
**Scripts:**
- `scripts/version-sync.py` — Reads version from `pyproject.toml`, updates all 4 version files. Supports `--version X.Y.Z` and `--bump {major,minor,patch}`.
- `scripts/changelog-gen.py` — Parses conventional commits since last tag, groups by type, generates markdown changelog with breaking change detection.
- `scripts/verify-versions.py` — Pre-release sanity check that all 4 version files are in sync.
- `scripts/tests/test_version_sync.py` — 5 tests for version-sync.py
- `scripts/tests/test_changelog_gen.py` — 23 tests for changelog-gen.py
**Workflows:**
- `.github/workflows/release.yml` — Unified release pipeline: detect → build → publish-pypi → publish-npm → publish-github-packages → create-release
- `.commitlintrc.json` — Conventional commit enforcement via `@commitlint/config-conventional`
**Local Testing (act):**
- `.actrc` — Default `act` flags (Ubuntu runner, reuse, quiet)
- `.github/act/dry-run.json` — `act` event file for dry-run testing
- `.github/act/push-feat.json` — `act` event file for simulating a feat commit
- `.actrc.local.example` — Local override template for `act`
- `.env.act.example` — Secrets documentation template for `act` local testing
**Documentation:**
- `docs/content/docs/releases.mdx` — Full documentation for the release pipeline, testing guide, and configuration reference
### Modified Files
- `.github/workflows/ci.yml` — Added `commitlint` job to enforce conventional commits
- `.github/workflows/publish.yml` — Changed from `release` trigger to `workflow_dispatch` only (superseded by `release.yml`)
- `.github/workflows/release.yml` — **Rewritten** with canonical+commit-height algorithm (no more commit loop)
- `.gitignore` — Added `!scripts/version-sync.py`, `!scripts/changelog-gen.py`, `!scripts/verify-versions.py`, `!scripts/tests/`, `.env.act`, `.actrc.local`
## Testing
- [x] Unit tests pass (`pytest`)
- `scripts/tests/test_version_sync.py` — 5/5 passing
- `scripts/tests/test_changelog_gen.py` — 23/23 passing
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) — pre-existing issue in `headroom/cli/wrap.py:487` (unrelated)
- [x] New tests added for new functionality
- [x] Workflow tested with `act` (dry-run passes all jobs through build step — no infinite loop)
## Algorithm Validation
The canonical+commit-height algorithm was validated with test cases:
- Canonical `0.5.25`, no prior tag, `feat:` commit → git tag `v0.6.0.0`, npm `0.6.0` ✅
- Canonical `0.5.25`, tag `v0.5.25.2`, `fix:` commit → git tag `v0.5.25.3`, npm `0.5.26` ✅
- Canonical `0.5.25`, no prior tag, `fix:` commit → git tag `v0.5.25.0`, npm `0.5.25` ✅
- Manual override `1.2.3` → git tag `v1.2.3`, npm `1.2.3` ✅
## Test Output
```
scripts/tests/test_version_sync.py .....
scripts/tests/test_changelog_gen.py .......................
```
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have made corresponding changes to the documentation
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
### Version Bump Logic
**Canonical + Commit Height Algorithm** — The workflow NEVER commits back to the repo. `pyproject.toml` is the canonical source of truth, updated manually before merging.
| Commit | Bump | Git Tag | npm Version |
|--------|------|---------|-------------|
| `fix:`, `ci:`, `chore:`, `perf:`, `refactor:` | patch | `v0.5.25.3` | `0.5.26` |
| `feat:` | minor | `v0.6.0.0` | `0.6.0` |
| `feat!:` or `feat:` + `BREAKING CHANGE` body | major | `v1.0.0.0` | `1.0.0` |
The git tag uses `v{canonical}.{height}` (e.g., `v0.5.25.3` = 3 commits since canonical `0.5.25`). npm versions use 3-part semver, bumped from canonical.
### Package Publishing Targets
| Package | Target | Status |
|---------|--------|--------|
| `headroom-ai` (Python) | PyPI | ✅ via `pypa/gh-action-pypi-publish` |
| `headroom-ai` (TypeScript SDK) | npmjs.org | ✅ via `npm publish` |
| `headroom-openclaw` | npmjs.org | ✅ via `npm publish` |
| `headroom-openclaw` | GitHub Package Registry | ✅ via `npm publish --registry npm.pkg.github.com` |
### Safety Gates
Each publish job requires both `dry_run != 'true'` **and** the corresponding skip variable not set:
| Variable | Effect |
|----------|--------|
| `PYPI_SKIP=true` | Skip PyPI publish |
| `NPM_SKIP=true` | Skip both npm publishes |
| `GH_PACKAGES_SKIP=true` | Skip GitHub Package Registry publish |
Set in: **GitHub repo → Settings → Variables → Actions Variables**.
### Workflow Triggers
- **Auto:** On push to `main` — analyzes latest commit, bumps version, builds, publishes, creates GitHub Release
- **Manual:** `workflow_dispatch` with optional `version` override and `dry_run` flag
- **Paths ignore:** Skips runs when only `docs/`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `scripts/`, `.commitlintrc.json`, `.actrc`, `.github/act/`, or `.env.act.example` change
### Local Testing
```bash
# Install act
winget install act
# Dry-run (no publishes)
act -W .github/workflows/release.yml -e .github/act/dry-run.json
# Test feat: commit (minor bump)
act -W .github/workflows/release.yml -e .github/act/push-feat.json
```
### Required GitHub Secrets
| Secret | Purpose |
|--------|---------|
| `NPM_TOKEN` | Publishing to npmjs.org |
| `GITHUB_TOKEN` | GitHub Package Registry (auto-provided by GitHub Actions) |
PyPI uses trusted publisher OIDC — no secret required, only the `pypi` GitHub Environment must be configured.
### First Release Note
The TypeScript packages are currently at `0.1.0` while Python is at `0.5.25`. The first release will align all three to the same version. Update `pyproject.toml` to the desired canonical version before merging, then use `workflow_dispatch` with a manual `version` input to set the target explicitly.
After each release, update `pyproject.toml` to match the published version to keep the canonical current and ensure unique git tags.
### Parameterized Configuration
All package names and registries are top-level `env` constants in `release.yml`:
```yaml
env:
PYPI_PACKAGE: headroom-ai
PYPI_ENVIRONMENT: pypi
NPM_REGISTRY_URL: https://registry.npmjs.org
NPM_SDK_PACKAGE: headroom-ai
NPM_OPENCLAW_PACKAGE: headroom-openclaw
GITHUB_PACKAGES_REGISTRY_URL: https://npm.pkg.github.com
```

5
README.md
View file

@ -26,8 +26,7 @@
<a href="#proof">Proof</a> ·
<a href="#agent-compatibility-matrix">Agents</a> ·
<a href="https://discord.gg/yRmaUNpsPJ">Discord</a> ·
<a href="llms.txt">llms.txt</a> ·
<a href="ENTERPRISE.md">Enterprise</a>
<a href="llms.txt">llms.txt</a>
</p>
<p align="center"><sub>
@ -181,7 +180,7 @@ unshaped as a control group: `export HEADROOM_OUTPUT_HOLDOUT=0.1`. The dashboard
shows an **Output Tokens Saved** card next to input compression, labelled
`measured` or `estimated` with the confidence band.
→ Full write-up incl. the measurement methodology: [`docs/proposals/output-token-reduction.md`](docs/proposals/output-token-reduction.md)
→ Full write-up incl. the measurement methodology: [Output token reduction](https://headroom-docs.vercel.app/docs/savings)
<a href="https://www.star-history.com/?repos=chopratejas%2Fheadroom&type=date&legend=top-left">
<picture>

5
RUST_DEV.md
View file

@ -288,9 +288,8 @@ doesn't rediscover them.
## Multi-worker deployment — CCR fragmentation
**Status:** PR-B7 (`REALIGNMENT/04-phase-B-live-zone.md`) introduced two
persistent CCR backends. The single-`--workers` recommendation no longer
applies once you select a persistent backend.
**Status:** two persistent CCR backends are available. The single-`--workers`
recommendation no longer applies once you select a persistent backend.
### Backend selection

132
docs/auth-modes.md
View file

@ -1,132 +0,0 @@
# Auth Modes
Headroom classifies every inbound request into one of three **auth modes** at request entry. The mode drives every downstream compression, cache, and header policy decision. Detection is a pure function of HTTP headers — no I/O, no allocation beyond a single `to_lowercase` of the User-Agent, runs in <10us per call.
The classifier ships in two equivalent implementations:
- **Rust:** `crates/headroom-core/src/auth_mode.rs` (`classify`)
- **Python:** `headroom/proxy/auth_mode.py` (`classify_auth_mode`)
Both are byte-for-byte identical on every header set covered by the parity test suite (`crates/headroom-core/tests/auth_mode.rs` + `tests/test_auth_mode.py`).
## The three modes
### `Payg` — pay-as-you-go API key
| Signal | Examples |
|---|---|
| `Authorization: Bearer sk-ant-api*` | Anthropic PAYG key |
| `Authorization: Bearer sk-*` (excluding `sk-ant-oat-`) | OpenAI PAYG key |
| `x-api-key: ...` | Anthropic API key style |
| `x-goog-api-key: ...` | Google Gemini key |
**Compression policy:** aggressive. The caller pays per token; compression directly saves them money. Cache hit-rate matters financially (Anthropic 1.25-2× cache write, 0.10× cache read). Live-zone compression, CCR, type-aware compressors all turn on. This is the OSS default.
### `OAuth` — OAuth bearer / IAM-signed
| Signal | Examples |
|---|---|
| `Authorization: Bearer sk-ant-oat-*` | Claude Pro / Max OAuth |
| `Authorization: Bearer <jwt>` (3-segment) | Codex / Cursor / Copilot OAuth |
| `Authorization: AWS4-HMAC-SHA256 ...` | Bedrock SigV4 |
| Any other non-`Bearer` Authorization scheme | Vertex ADC, custom proxies |
**Compression policy:** passthrough-prefer. Per-token cost is opaque (subscription) or zero from the caller's POV (IAM-bound usage); compression value is **extending effective context within rate-limit / quota windows**, not saving money. Cache safety is paramount because OAuth scopes pin to `(account, model, session)` and beta-header drift can break OAuth-issued scopes. **No auto-`cache_control`, no auto-`prompt_cache_key`, no lossy compressors.** Lossless-only.
### `Subscription` — UX-bound CLI / IDE
| Signal | UA prefix |
|---|---|
| Claude Code | `claude-code/` |
| Claude CLI | `claude-cli/` |
| Codex CLI | `codex-cli/` |
| Cursor | `cursor/` |
| Claude VS Code | `claude-vscode/` |
| GitHub Copilot | `github-copilot/` |
| Anthropic CLI | `anthropic-cli/` |
| Antigravity | `antigravity/` |
**Compression policy:** stealth. Provider rate-limits by request count; programmatic-fingerprint detection means Headroom MUST look like the upstream agent. Same compression policy as `OAuth` **plus**:
- Preserve `accept-encoding` byte-for-byte.
- Never inject `X-Headroom-*` headers on upstream-bound requests.
- Never mutate `User-Agent`.
- Skip `X-Forwarded-*` headers on upstream-bound requests (see Phase F PR-F4).
The Subscription UA wins over any bearer token shape — a Claude Code session that happens to carry a `sk-ant-oat-*` token is still a subscription client, never OAuth.
## Detection signals — full decision order
The classifier evaluates these in order; the **first** matching rule wins.
1. **User-Agent contains a `SUBSCRIPTION_UA_PREFIXES` entry** → `Subscription`.
2. **`Authorization: Bearer sk-ant-oat-*`** → `OAuth`.
3. **`Authorization: Bearer sk-ant-api*` or `Bearer sk-*`** → `Payg`.
4. **`Authorization: Bearer <jwt>`** (3 dot-separated segments) → `OAuth`.
5. **`Authorization` present but not `Bearer ...`** (e.g., `AWS4-HMAC-SHA256`) → `OAuth`.
6. **`x-api-key` present** → `Payg`.
7. **`x-goog-api-key` present** → `Payg`.
8. **Default** → `Payg`.
The subscription-UA check is intentionally first because the same OAuth token shape appears in both Claude Pro (web) and Claude Code (CLI), and only the User-Agent disambiguates them.
The "default to PAYG" rule is intentionally conservative: misclassifying a non-PAYG client as PAYG over-compresses, which only costs us a re-run; under-compressing a PAYG client leaves money on the table, which is worse for the OSS-default user.
## How to extend
### Adding a new subscription CLI
Add the UA prefix to **both** files — they must agree:
- `crates/headroom-core/src/auth_mode.rs` → `SUBSCRIPTION_UA_PREFIXES`
- `headroom/proxy/auth_mode.py` → `SUBSCRIPTION_UA_PREFIXES`
Then add a parametrised parity test case in **both**:
- `crates/headroom-core/tests/auth_mode.rs` → add a `#[test] fn ..._ua_classified_subscription`.
- `tests/test_auth_mode.py` → covered automatically by the existing `test_every_subscription_prefix_classified_subscription` parametrised test.
### Adding a new OAuth token shape
Add the prefix check **before** the `sk-` PAYG branch in both `classify` and `classify_auth_mode`. Order matters: any token shape that's a strict prefix of `sk-` must be checked first.
### Making the prefix list user-configurable
The list lives in a `const` so a future Phase F follow-up PR can swap it for a configurable source (env var, TOML config, CLI flag) without touching the function body. The recommended path:
1. Read the list from `Config::subscription_ua_prefixes` (Rust) / `headroom.config.Settings.subscription_ua_prefixes` (Python).
2. Default to the current static list if unset.
3. Pass the list as a parameter to `classify` / `classify_auth_mode`.
The classifier itself is a pure function — adding a parameter is a localized change.
## Performance
| Path | Per-call latency (p50) |
|---|---|
| Rust empty headers | ~20 ns |
| Rust PAYG (Bearer prefix match) | ~50 ns |
| Rust Subscription (UA lowercase + scan) | ~600 ns |
| Python empty headers | ~3 us |
| Python Subscription | ~12 us |
All paths are well under the <10us Rust budget and the <100us Python budget asserted by the test suite.
## Where the auth-mode value flows
After classification, the value is stored on the request object so downstream code reads it without re-classifying:
- **Rust:** `req.extensions_mut().insert(auth_mode)`. Read with `req.extensions().get::<headroom_core::auth_mode::AuthMode>()`.
- **Python:** `request.state.auth_mode`. Read with `request.state.auth_mode`.
A structured log line (`event = auth_mode_classified`) fires once per request at request entry; the value is logged as `auth_mode = "payg" | "oauth" | "subscription"`.
## Phase F roadmap
PR-F1 (this PR) lands the helper. The rest of Phase F wires it into specific policy gates:
- **PR-F2:** per-mode compression policy gates (auto-`cache_control`, `prompt_cache_key`, lossy compressors).
- **PR-F3:** TOIN per-tenant aggregation key includes `(auth_mode, model_family, structure_hash)`.
- **PR-F4:** `X-Forwarded-*` skipped on Subscription mode.
See `REALIGNMENT/08-phase-F-auth-mode.md` for the full phase plan.

218
docs/bedrock.md
View file

@ -1,218 +0,0 @@
# AWS Bedrock — Operator Guide
Headroom's Rust proxy ships a native AWS Bedrock InvokeModel surface. After Phase D (PRs D1–D3), Anthropic-on-Bedrock requests are signed, compressed, and observed by the proxy directly — no LiteLLM Python shim on the request path.
This document covers how to deploy the Bedrock-native surface, how compression policy is applied, and how to read the Prometheus metrics the proxy exports.
## What's in scope
| Capability | Status |
|---|---|
| `POST /model/{model}/invoke` | PR-D1 — native Rust handler |
| `POST /model/{model}/converse` | PR-D1 — same handler (Bedrock accepts both paths for the Anthropic envelope) |
| `POST /model/{model}/invoke-with-response-stream` | PR-D2 — binary EventStream parsed and translated to SSE |
| AWS SigV4 signing (post-compression) | PR-D1 |
| `AuthMode::OAuth` classification | PR-D3 — Bedrock IAM is OAuth-equivalent under the policy matrix |
| Per-model + per-region Prometheus metrics | PR-D3 — exposed at `GET /metrics` |
| OAuth compression policy gates (no auto cache_control, lossless-only) | Phase F PR-F2/F3 (gates the marker D3 wires) |
## Running the proxy
The native surface lives in the `headroom-proxy` binary, which ships in the published
container images (every `proxy`-extra tag) at `/usr/local/bin/headroom-proxy`. You can run
it directly from any published image — no separate build:
```sh
docker run --rm -p 8787:8787 \
-v "$HOME/.aws:/home/nonroot/.aws:ro" \
-e HEADROOM_PROXY_AWS_PROFILE=my-profile \
--entrypoint headroom-proxy \
ghcr.io/chopratejas/headroom:latest \
--listen 0.0.0.0:8787 \
--upstream https://bedrock-runtime.us-east-1.amazonaws.com \
--bedrock-region us-east-1
```
The published images default to the `nonroot` user (home `/home/nonroot`), so AWS
credentials are mounted at `/home/nonroot/.aws` — that is where the SDK looks for
`~/.aws`. For a root-based image (`RUNTIME_USER=root` build), mount to `/root/.aws`
instead, or pass `--user root`.
Then point the AWS SDK / CLI at the proxy:
```sh
AWS_ENDPOINT_URL_BEDROCK_RUNTIME=http://localhost:8787 \
aws bedrock-runtime invoke-model --model-id anthropic.claude-3-haiku-20240307-v1:0 ...
```
The proxy can also drop in front of the Python proxy (`--upstream http://127.0.0.1:8788`)
so non-Bedrock traffic is forwarded while Bedrock requests are signed + compressed
natively. The default `--enable-bedrock-native=true` mounts the Bedrock routes; everything
else is passed through to `--upstream`.
## AWS credential configuration
The proxy uses the [aws-config default credential chain](https://docs.aws.amazon.com/sdkref/latest/guide/standardized-credentials.html), resolved once at startup.
The chain searches in this order, stopping at the first source that yields valid credentials:
1. **Environment variables** — `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, optional `AWS_SESSION_TOKEN`. Useful for ECS task roles that inject creds via env or for `aws sts assume-role` shells.
2. **Shared credentials file** — `~/.aws/credentials`. Profile selected by `--aws-profile` (or `AWS_PROFILE`). Falls back to `[default]`.
3. **IAM instance profile / IMDS** — when running on EC2.
4. **ECS task role / EKS pod identity** — when running on the AWS-managed compute platforms.
5. **AWS SSO** — `~/.aws/sso/cache/...` when `aws sso login` has been run.
If the chain does NOT resolve any credentials at startup, the proxy logs `event=bedrock_credentials_unavailable` at WARN and continues to start. Bedrock invoke routes will then return `500` with `event=bedrock_credentials_missing` per request — there is **no silent fallback to unsigned requests**, by design.
### Required IAM permissions
The proxy needs:
- `bedrock:InvokeModel` for non-streaming
- `bedrock:InvokeModelWithResponseStream` for streaming
Scope these to the specific model ARNs you intend to use. Example IAM policy snippet:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": [
"arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0",
"arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
]
}
]
}
```
## Region configuration
```sh
headroom-proxy \
--upstream http://unused-when-bedrock-only \
--bedrock-region us-east-1
```
| Flag | Env var | Default | Notes |
|---|---|---|---|
| `--bedrock-region` | `HEADROOM_PROXY_BEDROCK_REGION` (or `AWS_REGION`) | `us-east-1` | Drives both the SigV4 region and the derived endpoint hostname. |
| `--bedrock-endpoint` | `HEADROOM_PROXY_BEDROCK_ENDPOINT` | derived from region | Override for FIPS endpoints (`bedrock-runtime-fips.{region}.amazonaws.com`), VPC endpoints, or local mock servers. |
| `--aws-profile` | `HEADROOM_PROXY_AWS_PROFILE` | unset | Selects the named profile from the shared credentials file. |
| `--enable-bedrock-native` | `HEADROOM_PROXY_ENABLE_BEDROCK_NATIVE` | `true` | Set to `false` to mount no Bedrock routes at all (Bedrock requests will then fall through to the catch-all and fail without SigV4). |
## Supported model IDs
The proxy classifies model IDs by **literal vendor match** — no regexes. It strips a known cross-region inference-profile geo prefix (`eu.`, `us.`, `apac.`, `global.`) if present, then takes the leading dot-segment as the vendor. A model is treated as Anthropic-shape when that canonical vendor is `anthropic` — so both the bare `anthropic.…` foundation models and the geo-prefixed inference profiles (`eu.anthropic.…`, `us.anthropic.…`, `apac.anthropic.…`, `global.anthropic.…`) qualify. For those, the live-zone compression dispatcher runs over the body, the envelope is re-emitted with `anthropic_version` preserved as the first key, and the request is signed with SigV4.
Examples that hit the Anthropic compression path:
- `anthropic.claude-3-haiku-20240307-v1:0` (foundation model)
- `anthropic.claude-3-5-sonnet-20241022-v2:0`
- `eu.anthropic.claude-haiku-4-5-20251001-v1:0` (EU cross-region inference profile)
- `us.anthropic.claude-3-5-sonnet-20241022-v2:0` (US inference profile)
- `global.anthropic.claude-haiku-4-5-20251001-v1:0`
Other Bedrock vendors (`amazon.titan-...`, `meta.llama3-...`, `cohere.command-...`, `ai21.j2-...`, `stability.stable-diffusion-...`, and their geo-prefixed inference profiles such as `eu.amazon.nova-...`) are signed and forwarded **without compression** — the proxy does not yet understand their body shapes and would risk corrupting them. These model IDs log `event=bedrock_compression_skipped, reason=non_anthropic_vendor` per request. Full Anthropic envelopes only.
The contract: **any new model ID that AWS adds under the `anthropic.` vendor (as a bare prefix or behind a cross-region geo prefix) automatically picks up the full compression + signing pipeline.** No code change in the proxy is needed for new versions of Claude on Bedrock.
## Compression behaviour
Bedrock requests are subject to the **same** live-zone compression rules as direct Anthropic (`/v1/messages`):
- Only the live-zone messages (latest user turn, latest tool/output blocks) are eligible for compression.
- The cache hot zone (older messages, system prompt, tools list) is byte-faithful passthrough.
- The dispatcher only mutates body bytes when at least one block compressed. The byte-equality invariant for unchanged blocks is enforced at `debug_assert!` granularity.
### OAuth policy (PR-D3 → PR-F2/F3)
The Bedrock auth-mode middleware classifies every Bedrock request as `AuthMode::OAuth`. Even when the inbound request has no Authorization header (the common case where the AWS SDK signs after our proxy), the middleware **coerces** to OAuth and emits `event=bedrock_auth_mode_unexpected` at WARN if F1's classifier disagreed — so the divergence is loud, not silent.
Under the OAuth policy matrix (see `docs/auth-modes.md`):
- **No auto-`cache_control` injection.** OAuth subscriptions pin the cache scope to `(account, model, session)`; auto-injecting markers can void cache hits.
- **No auto-`prompt_cache_key`.** Same reasoning.
- **Lossless-only compressors.** Lossy compressors (text rewriting, summarisation) are gated off for OAuth.
PR-D3 lands the classification + the resulting `AuthMode` in `request.extensions()`. PR-F2 and PR-F3 wire the actual policy gates that read it. Until those PRs land, the Bedrock route uses the existing dispatcher (which is a no-op in `compression_mode=off`); the OAuth contract above is the documented forward direction.
### Cache safety
The bytes signed by SigV4 are exactly the bytes Bedrock receives — the signer hashes the post-compression body. There is no "sign before compress" shortcut that would produce a signature mismatched to the wire payload. Compression mutates the body once, then the signer runs once, then the bytes are forwarded once.
## Prometheus metrics
The proxy exposes a `GET /metrics` endpoint that serves the standard Prometheus text-format scrape. Three Bedrock-specific metric families are exported:
| Metric | Type | Labels | Source |
|---|---|---|---|
| `bedrock_invoke_count_total` | Counter | `model`, `region`, `auth_mode` | One increment per `/model/.../invoke` (and `/converse` and `/invoke-with-response-stream`) request. |
| `bedrock_invoke_latency_seconds` | Histogram | `model`, `region` | Observed at request completion (success or failure). |
| `bedrock_eventstream_message_count_total` | Counter | `model`, `region`, `event_type` | One increment per parsed binary EventStream message in the streaming path. `event_type` is the `:event-type` header (`chunk`, `metadata`, `internalServerException`, etc.). |
All labels are bounded by infrastructure config (`region` from `--bedrock-region`, `auth_mode` from the 3-variant enum) or by the path parameter (`model`, supplied by the axum extractor — never by user-controlled body bytes). Cardinality is bounded by deployment fan-out, not by traffic volume.
### Sample PromQL queries
**p99 latency by model:**
```promql
histogram_quantile(
0.99,
sum by (model, le) (rate(bedrock_invoke_latency_seconds_bucket[5m]))
)
```
**Request rate by region (RPS):**
```promql
sum by (region) (rate(bedrock_invoke_count_total[1m]))
```
**EventStream message rate by event type (debugging the streaming path):**
```promql
sum by (event_type) (rate(bedrock_eventstream_message_count_total[1m]))
```
**Error breakdown by HTTP status (cross-references the structured logs `event=bedrock_upstream_error`):**
```promql
sum by (model) (rate(bedrock_invoke_count_total{auth_mode="oauth"}[5m]))
/ sum by (model) (rate(bedrock_invoke_latency_seconds_count[5m]))
```
(The denominator is the total observed latency samples — useful for sanity-checking that every counted invoke also got a histogram observation. They should be equal.)
### Structured-log correlation
Every metric increment in the Bedrock path is paired with a `tracing::debug!` log line carrying:
- `event = "metric_recorded"`
- `metric = "bedrock_invoke_count_total" | "bedrock_invoke_latency_seconds" | "bedrock_eventstream_message_count_total"`
- the same labels as the metric
Enable with `RUST_LOG=headroom_proxy::observability=debug` for incident correlation. In normal operation keep this at the default `info` level — debug volume per request is bounded by the same cardinality the metric uses.
## Live cloud validation
The PR-D1, D2, D3 implementations are exercised end-to-end against a wiremock upstream (`crates/headroom-proxy/tests/integration_bedrock_*.rs`). The wiremock-based tests are the canonical correctness gate.
A real Bedrock smoke test (`aws bedrock-runtime invoke-model ...` through the proxy) requires `bedrock:InvokeModel` permissions in the developer's AWS account. Set the proxy upstream to the proxy URL (`http://localhost:8787`) via the AWS SDK's `AWS_ENDPOINT_URL_BEDROCK_RUNTIME` env var:
```sh
AWS_ENDPOINT_URL_BEDROCK_RUNTIME=http://localhost:8787 \
aws bedrock-runtime invoke-model \
--model-id anthropic.claude-3-haiku-20240307-v1:0 \
--body '{"anthropic_version":"bedrock-2023-05-31","max_tokens":32,"messages":[{"role":"user","content":"hi"}]}' \
/tmp/out.json
```
If the SDK signs the request before sending to the proxy, the proxy will see a SigV4 `Authorization` header and classify as OAuth via the standard rule. If the SDK is configured to sign downstream of the proxy (some IAM-instance-profile setups), the proxy still classifies as OAuth via the middleware's coerce-and-log fallback.
## Rollback
Set `--enable-bedrock-native=false` to unmount all Bedrock routes; the catch-all proxy then forwards Bedrock requests unchanged to `--upstream`. This is an emergency rollback only — without SigV4 re-signing, the catch-all path will fail closed unless the upstream is itself a Bedrock-aware proxy (e.g., the Python LiteLLM converter on a different port).

163
docs/claude-code-vertex-headroom.md
View file

@ -1,163 +0,0 @@
# Claude Code + Google Vertex AI, with Headroom compression
*Validated end-to-end on 2026-06-19 (Claude Code 2.1.181, Headroom 0.27.0).*
This is the **working, tested** way to run **Claude Code** against **Claude models on
Google Vertex AI** with **Headroom compressing the context** in the middle.
## TL;DR
Run Claude Code in **normal Anthropic mode** (NOT Vertex mode) pointed at a local
Headroom proxy, and let **Headroom** be the thing that talks to Vertex:
```
Claude Code ──ANTHROPIC_BASE_URL──▶ Headroom proxy ──LiteLLM (vertex_ai)──▶ Vertex AI
(normal mode) (plain http) (compresses) (your GCP ADC) (Claude)
```
Two non-obvious requirements make the difference between "works" and "silently does nothing":
1. **`pip install "google-cloud-aiplatform>=1.38"`** into the proxy's environment —
LiteLLM's `vertex_ai` provider needs it, or every request 500s with
`No module named 'vertexai'`.
2. **Start the proxy with `--code-aware`** — coding sessions are mostly *source code*,
which routes to the AST/code-aware compressor. It is **disabled by default**, so
without this flag compression no-ops on code and you see `tokens_saved: 0`.
## Why not "just point Claude Code's Vertex URL at Headroom"?
That approach (Vertex mode + `ANTHROPIC_VERTEX_BASE_URL`=proxy) **does not work** with
Claude Code today. In Vertex mode Claude Code runs a **client-side `probeVertexModel`
check before any request**. When `ANTHROPIC_VERTEX_BASE_URL` points at a non-Google
host, that probe fails *instantly* (no network call is made) with a misleading
`"The model … is not available on your vertex deployment"`, and the proxy never
receives a byte. This is a Claude Code limitation, not a Headroom bug. The native
`:rawPredict` passthrough in Headroom is correct and compresses (verified by direct
curl) — but the client won't route to it. So we use the Anthropic-mode path below.
## Prerequisites
- **Google Cloud auth (ADC).** Run once: `gcloud auth application-default login`
(and `gcloud config set project <PROJECT>`). The proxy uses ADC to call Vertex; no
API key is held by Headroom. A service-account JSON via
`GOOGLE_APPLICATION_CREDENTIALS` works too.
- **Vertex Claude quota** for the model + location you intend to use. Confirm with a
direct call before involving Headroom:
```bash
ACCESS_TOKEN="$(gcloud auth application-default print-access-token)"
curl -sS -X POST \
-H "Authorization: Bearer ${ACCESS_TOKEN}" -H "Content-Type: application/json" \
"https://aiplatform.googleapis.com/v1/projects/<PROJECT>/locations/global/publishers/anthropic/models/claude-sonnet-4-6:rawPredict" \
-d '{"anthropic_version":"vertex-2023-10-16","max_tokens":20,"messages":[{"role":"user","content":"hi"}]}'
```
HTTP 200 → good. 429 → model exists but no quota in that location. 404 → model not
enabled in that project/location.
- **Headroom ML extra** for compression: `pip install "google-cloud-aiplatform>=1.38"`
plus the Kompress ML stack (`torch`, `transformers`, `onnxruntime` — the
`headroom-ai[ml]` extra). The `kompress-v2-base` model downloads from Hugging Face
on first use.
## Terminal 1 — start the Headroom proxy (Vertex backend)
```bash
cd /path/to/headroom
source .venv/bin/activate
export VERTEXAI_PROJECT=<YOUR_GCP_PROJECT>
export GOOGLE_CLOUD_PROJECT=<YOUR_GCP_PROJECT>
export VERTEXAI_LOCATION=global # match where your quota lives
headroom proxy --port 8787 \
--backend litellm-vertex_ai \ # NOTE: the _ai suffix is required
--region global \ # becomes LiteLLM vertex_location
--code-aware # REQUIRED for code compression
```
On startup, confirm components loaded: `curl -s localhost:8787/debug/warmup` should
show `kompress: loaded`, `code_aware: loaded`, `tree_sitter: loaded`,
`smart_crusher: loaded`.
## Terminal 2 — run Claude Code (normal Anthropic mode) against the proxy
```bash
cd /path/to/your/project
export ANTHROPIC_BASE_URL=http://127.0.0.1:8787
export ANTHROPIC_API_KEY=sk-placeholder-not-used # Claude Code needs *a* key to start
export ANTHROPIC_MODEL=claude-sonnet-4-6 # sent to Headroom, mapped to vertex_ai/claude-sonnet-4-6
export ANTHROPIC_SMALL_FAST_MODEL=claude-sonnet-4-6 # pin background model to one you have quota for
# Do NOT set CLAUDE_CODE_USE_VERTEX or ANTHROPIC_VERTEX_BASE_URL — those put Claude
# Code into Vertex mode and trigger the broken probe described above.
claude
```
Claude Code now talks plain Anthropic `/v1/messages` to Headroom; Headroom compresses
and forwards to Vertex via LiteLLM, then translates the answer back.
## Verify compression is happening
- Dashboard: <http://localhost:8787/dashboard> — "tokens saved" climbs as you work.
- `curl -s localhost:8787/stats` → `tokens.saved`, and `request_logs[].transforms_applied`
(look for `router:tool_result:mixed`, `kompress:*`, `code_aware:*`).
- Savings appear on **large tool outputs** (Bash/Grep/web fetches) and accumulate over
turns. Note: **`Read`/`Glob`/`Grep`/`Write`/`Edit` outputs are protected from the
ContentRouter by default** (safest for coding agents); stale `Read`s are handled
separately by the Read-lifecycle system. So the biggest wins come from non-excluded
large outputs and multi-turn sessions, not single one-shot reads.
## What `--code-aware` does — and what it never touches
**What it does.** Code-Aware is an **AST (tree-sitter) compressor for source code that
passes through the proxy inside a request**. It parses the code, **keeps the structure
that matters** — imports, function/class signatures, type annotations, error handlers —
and **shrinks the less-important function bodies**, always emitting **syntactically
valid code** (the output still parses). Languages: Python, JS, TS (tier 1); Go, Rust,
Java, C, C++ (tier 2). The original is **stored for retrieval (CCR, ~5-minute TTL)**, so
if the model needs the exact bytes it can pull them back via the `headroom_retrieve`
tool — compression is reversible, not destructive.
**What it does NOT touch:**
- **Your files on disk.** Headroom is a network proxy: it only rewrites the *request
body* in flight on the way to Vertex. It never reads, writes, or modifies any local
file. Code-Aware operates on text that is *already inside the API request*, not on
your repository.
- **Claude Code's `Read` tool output.** `Read`, `Glob`, `Grep`, `Write`, and `Edit`
are in `DEFAULT_EXCLUDE_TOOLS` and are **protected from the ContentRouter by default**
(`protect_recent_reads_fraction = 0.0` ⇒ protect-all). So when Claude Code opens a
file the normal way, **the model sees it verbatim** — Code-Aware does not alter it.
(Stale `Read`s — files you later edit — are handled separately and reversibly by the
Read-lifecycle, replacing the superseded copy with a retrievable marker.)
**Where it actually applies:** code that reaches the model through *other* channels —
most commonly **`Bash` output that prints code** (`cat file.py`, `sed`, `nl`, build
logs with snippets) or large code in results from non-excluded/custom tools. That is
the content that gets AST-compressed. In the validation run, the ~22% savings came
exactly from two `Bash` commands that dumped source files — not from `Read`.
**Net:** with the default config your real file reads and edits go to the model
untouched; Code-Aware only trims bulky *incidental* code (shell dumps, logs, pasted
snippets) and keeps the originals retrievable. Omit `--code-aware` if you want zero
code transformation at all (you lose code compression but keep everything else).
## Model-string notes (Vertex)
- Headroom maps clean ids to Vertex publisher ids: `claude-sonnet-4-6` →
`vertex_ai/claude-sonnet-4-6` (see `headroom/backends/litellm.py`). Newer models use
the bare alias; older ones are date-pinned (e.g. `claude-sonnet-4-5@20250929`).
- `--region global` works (LiteLLM targets the `aiplatform.googleapis.com` global
endpoint). Use a specific region (`us-east5`, `europe-west1`, …) only if that's where
your quota is.
## Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| `500 … No module named 'vertexai'` | LiteLLM vertex provider dep missing | `pip install "google-cloud-aiplatform>=1.38"`, restart proxy |
| `tokens_saved: 0` on code | Code-Aware disabled | start proxy with `--code-aware` |
| `tokens_saved: 0` everywhere, `/debug/warmup` shows `kompress: not installed` | ML extra missing | install `headroom-ai[ml]` (torch/transformers/onnxruntime) |
| `"model … not available on your vertex deployment"`, proxy logs nothing | Claude Code is in Vertex mode (probe) | unset `CLAUDE_CODE_USE_VERTEX` / `ANTHROPIC_VERTEX_BASE_URL`; use Anthropic mode above |
| `429 RESOURCE_EXHAUSTED` | no quota in that location | switch `--region`/`VERTEXAI_LOCATION` to where your quota is |
| `404 Publisher Model not found` | model not enabled in project/location | enable it in Vertex Model Garden / request quota |

2
docs/content/docs/configuration.mdx
View file

@ -233,7 +233,7 @@ headroom proxy --learn --min-evidence 3
| `HEADROOM_STATELESS` | Set to `true` to disable filesystem writes | `false` |
| `HEADROOM_MODEL_LIMITS` | Custom model config (JSON string or file path) | -- |
| `HEADROOM_BASE_URL` | Base URL of the Headroom proxy (TypeScript SDK) | `http://localhost:8787` |
| `HEADROOM_API_KEY` | API key for Headroom Cloud authentication | -- |
| `HEADROOM_API_KEY` | Optional API key for authenticated Headroom endpoints (TypeScript SDK) | -- |
| `HEADROOM_CONFIG_DIR` | Canonical config (read-mostly) root. Derives `models.json` and per-plugin config paths when set. | `~/.headroom/config` |
| `HEADROOM_WORKSPACE_DIR` | Canonical workspace (read-write state) root. Derives savings, memory DB, logs, TOIN, subscription state, and more when set. | `~/.headroom` |
| `HEADROOM_SAVINGS_PATH` | Override persistent savings file location. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |

247
docs/cortex-code.md
View file

@ -1,247 +0,0 @@
# Cortex Code + Headroom — Integration Guide
Headroom compresses the context Cortex Code (CoCo) sends to `claude-sonnet-4-6`
before it reaches the Snowflake Cortex inference endpoint. The result is 60–65%
fewer prompt tokens billed, with the same answers.
## Benchmark (measured, not estimated)
Token counts are from `usage.prompt_tokens` in the actual Snowflake Cortex API
response — not headroom's local estimate.
| Payload | Before | After | Saved |
|---|---:|---:|---:|
| Full CoCo session (tables + dbt + search) | 17,827 | 6,781 | **62%** |
| `INFORMATION_SCHEMA` tables (79 rows) | 10,161 | 3,979 | **61%** |
| `dbt` run-results (40 models) | 4,968 | 1,927 | **61%** |
| Cortex Search results (15 docs) | 2,764 | 956 | **65%** |
At 1,000 calls/day: **~$16/day saved**, **~$6,000/year saved**.
> Numbers above are per-call averages across the four benchmark payloads.
> The full-session payload alone saves ~$33/1,000 calls/day.
## How it works
```
CoCo (cortex CLI)
│ OPENAI_BASE_URL=http://127.0.0.1:8787/v1
▼
Headroom proxy (local, your data never leaves your machine)
│ SmartCrusher compresses JSON context
│ CacheAligner stabilises KV-cache prefixes
▼
Snowflake Cortex /api/v2/cortex/inference:complete
│ claude-sonnet-4-6
▼
Response (same answer, fewer billed tokens)
```
Headroom's **SmartCrusher** targets the large JSON blobs that CoCo produces:
`INFORMATION_SCHEMA` query results, `dbt` run-results, Cortex Search payloads,
and schema introspection output. These are highly repetitive structures that
compress 60–99% without any loss of information.
## Quick start
```bash
pip install "headroom-ai[all]"
headroom wrap cortex-code # starts proxy + prints the env var to set
```
`headroom wrap cortex-code` starts the local proxy and prints:
```
Headroom proxy is running. Configure Cortex Code (CoCo):
Set the following environment variable before launching cortex:
OPENAI_BASE_URL=http://127.0.0.1:8787/v1
```
Then in a new shell:
```bash
OPENAI_BASE_URL=http://127.0.0.1:8787/v1 cortex
```
Or add it to your shell profile so it applies to every CoCo session:
```bash
# ~/.zshrc or ~/.bashrc
export OPENAI_BASE_URL=http://127.0.0.1:8787/v1
```
## Manual proxy startup
If you prefer to manage the proxy lifecycle yourself:
```bash
# Terminal 1 — start the proxy
headroom proxy --port 8787
# Terminal 2 — launch CoCo through the proxy
OPENAI_BASE_URL=http://127.0.0.1:8787/v1 cortex
```
Point the proxy at your Snowflake Cortex endpoint explicitly with
`--openai-api-url`:
```bash
headroom proxy \
--port 8787 \
--openai-api-url https://<account>.snowflakecomputing.com
```
## Library mode (inline, no proxy)
If you are building an application on top of the Snowflake Cortex REST API
and want to compress context before every call:
```python
from headroom import compress
import json, urllib.request
# Build your messages (large JSON tool results, search results, etc.)
messages = [
{"role": "system", "content": json.dumps(cortex_search_results, indent=2)},
{"role": "assistant", "content": "I have reviewed the context."},
{"role": "user", "content": "What is failing and how do I fix it?"},
]
# Compress before sending — local, no API call, no data leaves your machine
result = compress(messages, model="claude-sonnet-4-6")
print(f"Saved {result.tokens_saved} tokens ({result.tokens_saved / result.tokens_before:.0%})")
# Send compressed messages to Snowflake Cortex REST API
response = call_cortex(result.messages, token=sf_token)
```
### What to put in the system message
The Snowflake Cortex REST API supports `system`, `user`, and `assistant` roles.
For maximum compression, inject large retrieved context into `system`:
```python
# Query results, search results, schema — these compress 60–99%
system_context = {
"tables": json.loads(show_tables_result),
"search_results": cortex_search_results,
"schema": describe_table_result,
"dbt_results": dbt_run_results_json,
}
messages = [
{"role": "system", "content": json.dumps(system_context, indent=2)},
{"role": "assistant", "content": "Context loaded."},
{"role": "user", "content": user_question},
]
result = compress(messages, model="claude-sonnet-4-6")
```
## Authentication
Cortex Code authenticates using your Snowflake connection. Headroom sits
between CoCo and the Cortex endpoint and forwards auth headers unchanged —
it never reads or stores your credentials.
If you use `snowflake-connector-python` directly, keep the connection open
while making API calls; closing it invalidates the OAuth session token:
```python
import snowflake.connector, sys, io
# Suppress connector's browser-auth console output
_s = sys.stdout; sys.stdout = io.StringIO()
conn = snowflake.connector.connect(connection_name="my_connection")
token = conn.rest.token
sys.stdout = _s
# Make all API calls while conn is open, then:
conn.close()
```
## Per-project savings attribution
Use `headroom wrap cortex-code --project <name>` to attribute savings to a
specific project in the headroom dashboard:
```bash
headroom wrap cortex-code --project my-dbt-project
```
The dashboard at `http://127.0.0.1:8787` shows per-project token and cost
savings across all your CoCo sessions.
## Verifying savings
After a CoCo session, check what headroom saved:
```bash
headroom perf # token savings for the last session
headroom perf --hours 24 # last 24 hours
```
Or run the included end-to-end benchmark against your own Snowflake account:
```bash
# Measures real usage.prompt_tokens from claude-sonnet-4-6
python3 tests/e2e_cortex_savings.py
```
## Testing
Unit tests for the provider slice:
```bash
uv run --with pytest pytest tests/test_provider_cortex_code.py -v
```
Compression benchmark (no API key needed — local only):
```bash
uv run --with pytest pytest tests/test_cortex_code_compression.py -v -s
```
Real E2E test against Snowflake Cortex (requires Snowflake connection):
```bash
python3 tests/e2e_cortex_savings.py
```
## How the provider is implemented
Cortex Code routes through headroom's OpenAI-compatible pipeline. The provider
slice lives in `headroom/providers/cortex_code/`:
| File | Purpose |
|---|---|
| `runtime.py` | `proxy_base_url(port)` → `http://127.0.0.1:{port}/v1`; `default_api_url()` reads `SNOWFLAKE_HOST` / `SNOWFLAKE_ACCOUNT` |
| `install.py` | `build_install_env()` → `{"OPENAI_BASE_URL": ...}`; `render_setup_lines()` |
| `__init__.py` | Public exports |
Registered in `headroom/providers/install_registry.py` under the key
`"cortex-code"`, which is what `headroom wrap cortex-code` resolves to.
## Limitations
- The Snowflake Cortex REST API at `/api/v2/cortex/inference:complete` does not
support `role: "tool"` messages or OpenAI-style `tool_calls`. Use the
`system` message to inject large retrieved context (where SmartCrusher
achieves the highest compression ratios).
- The headroom proxy cannot rewrite the Cortex inference path
(`/api/v2/cortex/inference:complete` ≠ `/v1/chat/completions`), so
**library mode** (`from headroom import compress`) is required when calling
the Cortex REST API directly. The proxy mode works for any
OpenAI-compatible client that points at Cortex via a gateway that exposes
`/v1/chat/completions`.
- Output-token reduction (`HEADROOM_OUTPUT_SHAPER=1`) is supported in proxy
mode. In library mode only input compression applies.
## See also
- [Architecture](ARCHITECTURE.md)
- [Proxy configuration](proxy.md)
- [CCR — reversible compression](ccr.md)
- [Claude Code + Vertex](claude-code-vertex-headroom.md)
- [Benchmarks](benchmarks.md)

145
docs/output-token-reduction-guide.md
View file

@ -1,145 +0,0 @@
# Output Token Reduction — User Guide
A plain-English guide to cutting the tokens the model **writes back**.
## Why this exists
Headroom normally shrinks the prompt you **send**. This feature shrinks what the
model **returns**. That matters because:
- Output tokens cost **5× more** than input on Opus-class models.
- A lot of model output is waste: "Great, let me help with that…" intros,
re-printing code you already showed it, restating tool results, and long
internal "thinking" even on trivial steps.
You don't change any code. It runs in the Headroom proxy.
## Turn it on
```bash
export HEADROOM_OUTPUT_SHAPER=1 # off by default
headroom proxy --port 8787
```
> **If a proxy is already running** (e.g. `headroom wrap claude` attaches to one
> on port 8787 instead of starting a fresh one), it reads this switch from the
> environment it was launched with — so exporting it afterwards wouldn't reach
> it. `headroom wrap` handles this for you: it hot-syncs your current output
> settings to the running proxy (loopback `POST /admin/runtime-env`), applied
> immediately with no restart. Set the variables before you run `wrap`. Because
> one proxy is shared by every session attached to it, these settings are global
> — the most recent explicit value wins.
That's it. Two things now happen on every request:
1. **Verbosity steering** — a short "be terse, don't restate context" instruction
is added to the **end** of the system prompt. (The end, so your prompt cache
still works.)
2. **Effort routing** — if a turn is just the model continuing after a tool ran
(e.g. it read a file and there were no errors), Headroom turns the model's
"thinking effort" down for that one turn. Real questions and error-handling
turns keep full effort.
## The verbosity dial (levels 0–4)
| Level | What the model is told | Good for |
|------:|------------------------|----------|
| 0 | (off) | disable steering |
| 1 | Skip the intro/outro chit-chat | people who read everything |
| 2 | Also: don't restate code/output already on screen | **default** — safe |
| 3 | Also: conclusions only, skip the reasoning | people who skim |
| 4 | Bare minimum, fragments OK | maximum savings, terse |
Set it by hand if you want:
```bash
export HEADROOM_VERBOSITY_LEVEL=3
```
Or — better — let Headroom learn it from your habits (next section).
## Let Headroom pick the level for you
People rarely *say* "be brief." They *show* it: they interrupt long answers, or
reply so fast they couldn't have read the whole thing. `headroom learn
--verbosity` reads your past sessions and picks a level from those signals.
```bash
# Preview what it found (doesn't change anything)
headroom learn --verbosity
# Save it — the proxy uses this level from now on
headroom learn --verbosity --apply
```
Example output:
```
Verbosity — headroom
Interrupts: 29 (11% of turns) ← push-back signal
Fast-skips: 31 / 119 long answers (26% unread) ← strongest signal
>> Recommended verbosity level: 3 (confidence: high)
```
Add `--llm-judge` to have an LLM double-check the level (needs an API key).
## See how much you saved
Here's the honest part. We **can't directly measure** output savings — we never
see what the model *would* have written without our nudge. So Headroom reports an
**estimate with a confidence range**, never a fake exact number:
```bash
headroom output-savings
```
```
Output-token reduction
Method: ESTIMATED (synthetic control)
Requests: 1,240 shaped
Saved: ~410,000 output tokens
Reduction: 28.0% (95% CI 24.1% … 31.9%)
```
- **ESTIMATED** = compared against a baseline of your past (unshaped) sessions.
- **MEASURED** = the gold standard, if you opt into a holdout (below).
### Want a *measured* number?
Leave a slice of traffic unshaped as a control group:
```bash
export HEADROOM_OUTPUT_HOLDOUT=0.1 # 10% of conversations stay unshaped
```
Now `headroom output-savings` compares shaped vs unshaped directly and reports a
**measured** reduction. The trade-off: you give up the savings on that 10%.
## On the dashboard
Open `http://localhost:8787/dashboard`. Next to the input-compression card
you'll see an **Output Tokens Saved** card showing the token count, the percent,
a `measured`/`estimated` badge, and the confidence range.
## FAQ
**Will this make answers worse?**
At level 2 (default), no — in our tests the model finds the same bugs and writes
the same fixes; it just stops re-printing code and skipping the "let me…" intro.
Levels 3–4 are terser by design; that's why learning the level per user matters.
**Does it break prompt caching?**
No. The steering text is added at the *end* of the system prompt and is
byte-stable, so your cached prefix is untouched.
**Is it safe with extended thinking / tool loops?**
Yes. It never disables thinking outright (that can error), it only lowers effort
on routine turns, and it never adds settings the model doesn't support.
**How do I turn it off?**
Unset `HEADROOM_OUTPUT_SHAPER` (or set it to `0`) and restart the proxy. You can
also send `x-headroom-bypass: true` on a request to skip it for that call.
---
Deep dive (design + the counterfactual math): [`proposals/output-token-reduction.md`](proposals/output-token-reduction.md)

461
docs/proposals/output-token-reduction.md
View file

@ -1,461 +0,0 @@
# Output Token Reduction
**Branches:** `feat/output-token-reduction` (Phase 1) → `feat/verbosity-learning-and-counterfactual` (Phase 2).
**Status:** Phase 1 (output shaper) and Phase 2 (`learn --verbosity`, AIMD controller, counterfactual estimator, dashboard) both implemented + tested. Runtime AIMD signal-capture is staged (controller built/tested, live emission off by default).
See **§7** for the counterfactual measurement methodology (how we honestly report a number we can't directly observe).
---
## 1. The problem in one line
Headroom's entire transform pipeline compresses what goes **into** the model.
Nothing today touches what comes **out**. But output tokens are billed at
5× input on Opus-class models ($25 vs $5 per MTok on `claude-opus-4-8`), and in
agentic coding loops a large fraction of the bill is output: thinking tokens,
restated code, ceremony ("Great, let me…"), and full-file rewrites where a
10-line edit would do.
**The constraint that shapes everything:** the proxy never generates output
tokens — the model does. Once a token is streamed it is already billed. So
every output-token lever is **request-side**: change what we ask for, cap what
we allow, or avoid the generation entirely. There is no post-hoc lever.
That gives three lever families plus a learning loop:
| Lever | Mechanism | Status |
|---|---|---|
| **Verbosity steering** | Append a terse-style instruction to the system-prompt tail | ✅ built |
| **Effort routing** | Lower `output_config.effort` on mechanical turns | ✅ built |
| **Thinking-budget clamp** | Clamp legacy `thinking.budget_tokens` on mechanical turns | ✅ built |
| **Per-user learned level** | Mine past sessions for the right verbosity per user (`learn --verbosity`) | ✅ built |
| **Counterfactual estimator** | Honestly estimate output tokens saved + dashboard surfacing | ✅ built |
| **Runtime AIMD auto-tune** | Adjust level live from interrupt / skip signals | 🟡 controller built/tested; live signal emission off by default |
---
## 2. Phase 1 — the output shaper (built)
### 2.1 What it is
`headroom/proxy/output_shaper.py` — a request-body rewriter invoked in
`handle_anthropic_messages` after every other body mutation (so the turn
classifier sees the final message list) and gated behind the same
`x-headroom-bypass` header as compression. Opt-in via `HEADROOM_OUTPUT_SHAPER=1`.
### 2.2 Lever A — verbosity steering
A deterministic instruction block is appended to the **tail** of the system
prompt. Five levels, cumulative:
- **L0** — off (touch nothing).
- **L1** — no ceremony: skip preamble/postamble, don't announce what you're about to do.
- **L2** — L1 + no echo: never restate code/diffs/tool output already in context; reference by path:line; don't narrate tool results. **(default)**
- **L3** — L2 + conclusions only, omit rationale unless asked, prefer smallest edit.
- **L4** — caveman: fragments, minimum tokens, nothing but the answer.
**Why the tail, not the head.** Prompt caching is a prefix match — any byte
change ahead of a `cache_control` breakpoint invalidates everything after it.
Prepending steering text would bust the provider prefix cache and cost more
than it saves. Appending after the last system block leaves the cached prefix
byte-identical; only the small, byte-stable steering block is reprocessed. The
steering text is frozen per level and applied idempotently (sentinel-tagged),
so repeated requests keep an identical prefix and a level change replaces the
block in place rather than stacking.
### 2.3 Lever B — effort routing
In an agentic loop, most API calls are **mechanical continuations**: the last
message is a clean `tool_result` (a file read, a passing test) and the model is
just resuming. Harnesses like Claude Code pin `output_config.effort` at `xhigh`
for *every* turn, including these — and effort drives thinking depth, which
bills as output. The router lowers effort to `low` on mechanical turns only.
Turn classification is **purely structural** — no content regexes, no keyword
lists (per the project's no-hardcoded-patterns rule):
| Last user message contains… | Classification | Action |
|---|---|---|
| Any text / image / document block | `NEW_USER_ASK` | leave effort alone |
| Only `tool_result`, none `is_error` | `MECHANICAL_CONTINUATION` | lower effort → `low` |
| Any `tool_result` with `is_error: true` | `ERROR_CONTINUATION` | leave effort alone (model must reason about the failure) |
| Anything else | `UNKNOWN` | leave alone |
### 2.4 Lever C — legacy thinking-budget clamp
On older models still sending `thinking: {type: "enabled", budget_tokens: N}`,
the router clamps `N` to the API floor (1024) on mechanical turns. The `type`
field is **never** toggled.
### 2.5 Safety rules (each prevents a concrete failure)
1. **Never inject `output_config.effort` where the client didn't send it.**
Models without effort support return 400 on it. Lowering an
already-present value is always valid — its presence proves the target
model accepts the param.
2. **Never toggle `thinking.type`.** Disabling thinking while history carries
thinking blocks 400s on some models, and the toggle busts the messages
cache tier (per the caching invalidation hierarchy).
3. **Byte-stable, idempotent steering** — repeated requests keep an identical
prefix; cache stays warm.
4. **Respect `x-headroom-bypass`** — sub-agent calls that opt out of
compression also opt out of shaping.
### 2.6 Configuration
| Env var | Default | Meaning |
|---|---|---|
| `HEADROOM_OUTPUT_SHAPER` | off | master switch (`1`/`true`/`yes`) |
| `HEADROOM_VERBOSITY_LEVEL` | `2` | 0–4 (clamped) |
| `HEADROOM_EFFORT_ROUTER` | on | set `0` to disable effort routing |
| `HEADROOM_MECHANICAL_EFFORT` | `low` | floor effort for mechanical turns |
### 2.7 Tests
`tests/test_output_shaper.py` — 34 tests, all passing. Covers turn
classification (every block-type path), cache-safe steering (string→block
conversion, append-after-`cache_control`, idempotency, level change),
effort routing (lower / never-inject / untouched-on-non-mechanical /
configurable target), legacy budget clamp, and the env gate. Ruff + mypy clean.
---
## 3. Live before/after results
Measured against `claude-opus-4-8` via `scripts/eval_output_shaper.py`,
comparing the exact body a client sends vs. the body the proxy forwards.
`usage.output_tokens` includes thinking.
### 3.1 Verbosity steering — complex code-review ask
Prompt: "review this TTLCache, find every bug, show fixes."
| Condition | Mean output tokens | Reduction |
|---|---|---|
| Baseline | ~1,800–1,930 | — |
| L2 (no ceremony, no echo) | ~1,180–1,470 | **−22% to −39%** |
| L3 (conclusions only) | ~670 | **−63%** |
**Quality check — same bugs found, only redundancy removed.** The baseline
opens with a title + "Let me go through them," finds the bugs, **re-prints the
entire fixed class**, then adds a **summary table restating all six bugs**,
then a trailing notes section — the same information appears ~2.5×. L2 opens
directly with the findings, gives the same fixed code block once, and stops.
Both correctly identify the no-locking race, `popitem(last=True)` evicting the
wrong end, and the mutate-during-iteration `RuntimeError`. Nothing of substance
is lost at L2; L3 additionally drops rationale prose (only for users who don't
read explanations).
### 3.2 Effort routing — agentic mechanical continuation
Transcript ending in a clean `tool_result`, `effort: xhigh` the way Claude Code
sends it.
| Condition | Mean output tokens | Reduction |
|---|---|---|
| Baseline (xhigh) | ~1,120 | — |
| Shaped (routed to low) | 793 | **−29%** |
### 3.3 Honest framing
Caveman (L4) would get ~60–70% but degrades the experience. The taxonomy-driven
default (L2 + effort routing on mechanical turns) gets a realistic **25–40%**
with the user barely noticing. The learning loop (Phase 2) finds where each
user sits between those poles.
---
## 4. Phase 2 — learning the right level per user
### 4.1 Why this is necessary
The right verbosity is **per-user**, not global. A fixed `HEADROOM_VERBOSITY_LEVEL`
is a guess. We can do better: mine the user's own past sessions to infer what
they actually tolerate — exactly the philosophy of `headroom learn`, which
already reads `~/.claude/projects/*.jsonl` and turns history into learned
context.
### 4.2 The key insight (validated on real data)
I prototyped the signal extraction and ran it over **24 real sessions** for
this project. The finding that shapes the whole design:
| Signal | This user's data | Carries signal? |
|---|---|---|
| Explicit "be brief" keywords | **1** of 210 human msgs | ❌ almost none |
| Explicit "explain more" keywords | **1** of 210 | ❌ almost none |
| **Interruptions** (user cuts Claude off) | **29** (~1 per 7 turns) | ✅ strong |
| **Fast-skips** (reply <30s after a >250-word answer) | **15 of 100 long outputs** | ✅ strong |
| Long-output frequency | 100 outputs >250 words | ✅ context |
**Users almost never *say* what verbosity they want — they show it
behaviorally.** Keyword matching (the obvious approach) is nearly empty. The
behavioral signals are rich. The strongest is **fast-skip**: a reply arriving
faster than the answer could have been read is a direct measurement of
"generated tokens nobody consumed," computable from timestamps + lengths
already in the JSONL.
For this user — 29 interrupts, 15% unread-long-output rate, zero "explain more"
requests — the data reads as a clear **L2, arguably L3** user.
### 4.3 Design — `headroom learn --verbosity`
Slots into the existing `learn` architecture. `headroom/learn/plugins/claude.py`
already parses the JSONL; `analyzer.py` already does cheap-extraction → digest →
LLM-returns-JSON. We add a verbosity analysis path:
**Step 1 — structural pass (pure Python, no patterns).** Per session, compute:
- **interrupt rate** — `[Request interrupted by user…]` markers per human turn
- **fast-skip rate** — human reply latency vs. preceding assistant output length
- **long-output frequency** — share of assistant texts over N words
- **echo ratio** — n-gram overlap between assistant output and prior context (restated code)
- reply-latency distribution vs. output length
All mechanical, all from data already on disk. (Prototype:
`scripts/verbosity_scan.py`, validated above.)
**Step 2 — LLM judgment pass (mirrors today's `learn`).** Feed a digest of
human messages + the structural stats to the analyzer LLM with one question:
*"Does this user read long explanations, and when they push back, is it for
more detail or less?"* This replaces brittle keyword lists with judgment —
consistent with the no-hardcoded-patterns rule. Returns a level + confidence +
rationale as JSON.
**Step 3 — output (this is what `--verbosity` produces).** See §4.4.
**Step 4 — runtime AIMD auto-tune.** The offline pass sets the *starting* level;
a live loop tracks drift. An interrupt or fast-skip nudges the level up one; an
"explain"/"why" follow-up drops it back. Hysteresis (require 2–3 consistent
signals before moving) prevents oscillation. Like TCP congestion control: probe
toward terser, back off on a "too terse" signal.
### 4.4 What `--verbosity` outputs — and how it helps
The command produces three concrete artifacts:
**(a) A human-readable report (stdout):**
```
Verbosity analysis — /Users/tcms/demo/headroom (24 sessions, 210 turns)
Interrupts: 29 (1 per 7.2 turns) ← strong "too much" signal
Fast-skips: 15 / 100 long outputs ← 15% of long answers unread
Explicit brevity: 1 explicit verbose: 1 ← behavioral, not stated
LLM read: "User interrupts frequently and rarely reads long
explanations; pushes back for less, never more detail."
Recommended verbosity level: 2 (confidence: high)
Estimated output-token reduction at L2: ~25–35%
```
**(b) A persisted setting** written to `~/.headroom/` (alongside the existing
savings tracker) — per-project verbosity level + confidence:
```json
{"project": "/Users/tcms/demo/headroom",
"verbosity_level": 2, "confidence": "high",
"signals": {"interrupt_rate": 0.138, "fast_skip_rate": 0.15},
"learned_at": "2026-06-12T…"}
```
**(c) The shaper reads it as its default.** `OutputShaperSettings.from_env()`
gains a fallback: if `HEADROOM_VERBOSITY_LEVEL` is unset, load the learned
per-project level instead of the hardcoded `2`. So **the output of `--verbosity`
directly becomes the live verbosity the proxy applies** — no manual tuning.
**How it helps, concretely:**
1. **Removes the guess.** Today you set `HEADROOM_VERBOSITY_LEVEL=2` by hand.
After `learn --verbosity`, the level is derived from *your* behavior — a
heavy-interrupter gets L3, a "read everything" user gets L1.
2. **Per-project, not global.** Your exploratory side-project and your
production repo can carry different levels.
3. **Justified, not magic.** The report shows the signals and the LLM's read,
so the recommendation is auditable (matches the dashboard philosophy of
showing directional data, not opaque scores).
4. **Seeds the runtime loop.** The learned level is the AIMD starting point;
live signals refine it without re-running the offline pass.
### 4.5 Mapping signals → level (initial heuristic, LLM-refined)
| Interrupt rate | Fast-skip rate | "explain more" present | → Level |
|---|---|---|---|
| low | low | yes | 1 |
| low–med | low–med | no | 2 |
| high | high | no | 3 |
| very high | very high | no | 4 (offer, don't auto-apply) |
The LLM judgment pass adjusts this — the table is the prior, not the verdict.
---
## 5. Files
| File | Status | Purpose |
|---|---|---|
| `headroom/proxy/output_shaper.py` | ✅ built | the shaper (steering + effort routing) |
| `headroom/proxy/handlers/anthropic.py` | ✅ wired | invoke shaper after body mutations |
| `tests/test_output_shaper.py` | ✅ 34 passing | unit coverage |
| `scripts/eval_output_shaper.py` | ✅ built | live before/after eval |
| `scripts/verbosity_scan.py` | 🔬 prototype | session-mining signal extraction |
| `headroom/learn/plugins/claude.py` (+ analyzer) | 🔜 extend | `--verbosity` analysis path |
| `~/.headroom/verbosity.json` | 🔜 | persisted per-project learned level |
---
## 6. Roadmap
1. ✅ **Measure** — live eval establishes the realistic ceiling and baseline.
2. ✅ **Effort router** — biggest win per unit risk, fully mechanical, invisible.
3. ✅ **Verbosity ladder at fixed L2** — safe default, cache-safe tail injection.
4. 🔜 **`learn --verbosity`** — derive the per-user starting level from sessions.
5. 🔜 **Runtime AIMD auto-tune** — refine the level live from interrupt/skip signals.
6. 🔭 **Waste taxonomy on the dashboard** — echo ratio, ceremony ratio,
full-file-rewrite detection, as token counts (no dollar estimates).
7. 🔭 **Budget whispering** — tell the model its token budget per turn-type,
sized from the historical output distribution in SQLite.
---
## 7. Counterfactual measurement — how we show a % we can't directly observe
This is the hard part, and it deserves its own section.
### 7.1 Why output savings are not directly measurable
Input compression is a **pure function**: Headroom takes a request, shrinks it,
and can count `tokens_before` and `tokens_after` — both are observable on the
same request. Output is different. When the shaper makes a request terser, the
model emits N output tokens. We **never observe** what it *would* have emitted
without the steering. Only one side of the counterfactual ever happens. So a
flat "we saved 30%" is a guess dressed as a fact.
The design rule that follows: **never report a single number as if it were
measured.** Separate what is genuinely measured from what is estimated, label
each, and always attach uncertainty.
### 7.2 Three tiers of honesty
**Tier 1 — Estimated (synthetic control).** Build a per-stratum baseline of
*unshaped* output tokens from session history that predates the shaper
(`learn --verbosity` does this in the same pass that picks the level). For each
shaped request, the expected unshaped output is the baseline mean for that
request's stratum. Aggregate estimate:
```
tokens_saved = Σ over shaped requests ( baseline_mean[stratum] − observed_output )
```
Summed as **signed** deltas — never clamped per-request. Clamping each delta at
zero would throw away the cases where a shaped response happened to be *longer*
by chance, biasing the total upward. Over many requests the noise averages out;
the systematic effect remains. Reported with a propagated 95% CI (see §7.5) and
always labelled "estimated."
**Tier 2 — Measured (A/B holdout).** Set `HEADROOM_OUTPUT_HOLDOUT=0.1` and 10%
of conversations are deliberately left **unshaped** as a control arm. Within
each stratum, `mean(control) − mean(treatment)` is an **unbiased causal
estimate** of the per-request saving. This is the only number we call
"measured." It self-corrects: if steering doesn't actually help on some
workload, the holdout reveals it. The cost is real (you forgo savings on 10% of
traffic), so it's opt-in; default holdout is 0 (estimate-only).
**Tier 3 — Direct waste (no counterfactual at all).** Echo ratio — the n-gram
overlap between a response and the context it was given — is a property of a
single response. "32% of this output restated context already on screen" needs
no counterfactual; it's a measured fact about output we *did* see, and it's
exactly what the shaper targets. Surfaced in `learn --verbosity` as
`mean_echo_ratio`.
### 7.3 Stratification — comparing like with like
You can't compare a "fix this typo" response to "design a caching layer." The
estimator buckets every request by features observable **before** the response:
```
stratum = model_family | turn_kind | input_token_bucket | has_tools
= e.g. "opus | mechanical_continuation | xl | tools"
```
Coarse on purpose (~25–50 strata) so per-stratum baselines stay dense. The live
proxy computes the stratum the exact same way the offline baseline does, so
treatment requests line up with their baseline. Unseen strata fall back
hierarchically (drop `has_tools`, then the bucket, …, then the global mean).
### 7.4 The two constraints that happen to align
Holdout assignment is **conversation-stable** — a whole conversation is either
treatment or control, decided by hashing a conversation-stable key (model +
first user message). This matters for two independent reasons that point the
same way:
1. **Measurement validity** — mixing shaped and unshaped turns within one
conversation would contaminate the comparison (the history itself differs).
2. **Cache safety** — flipping a conversation's verbosity mid-stream changes the
system-prompt tail, which busts the provider prefix cache.
So the same rule (assign per conversation, never per turn) is forced by both
the statistics and the caching. Nice when constraints agree.
### 7.5 The confidence interval
For the estimated tier, uncertainty comes from two sources, both propagated:
```
Var(tokens_saved) ≈ Σ_s [ n_s · σ²_observed,s + n_s² · σ²_baseline,s / m_s ]
└─ spread of shaped outputs ─┘ └─ finite-baseline error ─┘
```
where `n_s` is treatment count in stratum `s` and `m_s` the baseline sample
count. For the measured tier it's the standard difference-of-means variance
`σ²_c/n_c + σ²_t/n_t` per stratum. The 95% band is `point ± 1.96·√Var`, surfaced
everywhere the number is (CLI and dashboard), so the reader sees the precision,
not just a point estimate.
Output-token counts are right-skewed (a few huge responses). Means are still the
right statistic for *totals* (you're billed on the sum), but the CI widens
honestly when a stratum is dominated by a few large responses — which is the
correct signal that the estimate is soft there.
### 7.6 How it flows end to end
```
learn --verbosity --apply
├─ writes verbosity.json (the level the shaper applies)
└─ seeds output_savings.json (the per-stratum baseline = synthetic control)
proxy request (HEADROOM_OUTPUT_SHAPER=1)
├─ assign_arm(conversation) → treatment | control (holdout)
├─ stratum_key(request features)
├─ treatment: shape body; control: leave unshaped
└─ tag (arm, stratum) onto transforms_applied ← rides existing plumbing
response completes → emit_request_outcome (one funnel, all paths)
└─ recorder.record(arm, stratum, output_tokens) → output_savings.json
headroom output-savings / dashboard "Output Tokens Saved" card
└─ best_estimate(): measured if a holdout exists, else estimated; with CI
```
The recording rides the existing `transforms_applied` label channel, so it
works for streaming, non-streaming, and backend paths with no change to
`RequestOutcome` or its construction sites.
### 7.7 What the user sees
- **CLI:** `headroom output-savings` →
`Reduction: 31.7% (95% CI 27.7% … 35.7%) [MEASURED, 400 shaped requests]`
- **Dashboard:** an "Output Tokens Saved" hero card next to input compression —
token count, percent, a `measured`/`estimated` badge, and the CI band.
- **No dollar estimates** on the output card (per project convention) — token
counts and directional percentages only.
### 7.8 Honest limitations
- Estimated-tier accuracy depends on the baseline matching current workload; if
your tasks drift, re-run `learn --verbosity` or turn on a small holdout.
- The baseline must come from *unshaped* history. If you learn from sessions
where the shaper was already active, the baseline is contaminated — the live
holdout is the clean path forward.
- Runtime AIMD upward-ratcheting is gated off by default: we can reliably detect
"too much output" (fast-skip timing, stream cancellation) but not yet "too
little" at runtime without content heuristics, so auto-escalation stays
behind `HEADROOM_VERBOSITY_AUTOTUNE` until both directions are trustworthy.

266
docs/proposals/vertex-claude-compression-review.md
View file

@ -1,266 +0,0 @@
# Claude Code + Vertex AI + Headroom compression — does it work?
*A plain-English deep code review. Last updated 2026-06-18.*
## TL;DR (read this first)
**Yes — and the working path is now validated end-to-end (2026-06-19). See the
copy-paste runbook: [`docs/claude-code-vertex-headroom.md`](../claude-code-vertex-headroom.md).**
> **2026-06-19 update — tested against live Vertex quota (Claude Code 2.1.181):**
> Of the two setups below, **only Setup B works in practice.**
>
> - **Setup A (Vertex mode + `ANTHROPIC_VERTEX_BASE_URL`→proxy) is blocked by Claude
> Code itself.** In Vertex mode Claude Code runs a client-side `probeVertexModel`
> check *before any request*; pointing its Vertex URL at a non-Google host makes that
> probe fail instantly ("model … not available on your vertex deployment") and the
> proxy never receives a byte. Not a Headroom bug — the native `:rawPredict`
> passthrough is correct and compresses (verified by direct curl), but the client
> won't route to it.
> - **Setup B (normal Anthropic mode + `--backend litellm-vertex_ai`) is the working
> path.** Verified: Claude Code → Headroom → LiteLLM → Vertex (`global`), real
> answers, and **~22% context compression on a code-heavy request**. Two gotchas:
> (1) `pip install "google-cloud-aiplatform>=1.38"` or requests 500 with
> `No module named 'vertexai'`; (2) start the proxy with **`--code-aware`** or code
> content silently no-ops (it is disabled by default).
There is still **no `headroom wrap claude` turnkey** for Vertex, and the one backend
flag older help text advertises (`--backend litellm-vertex`) is broken — use
`--backend litellm-vertex_ai`.
Everything in the *middle* (compression, request/response translation, streaming,
tool calls) is implemented correctly. The gaps are all at the **edges**: how the
client is pointed at Headroom, one mis-named backend, a missing pip extra, a
default-off compressor, and a few env vars Headroom never sets for you.
> Correction to an earlier claim: it is **not** true that "the Python proxy just
> passes Vertex through without compressing." For the Anthropic publisher it runs
> the full compression pipeline. That earlier statement was based on an incomplete
> read of the routing code; the verified behavior is in this doc.
---
## The thing we're trying to do
An enterprise runs **Claude Code**, but their Claude models live on **Google
Vertex AI** (not the direct Anthropic API). They want **Headroom** in the middle so
their prompts get compressed (fewer input tokens = lower cost), without changing
the answers.
For that to happen, three things must all be true:
1. **The client's traffic must actually reach Headroom** (the proxy must be in the path).
2. **Headroom must compress it.**
3. **Headroom must forward it to Vertex correctly** (right URL, right auth, right body shape) and translate the answer back so Claude Code understands it.
This review checks all three.
---
## The map: where Claude-on-Vertex can run, and what compresses
There are **two proxies** in this repo and **three** possible routes. Only some compress.
| Route | What it is | Compresses? | Notes |
|---|---|---|---|
| **Python proxy, native Vertex `:rawPredict`** (publisher = `anthropic`) | Client sends a real Vertex request to Headroom | ✅ **Yes** | Runs the full Anthropic compression pipeline, keeps the Vertex body shape, forwards the client's own Google token. `proxy_routes.py:648` |
| **Python proxy, `--backend litellm-vertex_ai`** | Client speaks plain Anthropic; Headroom translates to Vertex | ✅ **Yes** (correct string only) | Full Anthropic↔Vertex translation incl. streaming + tools. **`litellm-vertex` is broken — must use `litellm-vertex_ai`.** |
| **Rust proxy, native Vertex `:rawPredict`** | A separate `headroom-proxy` binary | ✅ **Yes** | Correct and well-built — **but never run by `headroom proxy`/`wrap`.** Dead code for normal users. |
| **Python proxy, passthrough** (any *other* publisher) | Generic verbatim forward | ❌ No | Only used for non-Anthropic, non-Google publishers. `openai.py:6014` |
**Key takeaway:** the *compression engine* for Vertex+Claude exists and works in the
Python proxy. The problems are getting traffic into it and one naming bug.
---
## Does it work end-to-end? The honest answer
**Through `headroom wrap claude` with zero extra setup: no.** `wrap claude` only
sets `ANTHROPIC_BASE_URL`. If Claude Code is in Vertex mode it ignores that and
talks straight to Google — Headroom is never in the path. And `wrap claude` has no
`--backend`/`--region` flags and sets no Vertex environment variables.
**With manual setup: yes, one of two ways.** Both are below. Both work *around*
issues, not because the product wires them for you.
---
## Setup A — Claude Code stays in Vertex mode (recommended for Vertex shops)
Idea: Claude Code keeps using its native Vertex mode and its own Google login.
You just tell it "send Vertex requests to Headroom instead of straight to Google,"
and you tell Headroom where the real Vertex endpoint is.
```bash
# 1) Run Headroom, telling it the real Vertex endpoint (match your region!)
headroom proxy --port 8787 \
--vertex-api-url https://us-east5-aiplatform.googleapis.com # use YOUR region
# 2) Run Claude Code in Vertex mode, but point its Vertex base URL at Headroom
export CLAUDE_CODE_USE_VERTEX=1
export ANTHROPIC_VERTEX_PROJECT_ID=<your-gcp-project>
export CLOUD_ML_REGION=us-east5
export ANTHROPIC_VERTEX_BASE_URL=http://127.0.0.1:8787 # <-- the redirect that makes it work
claude
```
What happens: Claude Code → `ANTHROPIC_VERTEX_BASE_URL` (Headroom) → Headroom
matches the `:rawPredict` route, sees `publisher=anthropic`, **compresses**, then
forwards to the real Vertex endpoint using Claude Code's own Google token.
Caveats: you must set `--vertex-api-url` to your region (see Issue #6), and
`wrap claude` won't set `ANTHROPIC_VERTEX_BASE_URL` for you (Issue #3).
---
## Setup B — Claude Code in normal Anthropic mode; Headroom talks to Vertex
Idea: Claude Code thinks it's talking to plain Anthropic. Headroom holds the Google
credentials and is the one that actually talks to Vertex.
```bash
# Headroom does the Vertex talking — note the backend name carefully
export HEADROOM_BACKEND=litellm-vertex_ai # NOT "litellm-vertex" (that's broken — Issue #1)
export HEADROOM_REGION=us-east5 # becomes the Vertex location
export VERTEXAI_PROJECT=<your-gcp-project> # Headroom does NOT set this for you (Issue #4)
export GOOGLE_APPLICATION_CREDENTIALS=/path/sa.json # or use `gcloud auth application-default login`
export ANTHROPIC_API_KEY=placeholder-not-used # Claude Code needs *a* key to start (Issue #5)
# Do NOT set CLAUDE_CODE_USE_VERTEX here — Claude Code must stay in normal mode
headroom wrap claude
```
What happens: Claude Code → Headroom (plain Anthropic `/v1/messages`) →
**compresses** → LiteLLM converts to Vertex and calls Claude on Vertex → converts
the answer back to Anthropic shape → Claude Code reads it.
Caveats: the backend-name bug (Issue #1), the missing project env (Issue #4), and
this path has **no automated tests** (Issue #8) — smoke-test it before relying on it.
---
## How to verify compression is really happening
1. Open the dashboard: `http://localhost:8787/dashboard` — "tokens saved" should
climb as you use Claude Code.
2. Or check response headers on a request: `x-headroom-tokens-before`,
`x-headroom-tokens-after`, `x-headroom-tokens-saved`.
3. Confirm it actually hit Vertex (proxy logs show a `vertex_ai/claude-…` model or
a Vertex host, not `api.anthropic.com`).
If `tokens-saved` is 0 on large prompts, compression isn't running — re-check the
setup against the issues below.
---
## Every issue we found (the full list)
Severity: **BROKEN** = doesn't work; **GAP** = works only with manual workaround;
**BUG** = wrong behavior in an edge case; **HOUSEKEEPING** = confusing but harmless.
### 1. BROKEN — `--backend litellm-vertex` never reaches Vertex
The backend name is turned into a provider by chopping off `litellm-`, so
`litellm-vertex` becomes the provider `vertex`. But the Vertex integration is keyed
on `vertex_ai`, not `vertex`. So Headroom falls back to a generic "unknown
provider" mode: it builds the wrong model name (`vertex/claude-…` instead of
`vertex_ai/claude-…`), **ignores the region**, and mishandles auth.
**You must use `--backend litellm-vertex_ai`.** Worse: every help message and the
`wrap` example tell users the broken `litellm-vertex`.
*Where:* `providers/registry.py:174-178`, `backends/litellm.py:291,326-336,681-682`;
help text at `cli/proxy.py:524`, `cli/wrap.py:3645`, `proxy/server.py:3913`.
*Fix (small):* alias `vertex` → `vertex_ai` in `create_proxy_backend`, or add a
`"vertex"` entry to the provider registry. Then fix the help text.
### 2. GAP — `headroom wrap claude` has no Vertex support
The `claude` wrap command has no `--backend` and no `--region` (the `aider` wrap
command has both). It only ever sets `ANTHROPIC_BASE_URL`. So there's no flag to
turn on a Vertex backend for Claude Code — you must pre-export env vars.
*Where:* `cli/wrap.py:2780-2819` (vs `cli/wrap.py:3612,3615` for aider).
*Fix:* add `--backend`/`--region` to `wrap claude`, mirroring `aider`.
### 3. GAP — Vertex-mode Claude Code bypasses the proxy, and Headroom never sets the fix
With `CLAUDE_CODE_USE_VERTEX=1`, Claude Code ignores `ANTHROPIC_BASE_URL` and goes
straight to Google. There **is** a documented override — `ANTHROPIC_VERTEX_BASE_URL`
— that points Claude Code's Vertex traffic at a gateway. But Headroom never sets it
(0 references in the codebase). So the proxy has the right routes, but nothing
connects the client to them automatically.
*Where:* repo-wide grep for `ANTHROPIC_VERTEX_BASE_URL` = 0 hits.
*Fix:* in a Vertex-aware `wrap claude`, set `ANTHROPIC_VERTEX_BASE_URL` to the proxy.
### 4. GAP — the GCP project is never passed to LiteLLM
For Setup B, Headroom passes the region to LiteLLM but never the project. You must
export `VERTEXAI_PROJECT` (or `GOOGLE_CLOUD_PROJECT`) yourself or it fails.
*Where:* `backends/litellm.py:682` sets only `vertex_location`.
*Fix:* thread a project config/env through to the LiteLLM call.
### 5. GAP — no placeholder API key for Claude Code
With a custom `ANTHROPIC_BASE_URL`, Claude Code needs *an* `ANTHROPIC_API_KEY` to
start, even though the proxy uses Google creds upstream. `wrap claude` never sets a
placeholder, so the user must.
*Where:* `cli/wrap.py:2984-2988`.
*Fix:* set a placeholder key (or `ANTHROPIC_AUTH_TOKEN`) when launching.
### 6. BUG — region/host mismatch
Headroom pins the Vertex host to one region (default `us-central1`) but throws away
the region in the client's request path. If your client targets, say,
`europe-west1` while Headroom is on the default host, the request goes to the wrong
region unless you set `--vertex-api-url` to match.
*Where:* `copilot_auth.py:936` (host = base + path, no region reconciliation),
`providers/proxy_routes.py:647` (path `location` is discarded), `registry.py:16`.
*Fix:* derive the upstream host from the request path's `location`, or validate they match.
### 7. HOUSEKEEPING — the Rust Vertex proxy is correct but unwired
There's a second, Rust proxy (`crates/headroom-proxy/src/vertex/`) that compresses
Vertex traffic correctly. But `headroom proxy` and `headroom wrap` run the **Python**
server and never call it — it's a separate binary you'd run by hand. This is a
frequent source of "but I thought Vertex compression was added" confusion: it was,
in Rust, on a path nobody runs by default.
*Where:* `crates/headroom-proxy/Cargo.toml` (`[[bin]]`), no Python→Rust bridge to it.
*Fix:* either document that the Rust proxy is separate, or wire/retire it.
### 8. GAP — no tests for the Vertex compression paths
No test instantiates the LiteLLM Vertex backend with a mocked Vertex call, and the
native-Vertex compression route isn't covered against a real Vertex shape. The
translation code is correct by inspection, but unproven by CI.
*Fix:* add a mocked round-trip test (Anthropic in → compressed → Vertex call asserted → Anthropic out).
### 9. HOUSEKEEPING — stale Rust doc comment
`crates/headroom-proxy/src/vertex/mod.rs:42-47` describes a "synthetic model
injection" strategy the code no longer implements. Doc only; behavior is correct.
---
## What's actually solid (so we don't over-correct)
These were verified and are **correct**:
- **Native Vertex `:rawPredict` for `publisher=anthropic` compresses** and preserves
the Vertex body shape (keeps `anthropic_version`, never injects `model`).
`proxy_routes.py:648`, `anthropic.py:604-606,1941-1949`.
- **LiteLLM translation is real and complete** (with the `vertex_ai` provider):
response is rebuilt into Anthropic shape (`backends/litellm.py:575-628`), streaming
emits proper Anthropic SSE events (`streaming.py:1344-1472`, `litellm.py:736-947`),
and tool calls round-trip both directions (`litellm.py:525-565,593-602`).
- **Compression runs before the backend dispatch** (`anthropic.py:1671` then
`:1781`), so the backend always gets the compressed body.
- **Auth is forwarded correctly** on the native path — the client's Google token is
passed through untouched (`copilot_auth.py:1156-1157`).
---
## Recommended fixes, smallest-first
1. **Fix the backend name (Issue #1)** — one-line alias `vertex`→`vertex_ai`, then
correct the help text. This is the highest-impact, lowest-effort fix; it turns the
*documented* command from broken to working.
2. **Add `--backend`/`--region` to `wrap claude` (Issue #2)** — copy from `aider`.
3. **Add a Vertex mode to `wrap claude` (Issues #3, #5)** — detect/set
`ANTHROPIC_VERTEX_BASE_URL` → proxy, set a placeholder API key, and configure the
proxy's Vertex upstream — so Setup A becomes one command.
4. **Pass the GCP project (Issue #4)** and **reconcile the region/host (Issue #6).**
5. **Add a mocked round-trip test (Issue #8).**
6. **Decide the Rust proxy's fate (Issue #7)** — document-as-separate or wire it in.
After #1–#3, the honest customer message becomes: *"`headroom wrap claude` works
with Vertex out of the box."* Until then, it's *"works with a documented manual
setup."*

125
docs/rtk-loop-weighting.md
View file

@ -1,125 +0,0 @@
# Loop weighting in Headroom Learn + RTK-loop eval
**Status:** proposed (branch `purva/rtk-loop-evals`).
**Context:** Tejas asked for (1) an eval that reproduces the RTK loop, runs it
through Headroom Learn, and checks the generated rule prevents re-triggering,
and (2) a change so Headroom Learn gives loops more weight.
## The gap
Before this change, `headroom learn` ranked every recommendation by a single
LLM-guessed `estimated_tokens_saved`, with a flat hardcoded `confidence`
(`0.9`/`0.7`). It had **no notion of a loop**. Two consequences:
1. **RTK re-fetch loops were invisible.** RTK truncates a shell command's
output (`grep foo` → `grep foo | head -50`, see `docs/rtk-architecture.md`).
When the truncation drops what the agent needed, the agent re-runs a
*variant* to fetch more. **Those calls succeed** (`is_error=False`), so the
analyzer's failure-oriented path ignored them — and `analyze()` even
early-returned when a session had no failures and no events.
2. **Even when surfaced, a loop ranked no higher than a one-off.** A pattern
that wastes 5,000 tokens by repeating 6× was ranked the same as a one-time
200-token mistake, because ranking trusted the LLM's per-rule guess.
## The change
A new module `headroom/learn/loops.py`:
- **`detect_loops(sessions)`** groups tool calls within a session by a
*canonical signature* that collapses RTK re-fetch variants (it strips
pagination/limit fragments — `head -N`, `-n N`, `LIMIT N`, … — and bare
integers), then flags any signature repeated `>= 3×`. It classifies each as
an `error-loop` or an `rtk-refetch-loop` and computes **measured** wasted
tokens (error loops waste every call; re-fetch loops credit the first,
legitimate call and count the N−1 redundant re-fetches).
- **`format_loops_for_digest(loops)`** prepends a `=== Detected Loops (HIGHEST
PRIORITY) ===` block to the LLM digest, with each loop's measured waste.
- **`apply_loop_weighting(recs, loops)`** raises a matching recommendation's
`estimated_tokens_saved` to at least the loop's measured waste and tags it
`is_loop_guardrail=True`. Because measured loop waste aggregates many
repetitions, this reliably lifts loop guardrails above one-offs **without
trusting the LLM** to have weighted them.
Wiring in `analyzer.py`: loops are detected up front (so a no-failure re-fetch
loop is now a first-class reason to analyze, fixing the early-return), surfaced
in the digest, the system prompt makes loops the #1 priority, and weighting +
re-sort run after parsing.
### Why measured-waste weighting (vs. relying on the LLM's estimate)
The LLM's `estimated_tokens_saved` is a free-form guess, not grounded in the
transcript, so ranking on it alone is unreliable. Deriving the weight from
*observed repetition* — the real output bytes summed across the repeated calls
— is deterministic and auditable: the boost equals waste we actually counted.
Honest caveat on the current implementation: we do BOTH — the digest also tells
the model the measured waste and asks it to rank loops first. In real-LLM runs
that prompt hint is doing much of the work (the model echoes the measured
figure), while the post-hoc `apply_loop_weighting` boost is fuzzy-match-based
and does not always fire. Making the measured-waste boost the deterministic,
load-bearing mechanism — independent of the model's wording — is tracked as
follow-up.
## The eval
`benchmarks/rtk_loop_learn_eval.py` (CI wrapper: `tests/test_learn/
test_rtk_loop_eval.py`). Two phases:
- **Phase 1 — trigger + learn:** reproduce the RTK re-fetch loop, run the
analyzer, and score the guardrail: produced? ranked first? names the command?
prescribes a fix? does its savings estimate reflect measured waste?
- **Phase 2 — guardrail holds:** inject that guardrail as a prior pattern, feed
a session where the agent *followed* it (one full-output fetch, no loop), and
assert no new loop guardrail is re-emitted — i.e. once the rule exists and is
honored, the loop does not re-trigger.
Runs deterministically in CI (stubbed analyzer LLM) and against a real LLM with
`--real` — via an API key or an installed CLI backend (`HEADROOM_LEARN_CLI=claude`).
```
$ python benchmarks/rtk_loop_learn_eval.py
[PASS] loop_detected (1 loop(s), ~5,005 tok wasted)
[PASS] guardrail_produced
[PASS] ranked_first
[PASS] names_command
[PASS] prescribes_fix
[PASS] weight_reflects_waste
[PASS] guardrail_holds
RESULT: PASS — loop caught, guardrail ranked first, and it holds.
```
### Real-LLM run (claude CLI backend)
Running `--real` against the actual analyzer model proved the weighting works
end-to-end *and* caught an over-brittle check. The model produced this rule,
ranked **first** with the measured 5,005-token weight:
> **Commands** — When grepping logs (or any large file), never loop with
> increasing `| head -N` limits — tool output is capped at ~4 KB regardless of
> N, so repeated attempts return identical bytes. Instead: redirect to a temp
> file (`grep ... > /tmp/out.txt`) then read it, or use `grep -c` first…
That rule is *more general* than the fixture's — it identifies the looping
command (`grep` + `head -N`) without echoing the incidental search string. An
early `names_command` check required the literal "TimeoutError" and wrongly
failed; the real run exposed it, and the check now verifies the rule names the
looping command, not an incidental literal. This is exactly why the governance
treats real output — not mocks — as proof.
## Honest limitations / open questions for review
- **Phase 2 is a non-recurrence check, not a live agent.** It proves the
guardrail is *adequate* (names the command, prescribes the fix) and that a
guarded, non-looping session produces no new rule. It does **not** run a real
agent that obeys the rule end-to-end — that needs a live agent harness and is
the natural next step if we want a stronger claim.
- **Loop signature is heuristic.** The pagination-stripping regex covers the
common RTK truncation shapes (`head`/`tail`/`-n`/`LIMIT`/`OFFSET`); exotic
truncations may not collapse to one signature. Easy to extend as we see real
transcripts.
- **`min_occurrences = 3`** treats a single retry as not-yet-a-loop. If we have
data showing 2× re-fetches are already worth a rule, lower it.
- **Matching rules to loops is fuzzy** (token overlap between the rule text and
the looped command). A structured loop→rule id from the LLM would be tighter
but adds prompt/parse surface.

109
docs/spec/001-vision.md
View file

@ -1,109 +0,0 @@
# 001. Vision
**Status:** done
## What Headroom Is
Headroom is a **context compression proxy** for AI provider APIs. It sits between AI coding tools (Claude Code, Copilot, Codex, etc.) and provider APIs (OpenAI, Anthropic, Google, Cohere), compressing context before it reaches the provider to reduce token usage and costs.
### Core Value Proposition
1. **Token Savings** — 30-90% reduction in tokens sent to providers through semantic compression
2. **Cost Reduction** — Lower API costs via compression before provider transmission
3. **Context Window Extension** — Effective 2-10x larger context windows via compression
4. **Privacy** — Prompts never logged by default; all processing stays local
5. **Compatibility** — Works with existing AI coding tools via proxy or SDK
### What Headroom Is Not
- A model provider — Headroom does not host or run AI models
- A data store — No prompt storage by default (local SQLite is optional)
- A logging service — No prompt logging by default
- A billing service — Usage tracking is internal only
---
## Design Principles
### 1. Local-First Privacy
**Principle:** Prompt data never leaves the proxy unless explicitly exported.
**Implications:**
- All compression happens locally or through the proxy
- No third-party data sharing
- Optional SQLite storage with user control
- Export must be explicitly configured
### 2. Transparent Compression
**Principle:** Users see exactly what is being compressed and why.
**Implications:**
- Full observability into compression decisions
- Metrics and logs show savings
- Transform audit trail available
- Dashboard visualizes all compression activity
### 3. Composable Integration
**Principle:** Headroom works alongside existing tools without requiring workflow changes.
**Implications:**
- Proxy mode: route traffic through Headroom
- SDK mode: integrate into custom applications
- CLI mode: wrap existing AI commands
- Agent mode: MCP/LiteLLM/ASGI integrations
### 4. Production-Ready Defaults
**Principle:** Safe defaults that work out of the box.
**Implications:**
- Compression enabled by default
- No logging by default
- Cache enabled by default
- Learning disabled by default
---
## Core Guarantees
| Guarantee | Description |
|-----------|-------------|
| **Never logs prompts** | No prompt data in logs unless exporter configured |
| **Never leaves proxy** | All data stays local unless explicitly exported |
| **Composable** | Works alongside Claude Code, Copilot, Codex, etc. |
| **Transparent** | Full observability into compression decisions |
| **Type-safe** | Full type annotations, mypy compliance |
| **Test-covered** | Unit, integration, and E2E test coverage |
---
## Target Users
| User | Use Case |
|------|----------|
| **Individual Developers** | Reduce API costs for personal AI coding |
| **Development Teams** | Shared compression with learn plugins |
| **Enterprises** | Self-hosted deployment with security guarantees |
| **Plugin Authors** | Extend Headroom via plugin ABI |
---
## Success Metrics
| Metric | Target | Measurement |
|--------|--------|-------------|
| Token savings | >30% | (tokens_before - tokens_after) / tokens_before |
| Compression latency | <50ms | Per-request proxy overhead |
| Cache hit rate | >60% | cache_hits / total_requests |
| Zero data exfiltration | 100% | No prompts in logs by default |
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial vision document |

339
docs/spec/002-architecture.md
View file

@ -1,339 +0,0 @@
# 002. Architecture
**Status:** done
## System Overview
Headroom is a context compression proxy for LLM applications, featuring intelligent transforms, semantic caching, and CCR (Compress-Cache-Retrieve) architecture.
```
┌─────────────────────────────────────────────────────────────────┐
│ Headroom │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌─────────────┐ │
│ │ Proxy │ │ SDK │ │ Wrap │ │ CCR MCP │ │
│ │ Server │ │ (Python) │ │ CLI │ │ Server │ │
│ │ (8787) │ │ │ │ │ │ │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └──────┬──────┘ │
│ │ │ │ │ │
│ ┌────┴──────────────┴──────────────┴─────────────────┴──────┐ │
│ │ Compression Layer │ │
│ │ ┌────────────┐ ┌────────────┐ ┌────────────────────┐ │ │
│ │ │SmartCrusher│ │CacheAligner│ │ RollingWindow │ │ │
│ │ │(JSON array │ │(Prefix │ │ (Token cap) │ │ │
│ │ │ crush) │ │ stabilization)│ │ │ │ │
│ │ └────────────┘ └────────────┘ └────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ Learn System │ │
│ │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────────────┐ │ │
│ │ │Claude │ │Codex │ │Gemini │ │ Generic │ │ │
│ │ │Scanner │ │Scanner │ │Scanner │ │ Writer │ │ │
│ │ └────────┘ └────────┘ └────────┘ └────────────────┘ │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ Dashboard │ │ CCR │ │ TOIN │ │
│ │ (HTML) │ │ (Compress- │ │ (Telemetry-based │ │
│ │ │ │ Cache- │ │ Intelligence) │ │
│ │ │ │ Retrieve) │ │ │ │
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
---
## Component Specifications
### Proxy Server (`headroom/proxy/server.py`)
**`HeadroomProxy` class** — Main proxy server (default port 8787):
```python
class HeadroomProxy:
def __init__(self, config: ProxyConfig) -> None:
self.config = config
self.compression_cache = CompressionCache(...)
self.transform_pipeline = [...]
self.ccr_store = CompressionStore(...)
self.prefix_freeze = PrefixFreeze(...)
async def startup(self) -> None: ...
async def shutdown(self) -> None: ...
async def handle_request(request: Request) -> Response: ...
```
**`ProxyConfig` dataclass** (from `headroom/models/config.py`):
```python
@dataclass
class ProxyConfig:
store_url: str = "sqlite:///headroom.db"
default_mode: HeadroomMode = HeadroomMode.AUDIT
tool_crusher: ToolCrusherConfig = field(default_factory=ToolCrusherConfig)
smart_crusher: SmartCrusherConfig = field(default_factory=SmartCrusherConfig)
cache_aligner: CacheAlignerConfig = field(default_factory=CacheAlignerConfig)
rolling_window: RollingWindowConfig = field(default_factory=RollingWindowConfig)
cache_optimizer: CacheOptimizerConfig = field(default_factory=CacheOptimizerConfig)
ccr: CCRConfig = field(default_factory=CCRConfig)
prefix_freeze: PrefixFreezeConfig = field(default_factory=PrefixFreezeConfig)
```
**`HeadroomMode` enum** (actual modes):
```python
class HeadroomMode(str, Enum):
AUDIT = "audit" # Observe only, no modifications
OPTIMIZE = "optimize" # Apply deterministic transforms
SIMULATE = "simulate" # Return transform plan without API call
```
**HTTP Endpoints (actual):**
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/v1/messages` | POST | Proxy chat completions |
| `/v1/embeddings` | POST | Proxy embeddings |
| `/health` | GET | Basic health check |
| `/livez` | GET | Liveness check |
| `/readyz` | GET | Readiness check |
| `/metrics` | GET | Prometheus metrics |
| `/v1/compress` | POST | Direct compression |
| `/v1/retrieve` | POST | CCR retrieval |
| `/stats` | GET | Compression statistics |
**Default Port:** 8787 (not 8765)
---
### Python SDK (`headroom/client.py`)
**`HeadroomClient` class:**
```python
class HeadroomClient:
def __init__(
self,
api_key: str | None = None,
base_url: str = "http://localhost:8787",
timeout: float = 60.0,
fallback: bool = True,
retries: int = 3,
) -> None:
pass
async def compress(
self,
messages: list[dict],
options: CompressOptions | None = None,
) -> CompressResult: ...
async def get_stats(self) -> Stats: ...
async def close(self) -> None: ...
```
---
### Wrap CLI (`headroom/cli/wrap.py`)
**Commands (all use default port 8787):**
- `claude` — Wrap Claude Code
- `copilot` — Wrap GitHub Copilot
- `codex` — Wrap OpenAI Codex
- `aider` — Wrap Aider
- `cursor` — Wrap Cursor
- `openclaw` — Wrap OpenClaw
```python
@click.command()
@click.option("--port", "-p", default=8787, help="Proxy port")
@click.argument("command", nargs=-1, required=True)
def wrap(command: tuple, port: int) -> None:
"""Wrap a command with Headroom proxy."""
...
```
---
### CCR MCP Server (`headroom/ccr/mcp_server.py`)
Model Context Protocol server for Claude Desktop integration.
```python
class CCRMcpServer:
async def compress(messages: list[dict]) -> CompressResult: ...
async def retrieve(hash: str, query: str) -> RetrieveResult: ...
async def get_stats() -> Stats: ...
```
**MCP Tools:**
- `headroom_compress` — Compress messages
- `headroom_retrieve` — Retrieve cached content
- `headroom_stats` — Get statistics
---
### ASGI Middleware (`headroom/integrations/asgi.py`)
ASGI-compatible middleware for Python web frameworks.
```python
class HeadroomMiddleware:
def __init__(
self,
app: ASGIApplication,
headroom_url: str = "http://localhost:8787",
) -> None: ...
```
---
### LiteLLM Callback (`headroom/integrations/litellm_callback.py`)
Callback for LiteLLM proxy integration.
```python
class LiteLLMCallback:
def on_completion(self, completion_response: dict) -> dict: ...
def on_error(self, error: Exception) -> None: ...
```
---
## Compression Layer (`headroom/transforms/`)
### SmartCrusher (`headroom/transforms/smart_crusher.py`)
Statistical JSON array compression preserving schema.
```python
class SmartCrusher:
def __init__(self, config: SmartCrusherConfig) -> None: ...
def crush(self, content: str, context: TransformContext) -> TransformResult: ...
```
### CacheAligner (`headroom/transforms/cache_aligner.py`)
Prefix stabilization for provider cache optimization.
```python
class CacheAligner:
def __init__(self, config: CacheAlignerConfig) -> None: ...
def align(self, messages: list[dict]) -> TransformResult: ...
```
### RollingWindow (`headroom/transforms/rolling_window.py`)
Rolling window token cap.
```python
class RollingWindow:
def __init__(self, config: RollingWindowConfig) -> None: ...
def apply(self, messages: list[dict]) -> TransformResult: ...
```
### ContentRouter (`headroom/transforms/content_router.py`)
Routes content to appropriate compressor based on type.
```python
class ContentRouter:
def route(self, messages: list[dict]) -> list[dict]: ...
```
---
## Learn System (`headroom/learn/`)
**`LearnPlugin` interface** (actual):
```python
class LearnPlugin(ConversationScanner):
@property
def name(self) -> str: ...
@property
def display_name(self) -> str: ...
@abstractmethod
def detect(self) -> bool: ...
@abstractmethod
def discover_projects(self) -> list[ProjectInfo]: ...
@abstractmethod
def scan_project(self, project: ProjectInfo, max_workers: int = 1) -> list[SessionData]: ...
@abstractmethod
def create_writer(self) -> ContextWriter: ...
```
**Scanner implementations:**
- `ClaudeScanner` — Claude Code session parsing
- `CodexScanner` — Codex session parsing
- `CursorScanner` — Cursor session parsing
---
## CCR System (`headroom/ccr/`)
CCR (Compress-Cache-Retrieve) makes compression reversible.
```python
class CompressionStore:
def store(self, hash: str, original: str, metadata: dict) -> None: ...
def retrieve(self, hash: str) -> str | None: ...
class ContextTracker:
def track(self, session_id: str, messages: list[dict]) -> CCRContext: ...
def get_context(self, session_id: str) -> CCRContext | None: ...
```
**CCRConfig fields:**
- `enabled: bool = True`
- `store_max_entries: int = 1000`
- `store_ttl_seconds: int = 300`
- `inject_retrieval_marker: bool = True`
- `feedback_enabled: bool = True`
---
## TOIN (`headroom/telemetry/toin.py`)
Tool Output Intelligence Network — telemetry-based compression hints.
```python
class TOINCollector:
def record_retrieval(self, tool_name: str, field: str, query: str) -> None: ...
def get_hints(self, tool_name: str) -> dict[str, float]: ...
```
---
## Dashboard (`headroom/dashboard/`)
Simple HTML dashboard served by the proxy.
```python
def get_dashboard_html() -> str:
"""Load the dashboard HTML template."""
return (TEMPLATES_DIR / "dashboard.html").read_text()
```
---
## Data Flow
```
Client → Headroom Proxy → [ContentRouter] → [SmartCrusher/CacheAligner/RollingWindow]
│ │ │
│ [CCR Store] ←───────────────────────┘
│
│ [Telemetry/Metrics]
│
▼
Provider API
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial architecture document |

164
docs/spec/003-adrs.md
View file

@ -1,164 +0,0 @@
# 003. Architecture Decision Records
**Status:** done
## ADR-001: Why a Proxy Instead of SDK Injection
**Context:** Headroom could be implemented as an SDK that users import, or as a network proxy.
**Decision:** Network proxy with SDK for complex use cases.
**Rationale:**
- Works with any HTTP client without code changes
- No need to modify existing applications
- Centralized configuration
- Can intercept all traffic, not just explicit SDK calls
**Consequences:**
- Requires running a separate service (or using wrap mode)
- Network latency added (~5-10ms)
- Need to handle connection pooling
---
## ADR-002: Why SQLite for Local Storage
**Context:** Headroom needs to store compression cache, savings history, memory graphs.
**Decision:** SQLite with optional external stores (Redis, PostgreSQL, etc.).
**Rationale:**
- Zero configuration — works out of the box
- Single file, easy backup
- ACID compliant transactions
- Good performance for single-node deployments
- sqlite-vec for vector similarity search
- FTS5 for full-text search
**Consequences:**
- Not distributed by default
- Must configure external stores for multi-node deployments
- Cloud-hosted databases require additional setup
---
## ADR-003: Why CCR (Compress-Cache-Retrieve) Pattern
**Context:** Compression alone doesn't leverage repeated context patterns.
**Decision:** CCR pattern for semantic caching and retrieval.
**Rationale:**
- Semantic similarity enables cache hits across different phrasings
- Retrieves relevant compressed context for new requests
- Reduces provider API calls for similar patterns
- Enables cross-session learning
**Consequences:**
- Requires storing compressed content
- Semantic hashing adds latency
- Cache invalidation is complex
---
## ADR-004: Why Per-Agent Plugins for Learn System
**Context:** Different AI agents (Claude, Codex, Gemini) have different context patterns.
**Decision:** Plugin architecture with agent-specific analyzers.
**Rationale:**
- Tailored compression per agent type
- Extensible for new agents
- Clear interface contract
- Independent versioning
**Consequences:**
- Plugin API must be stable
- Multiple plugins may conflict
- Testing complexity increases
---
## ADR-005: Why ONNX for TOIN
**Context:** TOIN (Tenant-specific ONNX) requires ML inference.
**Decision:** Use ONNX Runtime for portable ML inference.
**Rationale:**
- Hardware acceleration (CPU/GPU)
- Cross-platform (Windows, Linux, macOS)
- Model interchange format
- Single model file deployment
**Consequences:**
- ONNX model files must be hosted
- Version compatibility issues
- Larger package size
---
## ADR-006: Why Python for Core Implementation
**Context:** Language choice for the main implementation.
**Decision:** Python as the primary language.
**Rationale:**
- Primary language for AI/ML ecosystem
- Easy integration with provider APIs
- Rich async ecosystem (asyncio, httpx)
- Strong type annotation support (mypy)
- Good testing infrastructure
**Consequences:**
- GIL limitations for threading
- Slower than compiled languages
- Type checking adds build time
---
## ADR-007: Why TypeScript SDK for npm
**Context:** JavaScript/TypeScript ecosystem for frontend integrations.
**Decision:** Official TypeScript SDK published to npm.
**Rationale:**
- Node.js compatibility
- TypeScript type safety
- Wide adoption in AI tooling
- ESM and CommonJS support
**Consequences:**
- Dual language maintenance
- Must keep SDK in sync with Python core
- Additional CI/CD pipeline needed
---
## ADR-008: Why HierarchicalMemory with SQLite + vec + FTS5
**Context:** Memory system needs to store, search, and reason over context.
**Decision:** Hierarchical memory using SQLite + sqlite-vec + FTS5.
**Rationale:**
- SQLite: ACID transactions, single file
- sqlite-vec: Vector similarity for semantic search
- FTS5: Full-text search for keyword matching
- Hierarchical: Session < Conversation < Message structure
**Consequences:**
- Memory hierarchy adds complexity
- SQLite limitations for concurrent writes
- Vector search accuracy depends on embedding quality
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial ADRs |

363
docs/spec/004-domain-model.md
View file

@ -1,363 +0,0 @@
# 004. Domain Model
**Status:** done
## Core Entities
### Session
A session represents a conversation context between a user and an AI agent.
**Fields:**
- `session_id: UUID` — Unique identifier
- `created_at: datetime` — Creation time
- `updated_at: datetime` — Last modification
- `agent_type: str` — "claude", "codex", "gemini", etc.
- `messages: list[Message]` — Conversation history
- `metadata: dict` — Agent-specific metadata
```python
@dataclass
class Session:
session_id: UUID
created_at: datetime
updated_at: datetime
agent_type: str
messages: list[Message] = field(default_factory=list)
metadata: dict = field(default_factory=dict)
```
---
### Message
A single message in a conversation.
**Fields:**
- `message_id: UUID` — Unique identifier
- `session_id: UUID` — Associated session
- `role: str` — "user", "assistant", "system", "tool"
- `content: str` — Message content
- `tokens: int | None` — Token count (if known)
- `created_at: datetime` — Creation time
- `attachments: list[Attachment] | None` — Optional attachments
```python
@dataclass
class Message:
message_id: UUID
session_id: UUID
role: str
content: str
tokens: int | None = None
created_at: datetime = field(default_factory=datetime.utcnow)
attachments: list[Attachment] | None = None
```
---
### Request
A single API request to an AI provider.
**Fields:**
- `request_id: UUID` — Unique identifier
- `session_id: UUID` — Associated session
- `provider: str` — "anthropic", "openai", "google", etc.
- `model: str` — Model identifier
- `input_tokens: int` — Tokens in request
- `output_tokens: int` — Tokens in response
- `compressed: bool` — Whether compression was applied
- `savings_percentage: float` — Savings as decimal
- `timestamp: datetime` — Request time
- `duration_ms: int` — Request duration
- `error: str | None` — Error message if failed
```python
@dataclass
class Request:
request_id: UUID
session_id: UUID
provider: str
model: str
input_tokens: int
output_tokens: int
compressed: bool = False
savings_percentage: float = 0.0
timestamp: datetime = field(default_factory=datetime.utcnow)
duration_ms: int = 0
error: str | None = None
```
---
### Savings
A record of token savings from a compressed request.
**Fields:**
- `savings_id: UUID` — Unique identifier
- `request_id: UUID` — Associated request
- `original_tokens: int` — Tokens before compression
- `compressed_tokens: int` — Tokens after compression
- `savings_percentage: float` — Savings as decimal
- `savings_amount: float` — Absolute token savings
- `provider: str` — Provider name
- `model: str` — Model used
- `window_start: datetime | None` — Billing window start
```python
@dataclass
class Savings:
savings_id: UUID
request_id: UUID
original_tokens: int
compressed_tokens: int
savings_percentage: float
savings_amount: float
provider: str
model: str
window_start: datetime | None = None
```
---
### CacheEntry
A cached compression result.
**Fields:**
- `cache_key: str` — Hash of input
- `input_hash: str` — Hash of original content
- `output: str` — Compressed output
- `compression_type: str` — "semantic", "summary", "ccr"
- `tokens_before: int` — Tokens before compression
- `tokens_after: int` — Tokens after compression
- `created_at: datetime` — Creation time
- `ttl: int` — Time to live in seconds
- `hit_count: int` — Number of cache hits
- `last_accessed: datetime` — Last access time
```python
@dataclass
class CacheEntry:
cache_key: str
input_hash: str
output: str
compression_type: str
tokens_before: int
tokens_after: int
created_at: datetime = field(default_factory=datetime.utcnow)
ttl: int = 3600
hit_count: int = 0
last_accessed: datetime = field(default_factory=datetime.utcnow)
```
---
### ToolDefinition
A tool/function definition available to an AI agent.
**Fields:**
- `tool_id: UUID` — Unique identifier
- `name: str` — Tool name
- `description: str` — Tool description
- `parameters: dict` — JSON Schema for parameters
- `provider: str` — Provider that defined this tool
- `created_at: datetime` — Creation time
```python
@dataclass
class ToolDefinition:
tool_id: UUID
name: str
description: str
parameters: dict
provider: str
created_at: datetime = field(default_factory=datetime.utcnow)
```
---
### PluginInterface
A learn plugin for agent-specific compression.
**Fields:**
- `plugin_id: UUID` — Unique identifier
- `name: str` — Plugin name
- `agent_type: str` — Supported agent type
- `version: str` — Plugin version
- `entry_point: str` — Import path or file path
- `config: dict` — Plugin configuration
- `enabled: bool` — Whether plugin is active
```python
@dataclass
class PluginInterface:
plugin_id: UUID
name: str
agent_type: str
version: str
entry_point: str
config: dict = field(default_factory=dict)
enabled: bool = True
```
---
### TOINTenant
A tenant configuration for TOIN (Tenant-specific ONNX).
**Fields:**
- `tenant_id: UUID` — Unique identifier
- `name: str` — Tenant name
- `model_path: str | Path` — Path to ONNX model
- `model_version: str` — Model version
- `config: dict` — Tenant-specific configuration
- `created_at: datetime` — Creation time
- `updated_at: datetime` — Last modification
```python
@dataclass
class TOINTenant:
tenant_id: UUID
name: str
model_path: str | Path
model_version: str
config: dict = field(default_factory=dict)
created_at: datetime = field(default_factory=datetime.utcnow)
updated_at: datetime = field(default_factory=datetime.utcnow)
```
---
### CCRContext
Claude Code Relay context tracking.
**Fields:**
- `context_id: UUID` — Unique identifier
- `session_id: UUID` — Associated session
- `agent_type: str` — "claude", "claude-desktop", etc.
- `window_start: datetime` — Context window start
- `window_end: datetime` — Context window end
- `messages_tracked: int` — Number of messages
- `compressed_content: str | None` — Compressed context
```python
@dataclass
class CCRContext:
context_id: UUID
session_id: UUID
agent_type: str
window_start: datetime
window_end: datetime
messages_tracked: int = 0
compressed_content: str | None = None
```
---
### SubscriptionState
Subscription/quota tracking state.
**Fields:**
- `subscription_id: UUID` — Unique identifier
- `provider: str` — Provider name
- `plan_name: str` — Plan name
- `window_start: datetime` — Billing window start
- `window_end: datetime` — Billing window end
- `max_tokens: int | None` — Maximum tokens in window
- `tokens_used: int` — Tokens used in window
- `tokens_remaining: int | None` — Tokens remaining
- `is_active: bool` — Whether subscription is active
- `last_updated: datetime` — Last update time
```python
@dataclass
class SubscriptionState:
subscription_id: UUID
provider: str
plan_name: str
window_start: datetime
window_end: datetime
max_tokens: int | None = None
tokens_used: int = 0
tokens_remaining: int | None = None
is_active: bool = True
last_updated: datetime = field(default_factory=datetime.utcnow)
```
---
## Relationships
```
Session 1───N Message
Session 1───N Request
Request 1───1 Savings
Request 1───1 CacheEntry
Session 1───N CCRContext
TOINTenant 1───N Request
ToolDefinition N───N Session (via tool_use)
```
---
## Value Objects
### CompressResult
```python
@dataclass
class CompressResult:
messages: list[dict]
tokens_before: int
tokens_after: int
tokens_saved: int
compression_ratio: float
transforms_applied: list[str]
content_type: ContentType
model: str | None
ccr_hash: str | None
cached: bool
```
### ContentType (Enum)
```python
class ContentType(Enum):
PLAINTEXT = "text/plain"
MARKDOWN = "text/markdown"
JSON = "application/json"
HTML = "text/html"
XML = "text/xml"
PYTHON = "text/x-python"
JAVASCRIPT = "text/javascript"
TYPESCRIPT = "text/typescript"
YAML = "text/yaml"
MARKDOWN_SNAPSHOT = "text/markdown-snapshot"
UNKNOWN = "application/octet-stream"
```
### ProxyMode (Enum)
```python
class ProxyMode(Enum):
PASSTHROUGH = "passthrough"
COMPRESS = "compress"
LEARN = "learn"
DETACHED = "detached"
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial domain model |

353
docs/spec/005-integrations.md
View file

@ -1,353 +0,0 @@
# 005. Integrations
**Status:** done
## Supported Agents
### Claude (`headroom/learn/plugins/claude/`)
**Plugin:** `ClaudeLearnPlugin`
**Capabilities:**
- Session branch comparison
- Token headroom mode detection
- Tool use tracking
- Multi-modal support (images)
**Interface:**
```python
class ClaudeLearnPlugin(LearnPlugin, ConversationScanner):
@property
def name(self) -> str:
return "claude"
@property
def display_name(self) -> str:
return "Claude Code"
def detect(self) -> bool:
"""Check if Claude Code has data on the current machine."""
pass
def discover_projects(self) -> list[ProjectInfo]:
"""Discover all projects with Claude sessions."""
pass
def scan_project(self, project: ProjectInfo, max_workers: int = 1) -> list[SessionData]:
"""Scan all sessions for a project."""
pass
def create_writer(self) -> ContextWriter:
"""Return Claude-specific ContextWriter."""
pass
```
**Configuration:**
```bash
HEADROOM_LEARN_CLI=claude
```
---
### Codex (OpenAI) (`headroom/learn/plugins/codex/`)
**Plugin:** `CodexLearnPlugin`
**Capabilities:**
- Rate limit handling
- Code completion optimization
- Batch request support
**Interface:**
```python
class CodexLearnPlugin(LearnPlugin, ConversationScanner):
@property
def name(self) -> str:
return "codex"
@property
def display_name(self) -> str:
return "OpenAI Codex"
def detect(self) -> bool:
pass
def discover_projects(self) -> list[ProjectInfo]:
pass
def scan_project(self, project: ProjectInfo, max_workers: int = 1) -> list[SessionData]:
pass
def create_writer(self) -> ContextWriter:
pass
```
**Configuration:**
```bash
HEADROOM_LEARN_CLI=codex
```
---
### Gemini (Google) (`headroom/learn/plugins/gemini/`)
**Plugin:** `GeminiLearnPlugin`
**Capabilities:**
- Multimodal inputs
- Function calling support
- Context caching API
**Interface:**
```python
class GeminiLearnPlugin(LearnPlugin, ConversationScanner):
@property
def name(self) -> str:
return "gemini"
@property
def display_name(self) -> str:
return "Google Gemini"
def detect(self) -> bool:
pass
def discover_projects(self) -> list[ProjectInfo]:
pass
def scan_project(self, project: ProjectInfo, max_workers: int = 1) -> list[SessionData]:
pass
def create_writer(self) -> ContextWriter:
pass
```
**Configuration:**
```bash
HEADROOM_LEARN_CLI=gemini
```
---
## Integration Points
### LiteLLM Callback (`headroom/integrations/litellm_callback.py`)
LiteLLM proxy callback for integrating with LiteLLM-based setups.
**`LiteLLMCallback` class:**
```python
class LiteLLMCallback:
def __init__(
self,
headroom_url: str = "http://localhost:8787",
api_key: str | None = None,
) -> None:
self.headroom_url = headroom_url
self.api_key = api_key
def on_completion(self, completion_response: dict) -> dict:
"""Called after completion. Can modify response."""
pass
def on_error(self, error: Exception) -> None:
"""Called on error."""
pass
```
**Usage:**
```python
from headroom.integrations import LiteLLMCallback
callback = LiteLLMCallback(headroom_url="http://localhost:8787")
# Register with LiteLLM proxy
```
---
### ASGI Middleware (`headroom/integrations/asgi.py`)
ASGI-compatible middleware for Python web frameworks (FastAPI, Starlette, etc.).
**`HeadroomMiddleware` class:**
```python
class HeadroomMiddleware:
def __init__(
self,
app: ASGIApplication,
headroom_url: str = "http://localhost:8787",
mode: ProxyMode = ProxyMode.COMPRESS,
) -> None:
self.app = app
self.headroom_url = headroom_url
self.mode = mode
async def __call__(
self,
scope: Scope,
receive: Receive,
send: Send,
) -> None:
"""ASGI application interface."""
pass
```
**Usage:**
```python
from headroom.integrations import HeadroomMiddleware
from fastapi import FastAPI
app = FastAPI()
app.add_middleware(
HeadroomMiddleware,
headroom_url="http://localhost:8787",
mode=ProxyMode.COMPRESS,
)
```
---
### MCP integration helpers (`headroom/integrations/mcp/server.py`)
Helpers for MCP-aware host applications and custom wrappers. This module does
not currently ship a standalone `HeadroomMCPProxy` server implementation.
**`HeadroomMCPCompressor` class:**
```python
class HeadroomMCPCompressor:
def __init__(
self,
config: HeadroomConfig | None = None,
profiles: list[MCPToolProfile] | None = None,
token_counter: Callable[[str], int] | None = None,
) -> None:
...
def compress(
self,
content: str,
tool_name: str,
tool_args: dict[str, Any] | None = None,
user_query: str = "",
) -> MCPCompressionResult:
"""Compress an MCP tool result."""
pass
```
**Companion helpers:**
- `compress_tool_result(...)` — standalone helper for host applications
- `HeadroomMCPClientWrapper` — wraps an MCP client and compresses tool results
- `create_headroom_mcp_proxy(...)` — returns config for a custom wrapper/proxy
**Ready-to-run MCP tools server:**
```bash
headroom mcp serve
```
---
### Strands (`headroom/integrations/strands/`)
Strands framework integration.
**Usage:**
```python
from headroom.integrations.strands import HeadroomStrandsPlugin
plugin = HeadroomStrandsPlugin()
```
---
### LangChain (`headroom/integrations/langchain/`)
LangChain callback handler integration.
**`HeadroomLangChainCallback` class:**
```python
class HeadroomLangChainCallback(BaseCallbackHandler):
def __init__(
self,
headroom_url: str = "http://localhost:8787",
api_key: str | None = None,
) -> None:
self.headroom_url = headroom_url
self.api_key = api_key
async def on_llm_start(self, serialized, prompts, **kwargs) -> None:
pass
async def on_llm_end(self, response, **kwargs) -> None:
pass
```
**Usage:**
```python
from langchain.callbacks import HeadroomLangChainCallback
callback = HeadroomLangChainCallback()
# Pass to LangChain chain
```
---
## Agent Contract
All learn plugins must implement the `LearnPlugin` interface:
```python
from abc import ABC, abstractmethod
from headroom.learn.base import ConversationScanner, ContextWriter
from headroom.learn.models import ProjectInfo, SessionData
class LearnPlugin(ConversationScanner):
"""A self-contained learn plugin for a single coding agent."""
@property
@abstractmethod
def name(self) -> str:
"""Short lowercase identifier (e.g., 'claude', 'cursor')."""
...
@property
@abstractmethod
def display_name(self) -> str:
"""Human-readable name (e.g., 'Claude Code', 'Cursor')."""
...
@abstractmethod
def detect(self) -> bool:
"""Return True if this agent has data on the current machine."""
...
@abstractmethod
def discover_projects(self) -> list[ProjectInfo]:
"""Discover all projects with conversation data."""
...
@abstractmethod
def scan_project(self, project: ProjectInfo, max_workers: int = 1) -> list[SessionData]:
"""Scan all sessions for a project."""
...
@abstractmethod
def create_writer(self) -> ContextWriter:
"""Return the appropriate ContextWriter for this agent."""
...
```
**Plugin Registration:**
```python
# Module-level instance for auto-discovery
plugin = MyAgentPlugin()
```
Plugins are auto-discovered from `headroom.learn.plugins.*` or via `headroom.learn_plugin` entry points.
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial integrations document |

230
docs/spec/006-actors.md
View file

@ -1,230 +0,0 @@
# 006. Actors
**Status:** done
## Actor Types
### End User
The developer using an AI coding agent with Headroom.
**Interactions:**
- Configures Headroom via environment variables or config file
- Uses wrapped CLI commands or SDK
- Views savings in dashboard
- Optionally enables learn mode
**Needs:**
- Transparent compression (doesn't break workflows)
- Clear savings metrics
- Easy opt-out of specific features
**Configuration:**
```bash
# Minimal setup
export ANTHROPIC_API_KEY=sk-...
# Optional overrides
export HEADROOM_MODE=token
```
---
### Operator
The person deploying and managing Headroom in production.
**Interactions:**
- Deploys Headroom (Docker, native, or embedded)
- Configures profiles and presets
- Monitors health endpoints
- Reviews metrics and logs
- Manages upgrades
**Needs:**
- Clear deployment documentation
- Health and readiness checks
- Metrics for capacity planning
- Upgrade and rollback procedures
**Configuration:**
```yaml
# ~/.headroom/config.yaml
proxy:
host: 0.0.0.0
port: 8787
compression:
enabled: true
cache:
enabled: true
ttl: 3600
telemetry:
metrics:
enabled: true
```
**Health Endpoints:**
```bash
curl http://localhost:8787/health
curl http://localhost:8787/livez
curl http://localhost:8787/readyz
curl http://localhost:8787/metrics
```
---
### Plugin Author
The developer creating a custom learn plugin for a specific agent.
**Interactions:**
- Implements `LearnPlugin` interface
- Plugins auto-discovered from `headroom/learn/plugins/` directory
- Contributes to Headroom
**Needs:**
- Clear plugin interface documentation
- Example plugins to reference
- Test utilities
**Plugin Template:**
```python
from headroom.learn.base import LearnPlugin, ConversationScanner
class MyAgentPlugin(LearnPlugin):
@property
def name(self) -> str:
return "my_agent"
@property
def display_name(self) -> str:
return "My Agent"
def detect(self) -> bool:
# Check if this agent has data on the current machine
pass
def discover_projects(self) -> list[ProjectInfo]:
# Discover all projects with conversation data
pass
def scan_project(self, project: ProjectInfo, max_workers: int = 1) -> list[SessionData]:
# Scan all sessions for a project
pass
def create_writer(self) -> ContextWriter:
# Return the appropriate ContextWriter for this agent
pass
```
---
### Enterprise Evaluator
The person assessing Headroom for organizational adoption.
**Interactions:**
- Reviews security documentation
- Assesses compliance guarantees
- Evaluates operational characteristics
**Needs:**
- Clear data handling guarantees
- Security and privacy documentation
- Compliance certifications (if any)
- SOC2/GDPR considerations
**Security Configuration:**
```bash
# Maximum privacy settings
HEADROOM_TELEMETRY=off
HEADROOM_STATELESS=true
headroom proxy --no-cache --no-optimize
```
---
## Interaction Patterns
### User → Headroom
```
┌──────────────┐ ┌────────────────┐ ┌─────────────┐
│ User's AI │──────▶│ Headroom │──────▶│ Provider │
│ Agent │ │ Proxy │ │ API │
└──────────────┘ └───────┬────────┘ └─────────────┘
│
▼
┌──────────────┐
│ Dashboard │
│ (optional) │
└──────────────┘
```
**Flow:**
1. User's AI agent sends request to Headroom proxy
2. Headroom compresses context
3. Compressed request sent to provider API
4. Response returned to agent
5. Optional: savings logged to dashboard
---
### Operator → Headroom
```
┌──────────┐ ┌────────────────┐ ┌─────────────┐
│ Operator │──────▶│ Health │──────▶│ Metrics │
│ │ │ Endpoints │ │ Server │
└──────────┘ └────────────────┘ └─────────────┘
│
▼
┌────────────────┐
│ Logs │
│ (stdout/file) │
└────────────────┘
```
**Flow:**
1. Operator checks health endpoints
2. Reviews Prometheus metrics
3. Monitors logs for errors
4. Manages configuration
---
### Plugin Author → Headroom
```
┌──────────────┐ ┌────────────────┐ ┌─────────────┐
│ Plugin │──────▶│ Plugin │──────▶│ Registry │
│ Author │ │ Interface │ │ + Tests │
└──────────────┘ └────────────────┘ └─────────────┘
```
**Flow:**
1. Plugin author implements LearnPlugin
2. Plugins are auto-discovered from `headroom/learn/plugins/` directory
3. Writes unit tests
4. Submits contribution
---
## Permissions Model
| Actor | Config | Read Metrics | Admin | Plugin |
|-------|--------|--------------|-------|--------|
| End User | ✓ (own) | ✓ (own) | - | - |
| Operator | ✓ (full) | ✓ (all) | ✓ | - |
| Plugin Author | - | - | - | ✓ (write) |
| Enterprise Evaluator | - | ✓ (security) | - | - |
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial actors document |

178
docs/spec/007-behavior.md
View file

@ -1,178 +0,0 @@
# 007. Behavior
**Status:** done
## Proxy Modes
### Passthrough
Headroom forwards requests without modification.
**Behavior:**
- All requests pass through unchanged
- Response headers may be modified for telemetry
- No compression applied
- Useful for testing or debugging
**Configuration:** `headroom proxy --no-optimize`
**Request Flow:**
```
Client → Proxy → Provider API → Response
```
---
### Token Mode
Headroom applies deterministic transforms to requests.
**Behavior:**
- SmartCrusher compresses JSON tool outputs
- CacheAligner stabilizes prefixes
- RollingWindow caps context tokens
- CCR caching enabled
- Token budget enforced
**Configuration:** `HEADROOM_MODE=token` or `headroom proxy --mode token`
**Request Flow:**
```
Client → Proxy → [SmartCrusher] → [CacheAligner]
→ [RollingWindow] → [CCR Cache]
→ Provider API → Response
```
---
### Cache Mode
Headroom preserves prior turns where possible to maximize provider prefix-cache hit rate.
**Behavior:**
- Freezes provider-confirmed cached prefixes
- Compresses the mutable tail of the request
- Trades some token savings for better cache stability
**Configuration:** `HEADROOM_MODE=cache` or `headroom proxy --mode cache`
---
## Session Modes
Session modes control how Headroom handles context windows.
| Mode | Description | Use Case |
|------|-------------|----------|
| `token` | Prioritize token removal | Default proxy mode |
| `cache` | Preserve prior turns for provider prefix-cache stability | Long Claude/Codex sessions |
| passthrough | Disable optimization with `--no-optimize` | Debugging |
---
## Request Lifecycle
```
1. Request received at proxy endpoint
│
▼
2. Session lookup/creation
│ - Extract session ID from headers
│ - Create new session if not found
│
▼
3. Mode determination
│ - Check HEADROOM_MODE
│ - Check runtime headers
│ - Determine active plugins
│
▼
4. Compression pipeline execution
│ a. Token counting
│ b. Semantic cache check
│ c. Content type detection
│ d. Transform selection
│ e. Summary compression (if eligible)
│ f. Token budget enforcement
│
▼
5. Forward to provider API
│ - Route to correct provider
│ - Apply API key from config
│ - Handle timeouts
│
▼
6. Response capture
│ - Log request/response metadata
│ - Calculate savings
│
▼
7. Savings calculation
│ - tokens_before - tokens_after
│ - percentage = savings / tokens_before
│
▼
8. Telemetry emission
│ - Prometheus metrics
│ - Optional tracing
│
▼
9. Response returned to client
- X-Headroom-Savings header
- X-Headroom-Original-Tokens header
- X-Headroom-Compressed-Tokens header
```
---
## Error Handling
| Error Type | HTTP Code | Behavior |
|------------|----------|----------|
| Provider timeout | 504 | Retry up to 3 times with exponential backoff |
| Invalid request | 400 | Return error details in body |
| Compression failure | 500 | Fall back to passthrough mode |
| Provider error | Provider code | Return provider error to client |
| Internal error | 500 | Return 500, log details |
| Rate limited | 429 | Return retry-after header |
**Retry Configuration:**
```python
@dataclass
class RetryConfig:
max_retries: int = 3
base_delay: float = 1.0
max_delay: float = 60.0
exponential_base: float = 2.0
```
---
## Response Headers
Headroom adds headers to all compressed responses:
```
X-Headroom-Savings: 0.35
X-Headroom-Original-Tokens: 8192
X-Headroom-Compressed-Tokens: 5325
X-Headroom-Compression-Type: semantic,summary
X-Headroom-Request-Id: abc123
X-Headroom-Cache-Hit: false
```
**Header Descriptions:**
- `X-Headroom-Savings` — Token savings percentage (0.35 = 35%)
- `X-Headroom-Original-Tokens` — Token count before compression
- `X-Headroom-Compressed-Tokens` — Token count after compression
- `X-Headroom-Compression-Type` — Types of compression applied
- `X-Headroom-Request-Id` — Unique request identifier
- `X-Headroom-Cache-Hit` — Whether result was from cache
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial behavior document |

254
docs/spec/008-capabilities.md
View file

@ -1,254 +0,0 @@
# 008. Capabilities
**Status:** done
## Feature Matrix
| Capability | Proxy | SDK | Wrap | MCP | ASGI | LiteLLM |
|------------|:-----:|:---:|:----:|:---:|:----:|:-------:|
| Semantic compression | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Summary compression | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Token budget management | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Learn mode | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Session tracking | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| CCR feedback | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| TOIN tagging | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Savings tracking | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Dashboard | ✓ | - | ✓ | - | - | - |
| Health endpoints | ✓ | - | - | - | - | - |
| Metrics (Prometheus) | ✓ | - | - | - | - | - |
---
## Compression Capabilities
### Proxy Cache
**Description:** The proxy has semantic cache support and CCR-backed retrieval, controlled by CLI configuration.
**Configuration:**
```bash
headroom proxy # cache enabled by default
headroom proxy --no-cache
```
**Behavior:**
1. Hash input content
2. Check cache for similar entries (configurable threshold)
3. Return cached response if match found (CCR retrieval)
4. Update cache on miss
5. Store compressed content for future retrieval
**CCR Configuration:**
```python
@dataclass
class CCRConfig:
enabled: bool = True
storage: ContentStorage | None = None
cache_ttl: int = 3600
similarity_threshold: float = 0.85
max_results: int = 10
```
---
### Summary Compression
**Description:** Compresses long context using summarization.
**Configuration:**
```bash
# Note: HEADROOM_SUMMARY_* env vars are not yet implemented.
# Summary compression is currently configured programmatically only.
```
**Behavior:**
1. Count input tokens
2. If over threshold, apply summary compression
3. Preserve key information (configurable priority)
4. Track summary statistics
**Summary Pipeline:**
```python
class SummaryCompressor:
def __init__(
self,
threshold: int = 5000,
target_ratio: float = 0.3,
priority_preservation: bool = True,
) -> None:
pass
def compress(
self,
content: str,
context: TransformContext,
) -> TransformResult:
pass
```
---
### Token Budget Manager
**Description:** Enforces token budget limits per session via RollingWindow.
**Configuration:**
```python
@dataclass
class RollingWindowConfig:
enabled: bool = True
keep_system: bool = True # Never drop system prompt
keep_last_turns: int = 2 # Never drop last N turns
output_buffer_tokens: int = 4000 # Reserve for output
```
**Behavior:**
1. Track token usage per session
2. Enforce budget limits via rolling window
3. Preserve system prompt and last N turns
4. Reserve output buffer tokens
---
## Intelligent Context Management
**Description:** Semantic-aware context management with TOIN integration.
```python
@dataclass
class IntelligentContextConfig:
enabled: bool = True
use_importance_scoring: bool = True
scoring_weights: ScoringWeights = field(default_factory=ScoringWeights)
toin_integration: bool = True
toin_confidence_threshold: float = 0.3
```
**Scoring Weights:**
```python
@dataclass
class ScoringWeights:
recency: float = 0.20
semantic_similarity: float = 0.20
toin_importance: float = 0.25
error_indicator: float = 0.15
forward_reference: float = 0.15
token_density: float = 0.05
```
---
## CCR Configuration
**Description:** Compress-Cache-Retrieve makes compression reversible.
```python
@dataclass
class CCRConfig:
enabled: bool = True
store_max_entries: int = 1000
store_ttl_seconds: int = 300
inject_retrieval_marker: bool = True
feedback_enabled: bool = True
min_items_to_cache: int = 20
inject_tool: bool = True
inject_system_instructions: bool = False
```
---
## SmartCrusher (Statistical JSON Compression)
**Description:** Preserves JSON schema while reducing array size via statistical analysis.
```python
@dataclass
class SmartCrusherConfig:
enabled: bool = True
min_items_to_analyze: int = 5
min_tokens_to_crush: int = 200
variance_threshold: float = 2.0
uniqueness_threshold: float = 0.1
similarity_threshold: float = 0.8
max_items_after_crush: int = 15
preserve_change_points: bool = True
use_feedback_hints: bool = True
toin_confidence_threshold: float = 0.3
relevance: RelevanceScorerConfig = field(default_factory=RelevanceScorerConfig)
anchor: AnchorConfig = field(default_factory=AnchorConfig)
dedup_identical_items: bool = True
first_fraction: float = 0.3
last_fraction: float = 0.15
```
**Relevance Scoring:**
```python
@dataclass
class RelevanceScorerConfig:
tier: Literal["bm25", "embedding", "hybrid"] = "hybrid"
bm25_k1: float = 1.5
bm25_b: float = 0.75
embedding_model: str = field(default_factory=lambda: ML_MODEL_DEFAULTS.sentence_transformer)
hybrid_alpha: float = 0.5
adaptive_alpha: bool = True
relevance_threshold: float = 0.25
```
**Anchor Allocation:**
```python
@dataclass
class AnchorConfig:
anchor_budget_pct: float = 0.25
min_anchor_slots: int = 3
max_anchor_slots: int = 12
default_front_weight: float = 0.5
default_back_weight: float = 0.4
default_middle_weight: float = 0.1
search_front_weight: float = 0.75
logs_back_weight: float = 0.75
```
---
## Agent-Specific Capabilities
### Claude
- Session branch comparison
- Token headroom mode
- Tool use tracking
- Multi-modal support (images)
### Codex
- Rate limit handling
- Code completion optimization
- Batch request support
### Gemini
- Multi-modal inputs
- Function calling support
- Context caching API
---
## Deployment Capabilities
| Feature | Docker | Native | Embedded |
|---------|:------:|:------:|:--------:|
| Standalone proxy | ✓ | ✓ | - |
| Health endpoints | ✓ | ✓ | - |
| Prometheus metrics | ✓ | ✓ | - |
| Dashboard | ✓ | ✓ | - |
| Volume mounts | ✓ | - | - |
| Environment overrides | ✓ | ✓ | ✓ |
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial capabilities document |

82
docs/spec/009-compliance.md
View file

@ -1,82 +0,0 @@
# 009. Compliance
**Status:** done
## Data Handling Guarantees
### Default Behavior (No Export)
| Data Type | Stored | Exported | Logged |
|-----------|:------:|:--------:|:------:|
| Prompts | Optional (cache) | Never | Never |
| Responses | Optional (cache) | Never | Never |
| Savings metrics | Yes | Never | Never |
| Session metadata | Yes | Never | Never |
| Telemetry | Aggregated | Never | Never |
---
### With Exporters Configured
**Warning:** Enabling exporters may send data outside your infrastructure.
| Exporter | Data Sent | Destination |
|----------|-----------|-------------|
| Prometheus | Metrics only | Prometheus server |
| OpenTelemetry | Traces/spans | OTLP endpoint |
| Custom webhook | Configurable | HTTP endpoint |
---
## Privacy Commitments
1. **No prompt logging by default** — Headroom never logs prompt content unless explicitly configured
2. **No data leaves proxy by default** — All processing happens locally
3. **User control** — All data handling is configurable via environment variables
4. **Transparency** — Response headers indicate compression was applied
5. **No telemetry to Headroom project** — No data sent to external servers without explicit opt-in
---
## Compliance Considerations
### SOC 2
*To be documented if applicable.*
### GDPR
| Requirement | Headroom Support |
|-------------|:----------------:|
| Data minimization | ✓ Default no-logging |
| Right to deletion | ✓ Cache can be cleared |
| Data portability | Export available via API |
| Breach notification | N/A (no external data) |
### HIPAA
*To be documented if applicable.*
---
## Configuration for Maximum Privacy
```bash
HEADROOM_TELEMETRY=off
HEADROOM_STATELESS=true
headroom proxy --no-cache --no-optimize
```
This configuration results in:
- No prompt data stored
- No data exported
- No analytics collected
- Headroom acts as a passthrough proxy
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial compliance document |

174
docs/spec/010-data.md
View file

@ -1,174 +0,0 @@
# 010. Data
**Status:** done
## Storage Overview
| Store | Location | Type | Purpose |
|-------|----------|------|---------|
| SQLite | `headroom.db` (cwd or `~/.headroom/`) | Local file | Metrics, sessions, compression store |
| Memory DB | `~/.headroom/headroom_memory.db` | Local file | Memory system (optional) |
| Vector Index | `~/.headroom/headroom_memory_vectors.db` | Local file | Semantic search (optional, requires sqlite-vec) |
| Graph Store | `~/.headroom/headroom_memory_graph.db` | Local file | Memory relationships |
| Compression cache | `~/.headroom/cache/` | Directory | Semantic + summary cache |
**Note:** Default store URL is `sqlite:///headroom.db` (relative to working directory).
---
## SQLite Schema
### Core Tables
**sessions:**
```sql
CREATE TABLE sessions (
session_id TEXT PRIMARY KEY,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
metadata TEXT -- JSON
);
```
**requests:**
```sql
CREATE TABLE requests (
id TEXT PRIMARY KEY,
timestamp TEXT NOT NULL,
model TEXT NOT NULL,
stream INTEGER NOT NULL,
mode TEXT NOT NULL,
tokens_input_before INTEGER NOT NULL,
tokens_input_after INTEGER NOT NULL,
tokens_output INTEGER,
block_breakdown TEXT NOT NULL, -- JSON
waste_signals TEXT NOT NULL, -- JSON
stable_prefix_hash TEXT,
cache_alignment_score REAL,
cached_tokens INTEGER,
transforms_applied TEXT NOT NULL, -- JSON
tool_units_dropped INTEGER DEFAULT 0,
turns_dropped INTEGER DEFAULT 0,
messages_hash TEXT,
error TEXT
);
CREATE INDEX idx_timestamp ON requests(timestamp);
CREATE INDEX idx_model ON requests(model);
CREATE INDEX idx_mode ON requests(mode);
```
**cache_entries:**
```sql
CREATE TABLE cache_entries (
cache_key TEXT PRIMARY KEY,
input_hash TEXT NOT NULL,
output TEXT NOT NULL,
compression_type TEXT NOT NULL,
tokens_before INTEGER NOT NULL,
tokens_after INTEGER NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
ttl INTEGER DEFAULT 3600,
hit_count INTEGER DEFAULT 0,
last_accessed TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```
**compression_store (CCR):**
```sql
CREATE TABLE compression_store (
hash TEXT PRIMARY KEY,
original TEXT NOT NULL,
metadata TEXT, -- JSON
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL
);
```
---
## Environment Variables
### Database
| Variable | Default | Description |
|----------|---------|-------------|
| `HEADROOM_WORKSPACE_DIR` | `~/.headroom` | Workspace root; all DBs live under this directory |
| `HEADROOM_CONFIG_DIR` | `~/.headroom/config` | Config root (read-mostly: models.json, per-plugin config) |
### Cache
| Variable | Default | Description |
|----------|---------|-------------|
| CLI `--no-cache` | unset | Disable semantic cache for the proxy process |
| `HEADROOM_WORKSPACE_DIR` | `~/.headroom` | Workspace root for proxy state, logs, memory, and savings |
| `HEADROOM_STATELESS` | `false` | Disable filesystem writes and keep runtime state in memory |
---
## Data Retention
| Data Type | Retention | Auto-cleanup |
|-----------|-----------|:------------:|
| Savings history | Forever | No |
| Session history | 30 days | Yes (configurable) |
| Compression cache | 7 days | Yes |
| Telemetry | 90 days | Yes |
| Dashboard state | 30 days | Yes |
---
## Data Export
### Savings Export
```bash
curl http://localhost:8787/stats
```
Response:
```json
{
"savings": [
{
"date": "2026-04-16",
"original_tokens": 8192,
"compressed_tokens": 5325,
"savings_percentage": 0.35
}
],
"total_savings": 1234567
}
```
### Session Export
```bash
curl http://localhost:8787/stats
```
---
## Backup
### Manual Backup
```bash
tar -czf headroom-backup.tar.gz ~/.headroom/
```
### Storage Location
Relocate all storage by setting the workspace root:
```bash
export HEADROOM_WORKSPACE_DIR=/mnt/state
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial data document |

177
docs/spec/011-deployment.md
View file

@ -1,177 +0,0 @@
# 011. Deployment
**Status:** done
## Deployment Profiles
### Docker Profile
**Image:** `headroom-ai/headroom:latest`
**Dockerfile:**
```dockerfile
FROM python:3.12-slim
RUN pip install headroom-ai
EXPOSE 8787
ENTRYPOINT ["headroom", "proxy"]
CMD ["--host", "0.0.0.0", "--port", "8787"]
```
**docker-compose.yml:**
```yaml
version: '3.8'
services:
headroom:
image: headroom-ai/headroom:latest
ports:
- "8787:8787"
environment:
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
- HEADROOM_MODE=token
volumes:
- headroom-data:/root/.headroom
volumes:
headroom-data:
```
**Run:**
```bash
docker-compose up -d
```
---
### Native Profile
**Installation:**
```bash
pip install headroom-ai
```
**Run:**
```bash
headroom proxy --host 0.0.0.0 --port 8787
```
---
### Embedded Profile
**Usage:**
```python
from headroom import HeadroomClient
client = HeadroomClient(
api_key="your-api-key",
base_url="http://localhost:8787"
)
result = await client.compress(messages)
```
---
## Cloud Presets
### AWS (EC2/ECS)
```yaml
# ~/.headroom/config.yaml
deployment:
profile: aws
instance_type: t3.medium
compression:
enabled: true
max_tokens: 8192
cache:
backend: redis
redis_url: redis://localhost:6379
```
### Google Cloud (Cloud Run)
```yaml
deployment:
profile: gcp
region: us-central1
memory: 512Mi
cpu: 1
```
### Azure (Container Apps)
```yaml
deployment:
profile: azure
resource_group: headroom-rg
```
---
## Runtime Configuration
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `HEADROOM_MODE` | `token` | Proxy mode (`token` or `cache`) |
| `HEADROOM_PORT` | `8787` | Proxy port |
| `HEADROOM_HOST` | `127.0.0.1` | Proxy host |
| `ANTHROPIC_API_KEY` | - | Anthropic API key |
| `OPENAI_API_KEY` | - | OpenAI API key |
| `HEADROOM_TELEMETRY` | `off` (opt-in) | Set to `on` to opt in to telemetry |
### Config File
```yaml
# ~/.headroom/config.yaml
proxy:
host: 0.0.0.0
port: 8787
compression:
enabled: true
max_tokens: 4096
overlap_tokens: 512
content_sensitivity: 0.5
preserve_system_messages: true
priority_tokens: 1024
cache:
enabled: true
ttl: 3600
max_size: 10000
telemetry:
metrics:
enabled: true
tracing:
enabled: false
learn:
enabled: false
```
---
## Resource Requirements
| Deployment | CPU | Memory | Storage |
|------------|-----|--------|---------|
| Minimal | 0.5 core | 512MB | 1GB |
| Default | 1 core | 1GB | 5GB |
| Enterprise | 2 cores | 2GB | 20GB |
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial deployment document |

140
docs/spec/012-diagrams.md
View file

@ -1,140 +0,0 @@
# 012. Diagrams
**Status:** done
## Component Diagram
```
┌─────────────────────────────────────────────────────────────────┐
│ Headroom │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌─────────────┐ │
│ │ Proxy │ │ SDK │ │ Wrap │ │ MCP │ │
│ │ Server │ │ (Python) │ │ CLI │ │ Server │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └──────┬──────┘ │
│ │ │ │ │ │
│ ┌────┴──────────────┴──────────────┴─────────────────┴──────┐ │
│ │ Compression Layer │ │
│ │ ┌────────────┐ ┌────────────┐ ┌────────────────────┐ │ │
│ │ │ Semantic │ │ Summary │ │ Token Budget │ │ │
│ │ │ Cache │ │ Compress │ │ Manager │ │ │
│ │ └────────────┘ └────────────┘ └────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ Learn System │ │
│ │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────────────┐ │ │
│ │ │Claude │ │ Codex │ │ Gemini │ │ ...more │ │ │
│ │ │Plugin │ │ Plugin │ │ Plugin │ │ plugins │ │ │
│ │ └────────┘ └────────┘ └────────┘ └────────────────┘ │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ Dashboard │ │ CCR │ │ TOIN │ │
│ │ (Next.js) │ │ (Claude │ │ (Tenant-specific │ │
│ │ │ │ Code Relay)│ │ ONNX) │ │
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
---
## Request Flow Sequence Diagram
```
Client Headroom Compression CCR Provider
Proxy Layer Cache API
│ │ │ │ │
│──POST /v1/msg──▶│ │ │ │
│ │ │ │ │
│ │──Count tokens──▶│ │ │
│ │ │ │ │
│ │────Cache check────▶│ │ │
│ │ │◀───hit──────│ │
│ │ │◀───CCR retrieval───────│
│ │ │ │ │
│ │───Summary compression───▶│ │
│ │ │◀──budget enforcement─────│
│ │ │ │ │
│ │─────Forward compressed─────▶│ │
│ │ │ │◀──Response──│
│ │◀──────────────────────────────────────│
│◀──Response──│ │ │ │
```
---
## CCR Cache Flow
```
Request Headroom CCR Storage
│ Proxy Cache
│ │ │
│──compress──▶│ │
│ │──hash────▶│
│ │ │──store──▶│
│ │◀──stored──│ │
│ │──CCR hash │
│◀──Response──│ │
│ │ │
```
---
## Learn System Flow
```
Session Headroom Learn Plugin
Data Proxy System Registry
│ │ │ │
│──session──▶│ │ │
│ │──analyze──▶│ │
│ │ │──load────▶│
│ │ │◀─Result───│
│ │◀──Feedback──│ │
│ │──Improve compression │
│◀───────────│ │ │
```
---
## Data Flow
```
┌──────────────┐
│ User's │
│ AI Agent │
└──────┬───────┘
│ HTTP
▼
┌──────────────┐ ┌─────────────────────────────────────┐
│ Headroom │────▶│ Compression Layer │
│ Proxy │ │ │
└──────┬───────┘ │ ┌─────────┐ ┌──────────────┐ │
│ │ │ Semantic│ │ Summary │ │
│ │ │ Cache │ │ Compression │ │
│ │ └────┬───┘ └──────┬───────┘ │
│ │ │ │ │
│ │ ┌────┴─────────────┴──────┐ │
│ │ │ Token Budget │ │
│ │ │ Manager │ │
│ │ └────────────┬───────────┘ │
│ └───────────────┼─────────────────┘
│ │
│ │ Compressed request
▼ ▼
┌──────────────┐ ┌──────────────┐
│ Dashboard │ │ Provider │
│ (optional) │ │ API │
└──────────────┘ └──────────────┘
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial diagrams document |

159
docs/spec/013-disaster-recovery.md
View file

@ -1,159 +0,0 @@
# 013. Disaster Recovery
**Status:** done
## Failure Modes
### Proxy Failures
| Failure | Impact | Recovery |
|---------|--------|----------|
| Proxy crash | Service unavailable | Restart or failover |
| OOM | Service unavailable | Restart with more memory |
| Network partition | Cannot reach providers | Retry with backoff |
### Compression Failures
| Failure | Impact | Recovery |
|---------|--------|----------|
| Compression timeout | Request delayed | Retry with passthrough |
| Transform error | Compression skipped | Log error, passthrough |
| Budget exceeded | Truncation | Notify via headers |
### Storage Failures
| Failure | Impact | Recovery |
|---------|--------|----------|
| SQLite corruption | Data loss | Restore from backup |
| Cache full | CCR disabled | Clear old entries |
| Disk full | Write failures | Expand storage |
---
## Backup Strategies
### Manual Backup
```bash
# Full backup
tar -czf headroom-backup-$(date +%Y%m%d).tar.gz ~/.headroom/
# Incremental (last 24h)
sqlite3 ~/.headroom/headroom_memory.db ".backup /tmp/headroom_incremental.db"
```
### Automated Backup
```bash
# Cron job (daily at 2am)
0 2 * * * tar -czf /backup/headroom-$(date +\%Y\%m\%d).tar.gz ~/.headroom/
```
### External Storage
> **Note:** External PostgreSQL/Redis storage is not yet implemented. Headroom uses SQLite at `~/.headroom/` (configurable via `HEADROOM_WORKSPACE_DIR`). The `HEADROOM_DB_URL` and `HEADROOM_CACHE_BACKEND` vars do not exist.
---
## Recovery Procedures
### Proxy Recovery
1. **Restart proxy:**
```bash
# Docker
docker-compose restart headroom
# Native
pkill headroom && headroom proxy &
```
2. **Check health:**
```bash
curl http://localhost:8787/health
curl http://localhost:8787/readyz
```
### Database Recovery
1. **Restore from backup:**
```bash
# Stop headroom
pkill headroom
# Restore SQLite
rm ~/.headroom/headroom_memory.db
tar -xzf headroom-backup-20260416.tar.gz -C ~/
# Restart headroom
headroom proxy &
```
2. **Verify data:**
```bash
curl http://localhost:8787/stats
```
---
## Data Migration
### Schema Migration
```python
async def migrate_v1_to_v2():
async with aiosqlite.connect(DB_PATH) as db:
await db.execute("""
ALTER TABLE sessions
ADD COLUMN agent_version TEXT
""")
await db.commit()
```
### Data Export/Import
```bash
# Export
curl http://localhost:8787/api/v1/export > backup.json
# Import
curl -X POST http://localhost:8787/api/v1/import \
-H "Content-Type: application/json" \
-d @backup.json
```
---
## High Availability
### Active-Active
```yaml
services:
headroom-1:
image: headroom-ai/headroom:latest
ports:
- "8787:8787"
headroom-2:
image: headroom-ai/headroom:latest
ports:
- "8788:8787"
redis:
image: redis:latest
```
### Health Check Failover
```bash
curl http://primary:8787/health || curl http://backup:8787/health
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial disaster recovery document |

129
docs/spec/014-governance.md
View file

@ -1,129 +0,0 @@
# 014. Governance
**Status:** done
## Project Structure
- **Repository:** `github.com/JerrettDavis/headroom`
- **License:** Apache 2.0
- **Main Languages:** Python (core), TypeScript (SDK/dashboard)
---
## Release Cadence
| Channel | Frequency | Stability |
|---------|-----------|-----------|
| Stable | Monthly | Production-ready |
| Beta | As needed | May have issues |
| Nightly | Daily | Unstable |
---
## Semantic Versioning
Headroom follows semver:
- **MAJOR:** Breaking changes to API, CLI, or core behavior
- **MINOR:** New features, backwards-compatible
- **PATCH:** Bug fixes, backwards-compatible
---
## Decision Making
### RFC Process
1. Open GitHub issue with `[RFC]` prefix
2. Gather community feedback (2 weeks minimum)
3. Core team reviews
4. Decision documented in ADRs
5. Implementation proceeds
### Criteria for Core Team Approval
- Consistent with project vision
- Does not break existing guarantees
- Implementation is feasible
- Tests can be written
---
## Spec-Driven Development
This specification is the **canonical source of truth** for Headroom behavior:
| Rule | Description |
|------|-------------|
| **Canonical** | When code and spec diverge, the spec is the target |
| **Living** | Spec updates required for behavior-changing changes |
| **Comprehensive** | Spec covers every user-visible surface |
| **Language-agnostic** | Enables complete rewrite in any language |
### Spec Change Process
1. Propose change in GitHub issue
2. Discuss in RFC format
3. Update relevant spec section
4. Update SPEC.md version and change log
5. Implement code change
6. Verify implementation matches spec
---
## Supply Chain Posture
| Component | Policy |
|-----------|--------|
| Dependencies | Pin versions, audit regularly |
| Build artifacts | Reproducible builds |
| Signing | Code signed for releases |
| Vulnerability reporting | See SECURITY.md |
---
## Contributing
### Development Setup
```bash
# Clone repository
git clone https://github.com/JerrettDavis/headroom.git
cd headroom
# Create virtual environment
python -m venv venv
source venv/bin/activate
# Install dependencies
pip install -e ".[dev]"
# Run tests
pytest
```
### Code Style
- Format: `ruff format`
- Lint: `ruff check`
- Type check: `mypy`
---
## Code of Conduct
See `CODE_OF_CONDUCT.md`.
---
## Security
See `SECURITY.md` for vulnerability reporting and response timeline.
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial governance document |

351
docs/spec/015-interfaces.md
View file

@ -1,351 +0,0 @@
# 015. Interfaces
**Status:** done
## CLI Surface
### `headroom proxy`
Start the Headroom proxy server.
```bash
headroom proxy [OPTIONS]
```
**Options:**
| Flag | Default | Description |
|------|---------|-------------|
| `--host` | `127.0.0.1` | Bind host |
| `--port` | `8787` | Bind port |
| `--mode` | `token` | Optimization mode: `token` or `cache` |
| `--workers` | `1` | Uvicorn worker processes |
| `--limit-concurrency` | `1000` | Maximum concurrent connections before 503 |
| `--no-optimize` | `false` | Passthrough mode |
| `--no-cache` | `false` | Disable semantic cache |
| `--no-rate-limit` | `false` | Disable rate limiting |
| `--memory` | `false` | Enable persistent memory |
| `--learn` | `false` | Enable live traffic learning |
| `--backend` | `anthropic` | Backend: anthropic, bedrock, openrouter, anyllm, or litellm-* |
| `--telemetry` | `false` | Opt in to anonymous telemetry (off by default) |
| `--no-telemetry` | `false` | Force anonymous telemetry off (already the default) |
| `--stateless` | `false` | Disable filesystem writes |
---
### `headroom evals`
Run evaluation suite.
```bash
headroom evals [OPTIONS]
```
**Options:**
| Flag | Default | Description |
|------|---------|-------------|
| `--suite` | `all` | Evaluation suite to run |
| `--output` | - | Output file for results |
---
### `headroom install`
Install agent integrations.
```bash
headroom install [OPTIONS]
```
**Options:**
| Flag | Default | Description |
|------|---------|-------------|
| `--agent` | - | Agent type (claude/copilot/codex/aider/cursor/openclaw) |
---
### `headroom mcp`
Manage the Headroom MCP server.
```bash
headroom mcp [OPTIONS] COMMAND [ARGS]...
```
**Commands:**
- `install` — Install the MCP server into detected coding agents
- `serve` — Start the stdio MCP server
- `status` — Check configuration status
- `uninstall` — Remove Headroom MCP config
---
### `headroom perf`
Run performance tests.
```bash
headroom perf [OPTIONS]
```
---
### `headroom wrap`
Wrap a command with Headroom proxy.
```bash
headroom wrap [OPTIONS] -- <command> [args...]
```
**Options:**
| Flag | Default | Description |
|------|---------|-------------|
| `--port` | `8787` | Proxy port |
| `--no-context-tool` / `--no-rtk` | `false` | Skip CLI context-tool setup |
**Supported Commands:**
- `claude` — Wrap Claude Code
- `copilot` — Wrap GitHub Copilot
- `codex` — Wrap OpenAI Codex
- `aider` — Wrap Aider
- `cursor` — Wrap Cursor
- `openclaw` — Wrap OpenClaw
---
### `headroom memory`
Memory system management (requires numpy/hnswlib).
```bash
headroom memory [OPTIONS]
```
**Commands:**
- `list` — List stored memories
- `stats` — Show memory statistics
- `search QUERY` — Search memories
---
### `headroom learn`
Run learn mode analysis.
```bash
headroom learn [OPTIONS]
```
**Options:**
| Flag | Default | Description |
|------|---------|-------------|
| `--project` | current directory | Project directory to analyze |
| `--all` | `false` | Analyze all discovered projects |
| `--apply` | `false` | Write recommendations instead of dry-run |
| `--agent` | `auto` | Agent to analyze: auto, claude, codex, gemini, or plugin |
| `--model` | auto | LLM model for analysis |
| `--workers` | auto | Parallel workers for session scanning |
---
### `headroom stats`
Show savings statistics.
```bash
headroom stats [OPTIONS]
```
**Options:**
| Flag | Default | Description |
|------|---------|-------------|
| `--period` | `24h` | Time period |
| `--format` | `table` | Output format (table, json, csv) |
---
### `headroom config`
Manage configuration.
```bash
headroom config [COMMAND] [OPTIONS]
```
**Commands:**
- `get KEY` — Get config value
- `set KEY VALUE` — Set config value
- `list` — List all config
- `export` — Export config to file
---
## HTTP API
### Endpoints
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/health` | Health check |
| `GET` | `/livez` | Liveness check |
| `GET` | `/readyz` | Readiness check |
| `POST` | `/v1/messages` | Proxy chat completions |
| `POST` | `/v1/embeddings` | Proxy embeddings |
| `POST` | `/v1/compress` | Direct compression |
| `POST` | `/v1/retrieve` | CCR retrieval |
| `GET` | `/stats` | Compression statistics |
| `GET` | `/metrics` | Prometheus metrics |
### Request/Response Examples
**POST /v1/messages:**
```bash
curl -X POST http://localhost:8787/v1/messages \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-..." \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Hello"}]
}'
```
**Response headers:**
```
X-Headroom-Savings: 0.35
X-Headroom-Original-Tokens: 8192
X-Headroom-Compressed-Tokens: 5325
```
---
## Environment Variables
### Core
| Variable | Default | Description |
|----------|---------|-------------|
| `HEADROOM_MODE` | `token` | Proxy optimization mode (`token` or `cache`) |
| `HEADROOM_PORT` | `8787` | Proxy port |
| `HEADROOM_HOST` | `127.0.0.1` | Proxy host |
| `HEADROOM_WORKERS` | `1` | Uvicorn worker count |
| `HEADROOM_LIMIT_CONCURRENCY` | `1000` | Maximum concurrent connections before 503 |
| `HEADROOM_MAX_CONNECTIONS` | `500` | Maximum upstream HTTP connections |
| `HEADROOM_MAX_KEEPALIVE` | `100` | Maximum upstream keep-alive connections |
| `HEADROOM_BUDGET` | - | Daily budget limit in USD |
| `HEADROOM_TELEMETRY` | `off` (opt-in) | Set to `on` to opt in to anonymous telemetry |
| `HEADROOM_STATELESS` | `false` | Disable filesystem writes |
### Provider
| Variable | Default | Description |
|----------|---------|-------------|
| `ANTHROPIC_API_KEY` | - | Anthropic API key |
| `OPENAI_API_KEY` | - | OpenAI API key |
| `GOOGLE_API_KEY` | - | Google AI API key |
| `COHERE_API_KEY` | - | Cohere API key |
### Features
| Variable | Default | Description |
|----------|---------|-------------|
| `HEADROOM_TELEMETRY` | `off` (opt-in) | Set to `on` to opt in to telemetry |
| `HEADROOM_MIN_EVIDENCE` | `5` | Minimum observations before live learning persists a pattern |
| `HEADROOM_PROXY_EXTENSIONS` | - | Comma-separated proxy extensions to enable |
| `HEADROOM_STATELESS` | `false` | Disable filesystem writes |
| `HEADROOM_MODEL_LIMITS` | - | Model limits override as JSON or file path |
### Compression
| Variable | Default | Description |
|----------|---------|-------------|
| `HEADROOM_MAX_TOKENS` | `4096` | Max tokens per request |
| `HEADROOM_TARGET_TOKENS` | - | Target tokens after compression |
| `HEADROOM_OVERLAP_TOKENS` | `512` | Overlap tokens for chunking |
| `HEADROOM_CONTENT_SENSITIVITY` | `0.5` | Content sensitivity (0-1) |
| `HEADROOM_PRESERVE_SYSTEM` | `true` | Preserve system messages |
---
## Plugin ABI
### Plugin Interface
```python
from abc import ABC, abstractmethod
from headroom.learn.base import ConversationScanner, ContextWriter
from headroom.learn.models import ProjectInfo, SessionData
class LearnPlugin(ConversationScanner):
"""A self-contained learn plugin for a single coding agent."""
@property
@abstractmethod
def name(self) -> str:
"""Short lowercase identifier (e.g., 'claude', 'cursor')."""
...
@property
@abstractmethod
def display_name(self) -> str:
"""Human-readable name (e.g., 'Claude Code', 'Cursor')."""
...
@abstractmethod
def detect(self) -> bool:
"""Return True if this agent has data on the current machine."""
...
@abstractmethod
def discover_projects(self) -> list[ProjectInfo]:
"""Discover all projects with conversation data."""
...
@abstractmethod
def scan_project(self, project: ProjectInfo, max_workers: int = 1) -> list[SessionData]:
"""Scan all sessions for a project."""
...
@abstractmethod
def create_writer(self) -> ContextWriter:
"""Return the appropriate ContextWriter for this agent."""
...
```
### Plugin Registration
Plugins are auto-discovered from `headroom/learn/plugins/` directory.
**Manual registration:**
```python
from headroom.learn import plugin_registry
plugin_registry.register(MyPlugin())
```
### Plugin Config
```yaml
# ~/.headroom/config.yaml
learn:
enabled: true
plugins:
- name: claude
enabled: true
config:
session_modes:
- auto
- learn
- disabled
- name: my_plugin
enabled: true
config:
custom_option: value
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial interfaces document |

173
docs/spec/016-observability.md
View file

@ -1,173 +0,0 @@
# 016. Observability
**Status:** done
## Telemetry
### Metrics
Headroom exposes Prometheus metrics at `/metrics`.
**Key Metrics:**
| Metric | Type | Description |
|--------|------|-------------|
| `headroom_requests_total` | Counter | Total requests |
| `headroom_tokens_original` | Counter | Original token count |
| `headroom_tokens_compressed` | Counter | Compressed token count |
| `headroom_savings_percent` | Histogram | Savings distribution |
| `headroom_cache_hits_total` | Counter | Cache hits |
| `headroom_cache_misses_total` | Counter | Cache misses |
| `headroom_compression_duration_seconds` | Histogram | Compression latency |
| `headroom_request_duration_seconds` | Histogram | Total request latency |
**Prometheus scrape config:**
```yaml
scrape_configs:
- job_name: 'headroom'
static_configs:
- targets: ['localhost:8787']
metrics_path: '/metrics'
```
---
### Tracing
OpenTelemetry tracing support.
**Configuration (Langfuse):**
```bash
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
HEADROOM_LANGFUSE_ENABLED=1
# Optional: override endpoint and service name
# LANGFUSE_BASE_URL=https://cloud.langfuse.com
# HEADROOM_LANGFUSE_SERVICE_NAME=headroom
```
**Spans:**
| Span | Description |
|------|-------------|
| `headroom.proxy.request` | Full request lifecycle |
| `headroom.compression` | Compression operation |
| `headroom.cache.lookup` | Cache check |
| `headroom.provider.call` | Provider API call |
---
### Logging
**Log Levels:**
| Level | Use Case |
|-------|----------|
| `DEBUG` | Detailed debugging |
| `INFO` | General operation |
| `WARNING` | Degraded operation |
| `ERROR` | Failures |
**Log Format (JSON):**
```json
{
"timestamp": "2026-04-16T12:00:00Z",
"level": "INFO",
"message": "Request completed",
"request_id": "abc123",
"savings": 0.45,
"duration_ms": 120
}
```
**Configuration:**
```bash
# Logging level is controlled via the --log-level CLI flag (headroom proxy --log-level debug)
# or RUST_LOG env var for the Rust proxy. No HEADROOM_LOG_LEVEL env var exists.
```
Or in config:
```yaml
logging:
level: INFO
format: json
```
---
## Dashboard
**URL:** `http://localhost:8787/dashboard`
**Metrics Shown:**
- Total savings over time
- Requests per day
- Cache hit rate
- Top compressed endpoints
- Session overview
**Requires:** the proxy process to be running. The dashboard is served by default at `/dashboard`.
---
## Alerting
### Recommended Alerts
| Alert | Condition | Severity |
|-------|-----------|----------|
| HighErrorRate | error_rate > 5% | warning |
| LowSavings | savings < 20% | warning |
| CacheDown | cache_hits < 10% for 1h | critical |
| ProxyDown | health check fails | critical |
**Alert rule example (Prometheus):**
```yaml
groups:
- name: headroom
rules:
- alert: HighErrorRate
expr: rate(headroom_errors_total[5m]) / rate(headroom_requests_total[5m]) > 0.05
for: 5m
labels:
severity: warning
annotations:
summary: "High error rate in Headroom"
```
---
## Health Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/health` | GET | Basic health check |
| `/livez` | GET | Liveness check (process alive) |
| `/readyz` | GET | Readiness check (can serve traffic) |
**Health response:**
```json
{
"status": "healthy",
"version": "1.0.0"
}
```
**Readiness response:**
```json
{
"ready": true,
"checks": {
"database": true,
"cache": true,
"provider": true
}
}
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial observability document |

176
docs/spec/017-operations.md
View file

@ -1,176 +0,0 @@
# 017. Operations
**Status:** done
## Health Endpoints
### `GET /health`
Basic health check. Returns 200 if process is running.
```bash
curl http://localhost:8787/health
```
**Response:**
```json
{
"status": "healthy",
"version": "1.0.0"
}
```
---
### `GET /livez`
Liveness check. Returns 200 if process is alive.
```bash
curl http://localhost:8787/livez
```
---
### `GET /readyz`
Readiness check. Returns 200 if ready to serve traffic.
```bash
curl http://localhost:8787/readyz
```
**Response:**
```json
{
"ready": true,
"checks": {
"database": true,
"cache": true,
"provider": true
}
}
```
---
## Logs
### Log Locations
| Installation | Location |
|-------------|----------|
| Docker | `docker logs headroom` |
| Native | `~/.headroom/logs/` |
| Systemd | `journalctl -u headroom` |
### Log Levels
Set via CLI flag or `RUST_LOG` env var for the Rust proxy:
```bash
# Python proxy
headroom proxy --log-level debug
# Rust proxy
RUST_LOG=debug headroom-proxy --upstream http://...
```
---
## Metrics
### Prometheus
**Scrape Config:**
```yaml
scrape_configs:
- job_name: 'headroom'
static_configs:
- targets: ['localhost:8787']
metrics_path: '/metrics'
```
---
## Upgrade Procedure
### Docker
```bash
docker pull headroom-ai/headroom:latest
docker-compose down
docker-compose up -d
```
### Native
```bash
pip install --upgrade headroom-ai
# Restart headroom service
```
### Embedded
```bash
pip install --upgrade headroom-ai
# Restart application
```
---
## Rollback
### Docker
```bash
docker-compose down
docker tag headroom-ai/headroom:latest headroom-ai/headroom:rollback
# Edit docker-compose.yml to use rollback tag
docker-compose up -d
```
---
## Monitoring
### Key Metrics to Watch
1. **Request rate** — requests per second
2. **Error rate** — 4xx + 5xx / total
3. **Savings rate** — average savings percentage
4. **Latency** — p50, p95, p99
5. **Cache hit rate** — hits / total
**Prometheus queries:**
```promql
# Request rate
rate(headroom_requests_total[5m])
# Error rate
rate(headroom_errors_total[5m]) / rate(headroom_requests_total[5m])
# Average savings
rate(headroom_tokens_original[5m] - headroom_tokens_compressed[5m]) / rate(headroom_tokens_original[5m])
# Cache hit rate
rate(headroom_cache_hits_total[5m]) / (rate(headroom_cache_hits_total[5m]) + rate(headroom_cache_misses_total[5m]))
```
---
## Runbook
| Symptom | Cause | Solution |
|---------|-------|----------|
| "Connection refused" | Proxy not running | Start it with `headroom proxy` |
| "Cache miss on every request" | Cache disabled | Start without `--no-cache` |
| "No savings shown" | Database locked | Check file permissions |
| "Provider timeout" | Network issue | Check firewall/proxy |
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial operations document |

118
docs/spec/018-policies.md
View file

@ -1,118 +0,0 @@
# 018. Policies
**Status:** done
## Default Behaviors
### Compression
| Setting | Default | Description |
|---------|---------|-------------|
| Compression enabled | `true` | Apply compression by default |
| Cache enabled | `true` | Use semantic cache |
| Summary enabled | `true` | Use summary compression |
| Token budget enforced | `false` | No budget limit by default |
### Telemetry
| Setting | Default | Description |
|---------|---------|-------------|
| Metrics enabled | `true` | Emit Prometheus metrics |
| Tracing enabled | `false` | No OTEL tracing |
| Dashboard enabled | `false` | No built-in dashboard |
### Learn
| Setting | Default | Description |
|---------|---------|-------------|
| Learn enabled | `false` | Learning disabled |
| Plugin auto-detect | `true` | Auto-load plugins |
| Feedback collection | `false` | No CCR feedback |
---
## Override Mechanisms
### Environment Variables
All settings can be overridden via environment variables:
```bash
HEADROOM_MODE=token
headroom proxy --no-cache
```
### Config File
```yaml
# ~/.headroom/config.yaml
proxy:
host: 0.0.0.0
port: 8787
compression:
enabled: true
cache:
enabled: true
ttl: 3600
telemetry:
metrics:
enabled: true
tracing:
enabled: false
learn:
enabled: false
```
### Runtime API
```bash
# Disable compression for single request
curl -X POST http://localhost:8787/v1/messages \
-H "X-Headroom-Compress: false"
```
### Runtime Headers
| Header | Description |
|--------|-------------|
| `X-Headroom-Compress` | Override compression (true/false) |
| `X-Headroom-Mode` | Override mode (passthrough/compress/learn) |
| `X-Headroom-Cache` | Override cache (true/false) |
---
## Policy Hierarchy
Settings are evaluated in this order (highest wins):
1. **Runtime headers** — Per-request overrides
2. **Environment variables** — Process-level
3. **Config file** — Persistent settings
4. **Defaults** — Built-in defaults
---
## Per-Tenant Policies (TOIN)
TOIN tenants can have custom policies:
```yaml
tenant:
id: "acme-corp"
policies:
compression:
enabled: true
threshold: 3000
budget:
daily_limit: 1000000
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial policies document |

111
docs/spec/019-quality.md
View file

@ -1,111 +0,0 @@
# 019. Quality
**Status:** done
## Test Pyramid
```
┌─────────────┐
│ E2E │ ← Few, slow, comprehensive
│ Tests │
└──────┬──────┘
│
┌──────┴──────┐
│ Integration │ ← Medium, moderate
│ Tests │
└──────┬──────┘
│
┌────────────┴────────────┐
│ │
┌────┴────┐ ┌────┴────┐
│ Unit │ │ Unit │
│ Tests │ │ Tests │
└─────────┘ └─────────┘
Many, fast, isolated
```
---
## Test Coverage
| Surface | Unit | Integration | E2E |
|---------|:----:|:------------:|:---:|
| Proxy Server | ✓ | ✓ | ✓ |
| SDK | ✓ | ✓ | - |
| Compression | ✓ | ✓ | ✓ |
| Cache | ✓ | ✓ | - |
| Learn | ✓ | ✓ | - |
| CCR | ✓ | ✓ | ✓ |
| TOIN | ✓ | ✓ | - |
| Dashboard | ✓ | - | ✓ |
---
## Coverage Targets
| Metric | Target | Threshold |
|--------|--------|-----------|
| Line coverage | 80% | 70% |
| Branch coverage | 70% | 60% |
| Critical path | 100% | 100% |
---
## Critical Paths
These must always pass:
1. **Compression pipeline** — Input → Compress → Output
2. **Cache hit path** — Input → Cache check → Return
3. **Provider proxy** — Request → Proxy → Provider → Response
4. **Learn feedback** — Session → Analyze → Compress → Store
---
## Performance Benchmarks
| Operation | Target | Threshold |
|-----------|--------|-----------|
| Compression | < 50ms | < 200ms |
| Cache lookup | < 5ms | < 20ms |
| Proxy latency | +10ms | +50ms |
---
## CI/CD
### Required Checks
| Check | Command | Timeout |
|-------|---------|---------|
| Lint | `ruff check` | 2m |
| Type check | `mypy` | 5m |
| Unit tests | `pytest tests/unit/` | 10m |
| Integration | `pytest tests/ -k integration` | 15m |
| E2E | `pytest e2e/` | 30m |
### Workflow (`.github/workflows/`)
1. **Lint** — `ruff check` + `ruff format --check`
2. **Type check** — `mypy src/`
3. **Unit tests** — `pytest tests/unit/ --cov`
4. **Integration** — `pytest tests/ -k integration`
5. **E2E** — `pytest e2e/ --api-key=$ANTHROPIC_API_KEY`
---
## Quality Gates
PRs must pass:
- All tests green
- Type checking passes (`mypy`)
- Lint passes (`ruff`)
- Coverage maintained or improved
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial quality document |

120
docs/spec/020-security.md
View file

@ -1,120 +0,0 @@
# 020. Security
**Status:** done
## Threat Model
### Threats
| Threat | Impact | Mitigation |
|--------|--------|------------|
| Prompt data exfiltration | High | No logging, local-only |
| API key theft | High | Key rotation, secrets management |
| Cache poisoning | Medium | Input validation |
| DoS via large prompts | Medium | Token limits |
| SSRF via redirects | Medium | URL validation |
---
### Trust Boundaries
```
┌──────────────────┐ ┌──────────────────┐
│ User's App │────▶│ Headroom Proxy │
└──────────────────┘ └────────┬─────────┘
│
┌────────┴─────────┐
│ │
┌─────▼─────┐ ┌──────▼──────┐
│ Provider │ │ Database │
│ APIs │ │ (SQLite) │
└───────────┘ └─────────────┘
```
---
## Security Controls
### Authentication
| Surface | Auth Method |
|---------|-------------|
| Proxy | API key (optional) |
| Dashboard | None by default |
| Health endpoints | None |
| Metrics | None |
### Authorization
- **No multi-user support** — Single-tenant by design
- **API keys** — For proxy authentication
- **CORS** — Configurable origins
---
## Data Protection
| Data | At Rest | In Transit |
|------|---------|------------|
| Prompts | Encrypted (if DB encrypted) | TLS |
| Responses | Encrypted (if DB encrypted) | TLS |
| API keys | Encrypted | TLS |
| Metrics | Plain text | TLS |
---
## Input Validation
- **Prompt length** — Enforced via token budget
- **URL validation** — For provider redirects
- **Schema validation** — For all API inputs
---
## Secrets Management
### Environment Variables
```bash
ANTHROPIC_API_KEY=sk-...
OPENAI_API_KEY=sk-...
```
### Secrets Management Systems
Headroom supports:
- AWS Secrets Manager
- HashiCorp Vault
- Azure Key Vault
---
## Supply Chain Security
### Dependencies
- **Pinned versions** — All dependencies pinned
- **Audit** — Regular `pip audit`
- **SBOM** — Software Bill of Materials generated
### Build
- **Reproducible** — Docker builds are reproducible
- **Signed releases** — Code signed
---
## Vulnerability Reporting
See `SECURITY.md` for:
- Reporting process
- Response timeline
- Disclosure policy
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial security document |

162
docs/spec/021-testing.md
View file

@ -1,162 +0,0 @@
# 021. Testing
**Status:** done
## Test Strategy by Surface
### Proxy Server
**Unit Tests:**
- Request routing
- Response header injection
- Mode switching
- Error handling
**Integration Tests:**
- Full request/response cycle
- Provider mocking
- Cache integration
**E2E Tests:**
- Against real provider APIs (with API keys)
---
### Compression
**Unit Tests:**
- Token counting
- Semantic hashing
- Summary compression
- Budget enforcement
**Integration Tests:**
- Cache round-trip
- Multi-stage compression
**Benchmarks:**
- Latency at scale
- Memory usage
---
### Learn System
**Unit Tests:**
- Plugin interface compliance
- Error classification
- Session analysis
**Integration Tests:**
- Plugin discovery
- Cross-plugin interaction
**E2E Tests:**
- Real session analysis
---
### CCR (Claude Code Relay)
**Unit Tests:**
- Context tracking
- Feedback collection
- Batch processing
**Integration Tests:**
- Tool injection
- Response handling
**E2E Tests:**
- Full CCR workflow
---
## Test Utilities
### Fixtures
Located in `tests/conftest.py`:
- `mock_provider` — Mock AI provider
- `sample_session` — Sample conversation
- `temp_db` — Temporary database
### Mock Libraries
- `responses` — HTTP mocking
- `pytest-mock` — Function mocking
- `aioresponses` — Async HTTP mocking
---
## Running Tests
### All Tests
```bash
pytest
```
### By Surface
```bash
pytest tests/test_proxy/
pytest tests/test_compression/
pytest tests/test_learn/
```
### With Coverage
```bash
pytest --cov=headroom --cov-report=html
```
### E2E Tests
```bash
# Requires API keys
pytest e2e/ --api-key=$ANTHROPIC_API_KEY
```
---
## Test Data
### Sample Sessions
Stored in `tests/fixtures/sessions/`:
- `claude_code_short.json` — Short Claude Code session
- `claude_code_long.json` — Long session with tool use
- `codex_completion.json` — Codex completion
- `gemini_multimodal.json` — Gemini with images
---
## Continuous Integration
### Required Checks
| Check | Command | Timeout |
|-------|---------|---------|
| Lint | `ruff check` | 2m |
| Type check | `mypy` | 5m |
| Unit tests | `pytest tests/` | 10m |
| Integration | `pytest tests/ -k integration` | 15m |
| E2E | `pytest e2e/` | 30m |
---
## Test Maintenance
- **Fixtures** — Keep realistic but small
- **Mocks** — Don't over-mock providers
- **Flaky tests** — Mark with `@pytest.mark.flaky`
- **Coverage drops** — PR blocked if coverage drops
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial testing document |

98
docs/spec/022-rust-migration.md
View file

@ -1,98 +0,0 @@
# 022. Rust Migration
**Status:** in-progress (Stage 0 complete)
Headroom is evolving from a pure-Python proxy into a Rust engine with a Python SDK layer. This document describes the motivation, target architecture, and phased execution plan. It is the source of truth for the migration; the Discord announcement and contributor docs are derived from it.
## Why Rust
Three forces push us toward Rust:
1. **Latency.** Every LLM request flows through compression transforms on the hot path. Rust runs that math 2–10× faster than Python, with negligible interpreter overhead and near-instant cold starts.
2. **Deployment.** A Rust proxy is a single static binary (~10 MB). No Python interpreter, no pip, no wheel matrix, no model downloads at startup. It drops cleanly into containers, serverless runtimes, or bare hosts.
3. **Non-Python consumers.** Integrations like the TypeScript OpenClaw plugin talk to Headroom over HTTP. A faster proxy makes every downstream client faster without changing any client code.
The Python code is not being thrown away. Rust lives alongside Python in the same repository, the HTTP contract stays stable, and existing users see no breaking changes.
## Target architecture
Two new artifacts, built over time:
- **`headroom-proxy`** — a standalone Rust binary that speaks the existing Headroom HTTP API and contains native implementations of the proxy's hot-path logic (routing, streaming, compression, telemetry). This is the deployable artifact.
- **`headroom-core`** — a Rust library with the compression transforms (CCR, log compressor, diff compressor, tokenizer, code compressor, content router, etc.). Consumed by `headroom-proxy` directly; optionally exposed to Python via a PyO3 binding if/when embedded SDK use warrants it.
Both live in a Cargo workspace under `crates/` at the repo root. They are built, tested, and released together with the Python package.
## Migration strategy
**Trunk-based, no long-lived branch.** Every Rust change ships as a small PR to `main`. Rust and Python live side by side. CI enforces parity on every PR — if the Rust port diverges from the Python reference, the build fails.
**Proxy-first, not transforms-first.** We build the Rust proxy binary as the primary deliverable, starting from a pure HTTP passthrough that forwards upstream to the existing Python proxy. Native Rust implementations of individual routes then replace passthroughs one at a time, gated by feature flags. This lets us ship a deployable Rust binary on day one and iterate without modifying Python internals.
**Feature-flagged cutover.** Each native Rust route is toggled by config. Default is passthrough-to-Python until a route has been shadow-tested and validated. Rollback is a flag flip, not a redeploy.
**Parity by shadow traffic.** Recorded input/output fixtures give us a unit-test-level parity check. Shadow mode — running both proxies on live traffic and diffing outputs — gives us the real validation gate before any cutover.
## Stages
### Stage 0 — Foundation ✅
Cargo workspace with four crates (`headroom-core`, `headroom-proxy`, `headroom-py`, `headroom-parity`), CI, build tooling (Makefile, GitHub Actions), and a parity test harness seeded with 125 recorded fixtures across 5 leaf transforms. No production behavior changed.
### Stage 1 — Rust proxy as passthrough
`headroom-proxy` accepts requests on the same HTTP contract as the Python proxy and forwards everything upstream to Python. No transforms yet, no intelligence. The point is a deployable binary that can run in front of the existing stack with zero risk.
### Stage 2 — First native route
Replace the passthrough for `/v1/chat/completions` (OpenAI) with a native Rust implementation: Rust transforms, direct provider call, streamed response. Feature-flagged. Python proxy handles everything else unchanged.
### Stage 3 — Shadow mode validation
Run both proxies on real traffic. Diff outputs with tolerance for chunk timing (SSE). Gate: one week of ≤ 0.1% content divergence before flipping the flag for real.
### Stage 4 — Provider expansion
Anthropic, Google, Cohere, Mistral, Bedrock, others. Each provider goes through its own shadow period before cutover.
### Stage 5 — Code-aware transforms
Port `code_compressor` (tree-sitter, already Rust-native upstream), `content_router`, `content_detector`. These are larger transforms but have no ML dependencies.
### Stage 6 — Storage layer
SQLite (`rusqlite` + `sqlite-vec`), HNSW vector index (`instant-distance`), graph store (`neo4rs`). Feature-flagged backend for the memory subsystem.
### Stage 7 — ONNX migration
Remove the torch-dependent LLMLingua compressor. Convert remaining ML models (SmartCrusher, IntelligentContext, memory embedders) to ONNX with fixed opset. Run via the `ort` crate. Remove `torch`, `transformers`, `sentence-transformers`, `llmlingua` from runtime dependencies.
### Stage 8 — Retire the Python proxy
Delete `headroom/proxy/server.py` and the Python HTTP routes. The Python package (`import headroom`) continues to exist for SDK users and integration adapters (LangChain, Agno, MCP, Strands), which talk HTTP and don't care which proxy implementation is behind the endpoint.
## What does not change
- `pip install headroom` continues to work.
- The HTTP API contract is preserved.
- LangChain, Agno, MCP, Strands integrations keep working unchanged.
- The CLI, dashboard, eval framework, examples, and documentation site are all unaffected.
- Python contributions remain welcome for integrations, tooling, examples, and any code paths not yet ported.
## Contributing
- **Python contributors** do not need to learn Rust. CI will flag a PR if a Python change diverges from a Rust-ported counterpart; in that case either update both implementations or disable the feature flag for the affected route.
- **Rust contributors** should read `RUST_DEV.md` for workspace setup, then pick a transform or proxy route from the open issues.
## Risks and open questions
- **ONNX export parity** for embedding models: numerical reproducibility must be validated per model before cutover. Some models may resist clean export and require keeping a Python inference path behind PyO3 as a fallback.
- **LLMLingua removal** is a feature removal visible to users relying on it; deprecation timing will be announced on Discord before Stage 7 begins.
- **PyO3 binding scope**: we may ultimately not need a PyO3-exposed `headroom._core` at all, if existing Python SDK users are happy with the HTTP contract. Decision deferred to Stage 8.
## References
- `RUST_DEV.md` — developer setup and workspace reference
- `crates/` — Rust sources
- `tests/parity/` — fixtures and parity harness
- `Makefile` — `make test`, `make test-parity`, `make build-proxy`, `make build-wheel`, `make fmt`, `make lint`

88
docs/spec/SPEC.md
View file

@ -1,88 +0,0 @@
# Headroom Living Specification
**Version:** 1.0.0-draft
**Date:** 2026-04-16
**Status:** Draft — In Progress
**Related Issue:** GitHub #183
---
## Constitution
This specification is the **canonical source of truth** for how Headroom is designed to behave. It serves:
1. **New contributors** — One place to understand how Headroom works
2. **Enterprises evaluating adoption** — Clear guarantees about behavior, privacy, security
3. **Operators running managed deployments** — Operational guidance for all surfaces
4. **Plugin authors** — Clear contracts for extension points
5. **PR review** — A checklist target: "does the code match the spec?"
### Spec Governance
| Rule | Description |
|------|-------------|
| **Canonical** | When code and spec diverge, the spec is the target; the code needs updating |
| **Living** | Spec updates are required for behavior-changing changes (PR checklist) |
| **Comprehensive** | Spec covers every user-visible surface, behavior, and guarantee |
| **Language-agnostic** | Spec enables complete rewrite in any language with parity |
| **Versioned** | Changes increment version; breaking changes require major version bump |
### Spec Sections
| # | Section | Status | Description |
|---|---------|:------:|-------------|
| 001 | [Vision](001-vision.md) | done | What Headroom is, what it is not |
| 002 | [Architecture](002-architecture.md) | done | Component diagram + descriptions |
| 003 | [ADRs](003-adrs.md) | done | Architecture Decision Records |
| 004 | [Domain Model](004-domain-model.md) | done | Core entities |
| 005 | [Integrations](005-integrations.md) | done | Agent contracts |
| 006 | [Actors](006-actors.md) | done | User types + interactions |
| 007 | [Behavior](007-behavior.md) | done | Mode-by-mode specification |
| 008 | [Capabilities](008-capabilities.md) | done | Feature matrix |
| 009 | [Compliance](009-compliance.md) | done | Data guarantees, privacy |
| 010 | [Data](010-data.md) | done | Storage, retention, env vars |
| 011 | [Deployment](011-deployment.md) | done | Profiles, presets, runtimes |
| 012 | [Diagrams](012-diagrams.md) | done | Component, sequence, data-flow |
| 013 | [Disaster Recovery](013-disaster-recovery.md) | done | Failure modes + recovery |
| 014 | [Governance](014-governance.md) | done | Decision-making, releases |
| 015 | [Interfaces](015-interfaces.md) | done | CLI, HTTP, env var, plugin ABI |
| 016 | [Observability](016-observability.md) | done | Telemetry, metrics, logs |
| 017 | [Operations](017-operations.md) | done | Health, logs, upgrades |
| 018 | [Policies](018-policies.md) | done | Defaults + overrides |
| 019 | [Quality](019-quality.md) | done | Test pyramid coverage |
| 020 | [Security](020-security.md) | done | Threat model, supply-chain |
| 021 | [Testing](021-testing.md) | done | Test strategy per surface |
---
## Quick Reference
### What Headroom Is
- A **context compression proxy** for AI provider APIs
- A **Python package** (`headroom-ai`) with proxy, SDK, and CLI
- A **TypeScript SDK** (`@headroom/sdk`) for Node.js
- A **dashboard** for visualizing savings
- A **learn system** with per-agent plugins
### What Headroom Is Not
- A model provider
- A data store for prompts (by default)
- A logging service (by default)
- A billing service
### Core Guarantees
1. **Never logs prompts by default** — No prompt data leaves the proxy unless an exporter is configured
2. **Never leaves the proxy by default** — All data stays local unless explicitly exported
3. **Composable** — Works alongside existing tools (Claude Code, Copilot, etc.)
4. **Transparent** — Full observability into what's being compressed and why
---
## Change Log
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0-draft | 2026-04-16 | Initial draft — 21 sections outlined |

699
docs/superpowers/plans/2026-06-19-deepseek-pricing.md
View file

@ -1,699 +0,0 @@
# DeepSeek V4 Pricing Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add `deepseek-v4-flash` and `deepseek-v4-pro` pricing to Headroom so cost estimation works when routing through `--anthropic-api-url https://api.deepseek.com/anthropic`.
**Architecture:** Four independent layers — (1) a new pricing data module following the `anthropic_prices.py` pattern, (2) runtime injection into `litellm.model_cost` so the primary cost-per-token path resolves DeepSeek V4, (3) a fallback in the Anthropic provider's `_get_pricing()` when LiteLLM is unavailable, and (4) vendored JSON entries for Rust-side context window lookups.
**Tech Stack:** Python 3.12+, LiteLLM, Rust (vendored JSON), pytest
## Global Constraints
- All prices in USD per 1 million tokens (`ModelPricing` dataclass convention)
- LiteLLM injection must not overwrite upstream entries if litellm already has DeepSeek V4
- Provider field must be `"deepseek"` for all DeepSeek models
- Follow exact patterns from `anthropic_prices.py` / `openai_prices.py` / `test_anthropic.py`
- Vendored JSON entries at `crates/headroom-proxy/data/model_prices_and_context_window.json`
---
### Task 1: DeepSeek Pricing Data Module
**Files:**
- Create: `headroom/pricing/deepseek_prices.py`
- Modify: `headroom/pricing/__init__.py`
- Test: `tests/test_providers/test_deepseek.py` (first test class)
**Interfaces:**
- Consumes: `ModelPricing`, `PricingRegistry` from `headroom.pricing.registry`
- Produces: `DEEPSEEK_PRICES: dict[str, ModelPricing]`, `get_deepseek_registry() -> PricingRegistry`, exported via `headroom.pricing`
- [ ] **Step 1: Write the failing tests for the pricing data module**
Create `tests/test_providers/test_deepseek.py`:
```python
"""Tests for DeepSeek model pricing and cost estimation."""
from headroom.pricing.deepseek_prices import (
DEEPSEEK_PRICES,
get_deepseek_registry,
)
from headroom.pricing.registry import PricingRegistry
class TestDeepSeekPricingModule:
"""Tests for the DeepSeek pricing data module."""
def test_deepseek_pricing_contains_v4_models(self):
assert "deepseek-v4-flash" in DEEPSEEK_PRICES
assert "deepseek-v4-pro" in DEEPSEEK_PRICES
assert len(DEEPSEEK_PRICES) == 2
def test_deepseek_v4_flash_pricing(self):
pricing = DEEPSEEK_PRICES["deepseek-v4-flash"]
assert pricing.input_per_1m == 0.14
assert pricing.output_per_1m == 0.28
assert pricing.cached_input_per_1m == 0.0028
assert pricing.context_window == 1_000_000
assert pricing.provider == "deepseek"
assert pricing.notes is not None
def test_deepseek_v4_pro_pricing(self):
pricing = DEEPSEEK_PRICES["deepseek-v4-pro"]
assert pricing.input_per_1m == 0.435
assert pricing.output_per_1m == 0.87
assert pricing.cached_input_per_1m == 0.003625
assert pricing.context_window == 1_000_000
assert pricing.provider == "deepseek"
assert pricing.notes is not None
def test_get_deepseek_registry(self):
registry = get_deepseek_registry()
assert isinstance(registry, PricingRegistry)
assert registry.get_price("deepseek-v4-flash") is not None
assert registry.get_price("deepseek-v4-pro") is not None
assert registry.get_price("nonexistent") is None
def test_registry_staleness_and_source_url(self):
registry = get_deepseek_registry()
assert registry.source_url == "https://api-docs.deepseek.com/quick_start/pricing"
assert not registry.is_stale()
def test_deepseek_registry_estimate_cost(self):
registry = get_deepseek_registry()
cost = registry.estimate_cost("deepseek-v4-flash", input_tokens=1_000_000)
assert cost.cost_usd == 0.14
assert "input" in cost.breakdown
assert cost.pricing_date is not None
def test_deepseek_registry_estimate_cost_with_cached(self):
registry = get_deepseek_registry()
cost = registry.estimate_cost(
"deepseek-v4-flash",
input_tokens=1_000_000,
cached_input_tokens=1_000_000,
)
assert cost.cost_usd == 0.14 + 0.0028
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
cd D:/headroom && python -m pytest tests/test_providers/test_deepseek.py::TestDeepSeekPricingModule -v
```
Expected: ModuleNotFoundError or ImportError — `deepseek_prices` doesn't exist yet.
- [ ] **Step 3: Create the pricing data module**
Create `headroom/pricing/deepseek_prices.py`:
```python
"""DeepSeek model pricing information."""
from datetime import date
from .registry import ModelPricing, PricingRegistry
# Last verified date for pricing information
LAST_UPDATED = date(2026, 6, 19)
# Official pricing page
SOURCE_URL = "https://api-docs.deepseek.com/quick_start/pricing"
# All prices are in USD per 1 million tokens
DEEPSEEK_PRICES: dict[str, ModelPricing] = {
"deepseek-v4-flash": ModelPricing(
model="deepseek-v4-flash",
provider="deepseek",
input_per_1m=0.14,
output_per_1m=0.28,
cached_input_per_1m=0.0028,
context_window=1_000_000,
notes="DeepSeek V4 Flash - 13B active params; non-thinking + thinking modes",
),
"deepseek-v4-pro": ModelPricing(
model="deepseek-v4-pro",
provider="deepseek",
input_per_1m=0.435,
output_per_1m=0.87,
cached_input_per_1m=0.003625,
context_window=1_000_000,
notes="DeepSeek V4 Pro - 49B active params",
),
}
def get_deepseek_registry() -> PricingRegistry:
"""Create and return a DeepSeek pricing registry.
Returns:
PricingRegistry configured with DeepSeek model prices.
"""
return PricingRegistry(
last_updated=LAST_UPDATED,
source_url=SOURCE_URL,
prices=DEEPSEEK_PRICES.copy(),
)
```
- [ ] **Step 4: Wire into `__init__.py`**
Edit `headroom/pricing/__init__.py` — add these imports after the existing OpenAI imports:
```python
from .deepseek_prices import (
DEEPSEEK_PRICES,
get_deepseek_registry,
)
from .deepseek_prices import (
LAST_UPDATED as DEEPSEEK_LAST_UPDATED,
)
```
And add to `__all__`:
```python
# DeepSeek
"DEEPSEEK_LAST_UPDATED",
"DEEPSEEK_PRICES",
"get_deepseek_registry",
```
- [ ] **Step 5: Run tests to verify they pass**
```bash
cd D:/headroom && python -m pytest tests/test_providers/test_deepseek.py::TestDeepSeekPricingModule -v
```
Expected: 7 passed
- [ ] **Step 6: Commit**
```bash
git add headroom/pricing/deepseek_prices.py headroom/pricing/__init__.py tests/test_providers/test_deepseek.py
git commit -m "feat(pricing): add DeepSeek V4 pricing data module
Add deepseek-v4-flash and deepseek-v4-pro ModelPricing entries and
registry factory, following the pattern of anthropic_prices.py.
Wire into pricing/__init__.py exports.
Co-Authored-By: Claude <noreply@anthropic.com>"
```
---
### Task 2: LiteLLM Runtime Injection
**Files:**
- Modify: `headroom/pricing/litellm_pricing.py` (add injection after `LITELLM_AVAILABLE` block)
- Test: `tests/test_providers/test_deepseek.py` (add `TestDeepSeekLiteLLMInjection` class)
**Interfaces:**
- Consumes: `litellm.model_cost` dict (available after import)
- Produces: `"deepseek-v4-flash"`, `"deepseek/deepseek-v4-flash"`, `"deepseek-v4-pro"`, `"deepseek/deepseek-v4-pro"` keys in `litellm.model_cost`
- Depends on: DEEPSEEK_V4_PRICING constant defined within litellm_pricing.py
- [ ] **Step 1: Write failing injection tests**
Append to `tests/test_providers/test_deepseek.py`:
```python
import pytest
class TestDeepSeekLiteLLMInjection:
"""Tests for DeepSeek V4 pricing injection into litellm."""
def test_deepseek_v4_models_in_litellm_model_cost(self):
from headroom.pricing.litellm_pricing import litellm, LITELLM_AVAILABLE
if not LITELLM_AVAILABLE:
pytest.skip("litellm not available")
assert "deepseek-v4-flash" in litellm.model_cost
assert "deepseek-v4-pro" in litellm.model_cost
def test_deepseek_v4_prefixed_models_in_litellm_model_cost(self):
from headroom.pricing.litellm_pricing import litellm, LITELLM_AVAILABLE
if not LITELLM_AVAILABLE:
pytest.skip("litellm not available")
assert "deepseek/deepseek-v4-flash" in litellm.model_cost
assert "deepseek/deepseek-v4-pro" in litellm.model_cost
def test_deepseek_v4_flash_litellm_pricing(self):
from headroom.pricing.litellm_pricing import litellm, LITELLM_AVAILABLE
if not LITELLM_AVAILABLE:
pytest.skip("litellm not available")
flash = litellm.model_cost["deepseek-v4-flash"]
assert flash["input_cost_per_token"] == 0.14 / 1_000_000
assert flash["output_cost_per_token"] == 0.28 / 1_000_000
assert flash["cache_read_input_token_cost"] == 0.0028 / 1_000_000
assert flash["litellm_provider"] == "deepseek"
def test_deepseek_v4_pro_litellm_pricing(self):
from headroom.pricing.litellm_pricing import litellm, LITELLM_AVAILABLE
if not LITELLM_AVAILABLE:
pytest.skip("litellm not available")
pro = litellm.model_cost["deepseek-v4-pro"]
assert pro["input_cost_per_token"] == 0.435 / 1_000_000
assert pro["output_cost_per_token"] == 0.87 / 1_000_000
assert pro["cache_read_input_token_cost"] == 0.003625 / 1_000_000
assert pro["litellm_provider"] == "deepseek"
def test_cost_per_token_resolves_deepseek_v4_flash(self):
from headroom.pricing.litellm_pricing import litellm, LITELLM_AVAILABLE
if not LITELLM_AVAILABLE:
pytest.skip("litellm not available")
input_cost, output_cost = litellm.cost_per_token(
model="deepseek-v4-flash",
prompt_tokens=1_000_000,
completion_tokens=1_000_000,
)
assert input_cost == pytest.approx(0.14, rel=0.01)
assert output_cost == pytest.approx(0.28, rel=0.01)
def test_injection_does_not_overwrite_existing_upstream_entries(self):
"""If litellm upstream already has these, our injection is a no-op."""
from headroom.pricing.litellm_pricing import litellm, LITELLM_AVAILABLE
if not LITELLM_AVAILABLE:
pytest.skip("litellm not available")
# Force-inject with wrong value, then verify the injection guard
litellm.model_cost["deepseek-v4-flash"] = {"input_cost_per_token": 999}
# Reimport to trigger _inject_deepseek_pricing — but it should NOT overwrite
import importlib
import headroom.pricing.litellm_pricing as lp
importlib.reload(lp)
assert litellm.model_cost["deepseek-v4-flash"]["input_cost_per_token"] == 999
# Reset to correct value
litellm.model_cost["deepseek-v4-flash"] = {
"input_cost_per_token": 0.14 / 1_000_000,
"output_cost_per_token": 0.28 / 1_000_000,
"cache_read_input_token_cost": 0.0028 / 1_000_000,
"litellm_provider": "deepseek",
"max_tokens": 384_000,
"max_input_tokens": 1_000_000,
}
```
- [ ] **Step 2: Run the injection tests to verify they fail**
```bash
cd D:/headroom && python -m pytest tests/test_providers/test_deepseek.py::TestDeepSeekLiteLLMInjection -v
```
Expected: Tests fail because `litellm.model_cost` doesn't have DeepSeek V4 entries yet.
- [ ] **Step 3: Add the runtime injection to `litellm_pricing.py`**
At the end of `headroom/pricing/litellm_pricing.py`, before the `_resolved_model_cache` and function definitions, add:
```python
# ============================================================
# DeepSeek V4 pricing injection
# ============================================================
# Vendored LiteLLM JSON predates DeepSeek V4 models. Inject pricing at
# import time so the primary cost-per-token path resolves them. Once
# upstream litellm adds these entries, injection becomes a no-op.
# ============================================================
_DEEPSEEK_V4_PRICING: dict[str, dict[str, float | str | int]] = {
"deepseek-v4-flash": {
"input_cost_per_token": 0.14 / 1_000_000,
"output_cost_per_token": 0.28 / 1_000_000,
"cache_read_input_token_cost": 0.0028 / 1_000_000,
"litellm_provider": "deepseek",
"max_tokens": 384_000,
"max_input_tokens": 1_000_000,
},
"deepseek-v4-pro": {
"input_cost_per_token": 0.435 / 1_000_000,
"output_cost_per_token": 0.87 / 1_000_000,
"cache_read_input_token_cost": 0.003625 / 1_000_000,
"litellm_provider": "deepseek",
"max_tokens": 384_000,
"max_input_tokens": 1_000_000,
},
}
def _inject_deepseek_pricing() -> None:
"""Inject DeepSeek V4 pricing into litellm's model_cost dict.
Only injects entries not already present, so upstream litellm additions
(once available) take precedence. Both bare and provider-prefixed keys
are added so resolve_litellm_model() catches them via its deepseek/
prefix loop.
"""
if not LITELLM_AVAILABLE:
return
for model_name, pricing in _DEEPSEEK_V4_PRICING.items():
if model_name not in litellm.model_cost:
litellm.model_cost[model_name] = pricing
prefixed = f"deepseek/{model_name}"
if prefixed not in litellm.model_cost:
litellm.model_cost[prefixed] = pricing
_inject_deepseek_pricing()
```
- [ ] **Step 4: Run tests to verify they pass**
```bash
cd D:/headroom && python -m pytest tests/test_providers/test_deepseek.py::TestDeepSeekLiteLLMInjection -v
```
Expected: All 6 tests pass (or some skip if litellm is unavailable — that's acceptable).
- [ ] **Step 5: Commit**
```bash
git add headroom/pricing/litellm_pricing.py tests/test_providers/test_deepseek.py
git commit -m "feat(pricing): inject DeepSeek V4 pricing into litellm model_cost
Add runtime injection so litellm.cost_per_token() resolves
deepseek-v4-flash and deepseek-v4-pro pricing. Includes both bare
and provider-prefixed keys. No-ops if entries already exist.
Co-Authored-By: Claude <noreply@anthropic.com>"
```
---
### Task 3: Anthropic Provider DeepSeek Fallback
**Files:**
- Modify: `headroom/providers/anthropic.py` (add `_get_deepseek_pricing()` helper and call in `_get_pricing()`)
- Test: `tests/test_providers/test_deepseek.py` (add `TestDeepSeekAnthropicProviderFallback` class)
**Interfaces:**
- Consumes: `_get_pricing(self, model: str) -> dict | None` (inside `AnthropicProvider`)
- Produces: DeepSeek pricing dicts from `_get_pricing("deepseek-*")` and `estimate_cost("deepseek-*")`
- [ ] **Step 1: Write failing fallback tests**
Append to `tests/test_providers/test_deepseek.py`:
```python
class TestDeepSeekAnthropicProviderFallback:
"""Tests that Anthropic provider's _get_pricing handles DeepSeek models."""
def test_deepseek_v4_flash_fallback(self):
from headroom.providers.anthropic import AnthropicProvider
provider = AnthropicProvider()
pricing = provider._get_pricing("deepseek-v4-flash")
assert pricing is not None
assert pricing["input"] == 0.14
assert pricing["output"] == 0.28
assert pricing["cached_input"] == 0.0028
def test_deepseek_v4_pro_fallback(self):
from headroom.providers.anthropic import AnthropicProvider
provider = AnthropicProvider()
pricing = provider._get_pricing("deepseek-v4-pro")
assert pricing is not None
assert pricing["input"] == 0.435
assert pricing["output"] == 0.87
assert pricing["cached_input"] == 0.003625
def test_deepseek_unknown_model_returns_none(self):
from headroom.providers.anthropic import AnthropicProvider
provider = AnthropicProvider()
pricing = provider._get_pricing("deepseek-unknown-model")
assert pricing is None
def test_deepseek_partial_match_v4_flash_alias(self):
from headroom.providers.anthropic import AnthropicProvider
provider = AnthropicProvider()
# Should match via partial match (flash in v4-flash)
pricing = provider._get_pricing("deepseek-v4-flash-v1")
assert pricing is not None
def test_estimate_cost_deepseek_v4_flash(self):
from headroom.providers.anthropic import AnthropicProvider
provider = AnthropicProvider()
cost = provider.estimate_cost(
input_tokens=1_000_000,
output_tokens=0,
model="deepseek-v4-flash",
)
assert cost is not None
assert cost == 0.14
def test_estimate_cost_deepseek_v4_flash_with_cache(self):
from headroom.providers.anthropic import AnthropicProvider
provider = AnthropicProvider()
cost = provider.estimate_cost(
input_tokens=1_000_000,
output_tokens=0,
model="deepseek-v4-flash",
cached_tokens=1_000_000,
)
assert cost is not None
assert cost == 0.14 + 0.0028
```
- [ ] **Step 2: Run fallback tests to verify they fail**
```bash
cd D:/headroom && python -m pytest tests/test_providers/test_deepseek.py::TestDeepSeekAnthropicProviderFallback -v
```
Expected: Fail because `_get_pricing()` returns `None` for `deepseek-*` models.
- [ ] **Step 3: Add DeepSeek fallback to Anthropic provider**
In `headroom/providers/anthropic.py`, add a new helper function after the `_UNKNOWN_CLAUDE_DEFAULT` dict (around line 151) and modify `_get_pricing()`:
Add the helper (after `_UNKNOWN_CLAUDE_DEFAULT` at line 151):
```python
# DeepSeek fallback pricing for --anthropic-api-url deepseek routing
_DEEPSEEK_FALLBACK_PRICING: dict[str, dict[str, float]] = {
"deepseek-v4-flash": {"input": 0.14, "output": 0.28, "cached_input": 0.0028},
"deepseek-v4-pro": {"input": 0.435, "output": 0.87, "cached_input": 0.003625},
}
def _get_deepseek_pricing(model: str) -> dict[str, float] | None:
"""Get fallback pricing for a DeepSeek model.
Used when the Anthropic provider encounters a deepseek-* model name
(via --anthropic-api-url pointing at DeepSeek's Anthropic-compatible
endpoint) and LiteLLM is unavailable.
Args:
model: The model name to look up.
Returns:
Pricing dict with input/output/cached_input keys, or None.
"""
# Direct match
if model in _DEEPSEEK_FALLBACK_PRICING:
return cast(dict[str, float], _DEEPSEEK_FALLBACK_PRICING[model])
# Partial match
for known_model, prices in _DEEPSEEK_FALLBACK_PRICING.items():
if model in known_model or known_model in model:
return cast(dict[str, float], prices)
return None
```
Modify `_get_pricing()` in the `AnthropicProvider` class. Add after the Claude default check (line 698):
```python
# Default for unknown Claude models
if model.startswith("claude"):
return cast(dict[str, float], _UNKNOWN_CLAUDE_DEFAULT["pricing"])
# DeepSeek model fallback (via --anthropic-api-url)
if model.startswith("deepseek"):
return _get_deepseek_pricing(model)
return None
```
- [ ] **Step 4: Run tests to verify they pass**
```bash
cd D:/headroom && python -m pytest tests/test_providers/test_deepseek.py::TestDeepSeekAnthropicProviderFallback -v
```
Expected: All 6 tests pass.
- [ ] **Step 5: Run full test suite for pricing-related tests**
```bash
cd D:/headroom && python -m pytest tests/test_providers/test_anthropic.py tests/test_providers/test_deepseek.py tests/test_pricing.py tests/test_pricing_litellm.py tests/test_litellm_optional.py -v
```
Expected: All existing tests still pass, plus new tests pass.
- [ ] **Step 6: Commit**
```bash
git add headroom/providers/anthropic.py tests/test_providers/test_deepseek.py
git commit -m "feat(providers): add DeepSeek pricing fallback to Anthropic provider
When routing through --anthropic-api-url with a DeepSeek endpoint,
_get_pricing() now handles deepseek-* model names. Falls back to
hardcoded V4 pricing when LiteLLM is unavailable.
Co-Authored-By: Claude <noreply@anthropic.com>"
```
---
### Task 4: Vendored Rust JSON Entries
**Files:**
- Modify: `crates/headroom-proxy/data/model_prices_and_context_window.json`
**Interfaces:**
- Consumes: The existing DeepSeek JSON entry structure around line 12516
- Produces: `"deepseek-v4-flash"` and `"deepseek-v4-pro"` entries in the JSON
- [ ] **Step 1: Add DeepSeek V4 entries to the vendored JSON**
Open `crates/headroom-proxy/data/model_prices_and_context_window.json` and insert the following after the `"deepseek/deepseek-v3.2"` entry (after line 12516):
```json
"deepseek-v4-flash": {
"input_cost_per_token": 1.4e-07,
"output_cost_per_token": 2.8e-07,
"cache_read_input_token_cost": 2.8e-09,
"litellm_provider": "deepseek",
"max_input_tokens": 1000000,
"max_output_tokens": 384000,
"max_tokens": 384000,
"mode": "chat",
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_prompt_caching": true,
"source": "https://api-docs.deepseek.com/quick_start/pricing"
},
"deepseek-v4-pro": {
"input_cost_per_token": 4.35e-07,
"output_cost_per_token": 8.7e-07,
"cache_read_input_token_cost": 3.625e-09,
"litellm_provider": "deepseek",
"max_input_tokens": 1000000,
"max_output_tokens": 384000,
"max_tokens": 384000,
"mode": "chat",
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_prompt_caching": true,
"source": "https://api-docs.deepseek.com/quick_start/pricing"
},
```
Also add provider-prefixed variants after the bare entries (following the pattern where `deepseek/deepseek-v3.2` exists alongside `deepseek-v3-2-251201`):
```json
"deepseek/deepseek-v4-flash": {
"input_cost_per_token": 1.4e-07,
"output_cost_per_token": 2.8e-07,
"cache_read_input_token_cost": 2.8e-09,
"input_cost_per_token_cache_hit": 2.8e-09,
"litellm_provider": "deepseek",
"max_input_tokens": 1000000,
"max_output_tokens": 384000,
"max_tokens": 384000,
"mode": "chat",
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_prompt_caching": true,
"supports_assistant_prefill": true,
"source": "https://api-docs.deepseek.com/quick_start/pricing"
},
"deepseek/deepseek-v4-pro": {
"input_cost_per_token": 4.35e-07,
"output_cost_per_token": 8.7e-07,
"cache_read_input_token_cost": 3.625e-09,
"input_cost_per_token_cache_hit": 3.625e-09,
"litellm_provider": "deepseek",
"max_input_tokens": 1000000,
"max_output_tokens": 384000,
"max_tokens": 384000,
"mode": "chat",
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_prompt_caching": true,
"supports_assistant_prefill": true,
"source": "https://api-docs.deepseek.com/quick_start/pricing"
},
```
Note: The prefixed variants use `input_cost_per_token_cache_hit` (LiteLLM modern field) alongside `cache_read_input_token_cost` (legacy field) following the convention of existing entries like `deepseek/deepseek-v3.2`.
- [ ] **Step 2: Validate the JSON is still well-formed**
```bash
cd D:/headroom && python -m json.tool crates/headroom-proxy/data/model_prices_and_context_window.json > /dev/null && echo "Valid JSON"
```
Expected: "Valid JSON"
- [ ] **Step 3: Run full test suite to confirm no regressions**
```bash
cd D:/headroom && python -m pytest tests/test_pricing.py tests/test_pricing_litellm.py tests/test_providers/test_deepseek.py -v
```
Expected: All tests pass.
- [ ] **Step 4: Commit**
```bash
git add crates/headroom-proxy/data/model_prices_and_context_window.json
git commit -m "feat(proxy): add DeepSeek V4 context window entries to vendored JSON
Add bare and provider-prefixed JSON entries for deepseek-v4-flash
and deepseek-v4-pro so Rust-side context window lookups work.
Co-Authored-By: Claude <noreply@anthropic.com>"
```
---
### Task 5: Run All Tests and Final Verification
- [ ] **Step 1: Run all pricing-related tests**
```bash
cd D:/headroom && python -m pytest tests/test_pricing.py tests/test_pricing_litellm.py tests/test_litellm_optional.py tests/test_providers/test_anthropic.py tests/test_providers/test_openai.py tests/test_providers/test_deepseek.py tests/test_providers/test_universal.py tests/test_cost_tracker_counterfactual.py tests/test_proxy_savings_history.py tests/test_provider_model_fallback.py -v
```
Expected: All tests pass. Note any skips (some may require litellm installed).
- [ ] **Step 2: Verify imports work cleanly**
```bash
cd D:/headroom && python -c "from headroom.pricing import DEEPSEEK_PRICES, get_deepseek_registry; print('Direct import OK:', list(DEEPSEEK_PRICES.keys()))"
cd D:/headroom && python -c "from headroom.pricing.litellm_pricing import get_model_pricing; p = get_model_pricing('deepseek-v4-flash'); print('LiteLLM pricing OK:', p.input_cost_per_1m if p else 'None')"
cd D:/headroom && python -c "from headroom.providers.anthropic import AnthropicProvider; p = AnthropicProvider(); print('Anthropic fallback OK:', p._get_pricing('deepseek-v4-flash'))"
```
Expected: All three commands print correct pricing values without errors.
- [ ] **Step 3: Verify the full proxy cost tracker end-to-end**
```bash
cd D:/headroom && python -c "
from headroom.proxy.cost import CostTracker
t = CostTracker()
cost = t.estimate_cost('deepseek-v4-flash', input_tokens=1000000, output_tokens=1000000)
print(f'CostTracker estimate: \${cost:.4f}' if cost else 'CostTracker returned None')
"
```
Expected: Prints `CostTracker estimate: $0.4200` (0.14 input + 0.28 output, may vary if litellm is available).
- [ ] **Step 4: Final commit if any fixes were needed**
```bash
git log --oneline -5
```
Expected: Shows the 4 commit history for this feature.

169
docs/superpowers/specs/2026-06-25-kompress-finetune-design.md
View file

@ -1,169 +0,0 @@
# Kompress Fine-Tune Design
**Date:** 2026-06-25
**Status:** Approved
**Owner:** peterlodri-sec
## Goal
Fine-tune `chopratejas/kompress-v2-base` (ModernBERT ~149M) to:
- **B — Quality push:** lower keep_rate from 0.81 toward 0.72–0.75 while holding must_keep_recall above 0.97
- **C2 — Domain profiles:** teach domain-specific compression intuitions for five input types (code diffs, log streams, JSON blobs, prose/markdown, file trees)
- **C3 — Self-distillation:** use headroom's own proxy compression logs as labeled training data
Deliverables: fine-tuned model + ONNX artifacts, a blog post, and a Jupyter notebook with a Colab quick-start path and a vast.ai production path.
---
## 1. Data Pipeline
### 1.1 Domain-tagged datasets
Each input sequence is prefixed with a domain token so the model builds per-domain compression intuitions rather than one global policy.
| Domain | Prefix | Source | Keep signal | Drop signal |
|--------|--------|--------|-------------|-------------|
| Code diffs | `[CODE]` | `codeparrot/github-code` + open PR diffs | `+`/`-` lines, function/class signatures, imports | whitespace, unchanged context, comments |
| Log streams | `[LOG]` | Loghub (Apache/HDFS/Linux) | ERROR/WARN/EXCEPTION, stack frames, unique messages | repeated INFO, timestamps, DEBUG noise |
| JSON blobs | `[JSON]` | headroom test data + synthetic API responses | non-null leaf values, rare keys (<5% frequency) | null, empty arrays, boilerplate schema |
| Prose/markdown | `[PROSE]` | HuggingFace docs, GitHub READMEs | key claims, numbers, definitions (TF-IDF top-20%) | transition sentences, repeated examples |
| File trees | `[TREE]` | synthetic from real filesystem structures | non-standard paths, recently modified indicators | `.git/`, stdlib paths, permission columns |
**Total:** ~50k samples. Colab subset: ~3k (one domain).
**Split:** 80/10/10 train/val/test, stratified by domain.
### 1.2 C3 — Headroom self-distillation
headroom's proxy compression logs contain (original_text, compressed_text) pairs from real production requests. Token-level keep/drop labels are recovered by diffing the token sequences. These are real usage decisions — the strongest training signal for the model's actual deployment context.
Extraction script: reads from headroom's local proxy log directory, tokenizes both sides with the kompress tokenizer, aligns, and outputs labeled sequences tagged `[HDR]`.
### 1.3 Labeling heuristics
Heuristics generate weak labels for the five domain datasets. They are intentionally conservative — false positives (keep too much) are preferred over false negatives (drop something important). The model learns to be more aggressive; the heuristics just establish the floor.
---
## 2. Training Setup
**Platform:** RTX 4090 24GB on vast.ai. ~$0.50/hr. Full run costs ~$0.70–1.00. Budget ($6–7) covers 6–10 experiments.
**Full fine-tune, no LoRA.** ModernBERT at 149M sits at ~4GB in bf16. A 4090 has 24GB — no reason to constrain.
### 2.1 Hyperparameters
| Setting | Value | Reason |
|---------|-------|--------|
| Base model | `chopratejas/kompress-v2-base` | existing checkpoint |
| Task | token classification, binary (0=drop, 1=keep) | same as kompress v2 |
| Loss | weighted cross-entropy, keep_weight=2.5 | penalizes false drops to protect recall |
| Learning rate | 2e-5, cosine decay | standard for BERT-class fine-tune |
| Warmup | 10% of total steps | |
| Batch size | 32 sequences, seq_len=512 | fits 4090; matches kompress inference window |
| Epochs | 3 with early stopping on val must_keep_recall | |
| Optimizer | AdamW, weight_decay=0.01 | |
| Precision | bf16 | |
### 2.2 B — Threshold calibration (post-training)
After training, sweep the classification threshold from 0.3 to 0.7 in steps of 0.02. For each threshold compute keep_rate and must_keep_recall on the validation set. Select the highest-compression threshold (lowest keep_rate) where must_keep_recall stays above 0.97. One forward pass — no retraining.
### 2.3 Evaluation
Metrics reported per domain and blended:
- **f1** — overall classification quality
- **must_keep_recall** — fraction of ground-truth keep tokens that are kept; hard floor 0.97
- **keep_rate** — fraction of tokens kept; target 0.72–0.75 (down from 0.81)
Compared against kompress-v2-base baseline on the same held-out test splits.
### 2.4 ONNX export
Two artifacts, matching headroom's existing naming:
- `kompress-int8-wo.onnx` — weight-only int8 (MatMulNBits), drop-in replacement for the current 261MB artifact
- `kompress-fp32.onnx` — lossless reference
Both pushed to HuggingFace Hub as a new model repo.
---
## 3. Blog Post
**Title:** "Language Immersion at 149M Parameters"
**Published to:** `pocoo.vaked.dev` (existing post format)
**Framing:** The Sapir-Whorf hypothesis says the language you speak shapes the thoughts you can have. Kompress is trained to think in compressed language — not filtering noise but internalizing a new grammar where redundant tokens don't exist. Domain fine-tuning is immersion: the model develops native compression intuitions per dialect (code, logs, JSON, prose, trees) instead of one blunt global policy.
**Structure:**
1. **Hook** — the hypothesis; one paragraph; "what if the way an AI reads context determines what it's capable of thinking?"
2. **The problem** — tool output noise filling context; kompress's token classification job
3. **What kompress does** — ModernBERT architecture, current metrics (f1=0.913, keep_rate=0.81)
4. **Domain immersion** — why code diffs compress differently than log streams; the domain prefix token trick
5. **The dogfood loop** — headroom proxy traffic as teacher; self-distillation; eating your own cooking
6. **The training run** — vast.ai, the numbers, total cost (~$0.70)
7. **Results** — before/after metrics table per domain; threshold calibration curve
8. **Notebook** — link + how to reproduce on Colab or your own GPU
---
## 4. Jupyter Notebook
**File:** `kompress-finetune.ipynb`
**Hosted:** HuggingFace Hub alongside the model, linked from the blog post.
### Part 1: Quick Start (Colab / Kaggle, T4, ~15 min)
- Install deps (`transformers`, `datasets`, `torch`)
- Load `kompress-v2-base`
- Load 3k-sample subset (one domain, pre-labeled)
- Fine-tune 1 epoch
- Threshold calibration sweep
- Eval: f1 / must_keep_recall / keep_rate
### Part 2: Production Run (vast.ai / self-hosted 4090)
- Rent instance walkthrough (vast.ai CLI commands)
- Full 5-domain data pipeline
- Headroom log extraction (C3 self-distillation)
- Full fine-tune (3 epochs)
- Per-domain eval
- ONNX export (int8-wo + fp32)
- Push to HuggingFace Hub
Both parts share the same training cell. Only the dataset loading and export differ. A Colab reader sees the full pipeline structure and can swap in their own data later.
---
## 5. File Layout
```
headroom/
scripts/
kompress_finetune/
data/
build_dataset.py # domain dataset builder + headroom log extractor
label_heuristics.py # weak labeling per domain
train.py # training entry point (HF Trainer)
calibrate.py # threshold sweep post-training
export_onnx.py # int8-wo + fp32 ONNX export
eval.py # per-domain metrics
notebooks/
kompress-finetune.ipynb # the split notebook
pocoo.vaked.dev/
src/posts/
kompress-finetune-sapir-whorf.md # blog post
```
---
## 6. Out of Scope
- LoRA / adapter-per-domain (full fine-tune is sufficient at 149M)
- Contrastive training objective (deferred to a future run)
- Deploying the new model to headroom main branch (separate PR, after eval)
- Training on GPU larger than 4090 (A100 is overkill and burns budget)

1
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/.bin/esbuild generated vendored
View file

@ -1 +0,0 @@
../esbuild/bin/esbuild

1
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/.bin/tsc generated vendored
View file

@ -1 +0,0 @@
../typescript/bin/tsc

1
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/.bin/tsserver generated vendored
View file

@ -1 +0,0 @@
../typescript/bin/tsserver

1
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/.bin/tsx generated vendored
View file

@ -1 +0,0 @@
../tsx/dist/cli.mjs

324
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/.package-lock.json generated vendored
View file

@ -1,324 +0,0 @@
{
"name": "@example/node-headroom-compression",
"version": "0.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"node_modules/@ai-sdk/anthropic": {
"version": "3.0.64",
"resolved": "https://registry.npmjs.org/@ai-sdk/anthropic/-/anthropic-3.0.64.tgz",
"integrity": "sha512-rwLi/Rsuj2pYniQXIrvClHvXDzgM4UQHHnvHTWEF14efnlKclG/1ghpNC+adsRujAbCTr6gRsSbDE2vEqriV7g==",
"license": "Apache-2.0",
"dependencies": {
"@ai-sdk/provider": "3.0.8",
"@ai-sdk/provider-utils": "4.0.21"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"zod": "^3.25.76 || ^4.1.8"
}
},
"node_modules/@ai-sdk/gateway": {
"version": "3.0.80",
"resolved": "https://registry.npmjs.org/@ai-sdk/gateway/-/gateway-3.0.80.tgz",
"integrity": "sha512-uM7kpZB5l977lW7+2X1+klBUxIZQ78+1a9jHlaHFEzcOcmmslTl3sdP0QqfuuBcO0YBM2gwOiqVdp8i4TRQYcw==",
"license": "Apache-2.0",
"dependencies": {
"@ai-sdk/provider": "3.0.8",
"@ai-sdk/provider-utils": "4.0.21",
"@vercel/oidc": "3.1.0"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"zod": "^3.25.76 || ^4.1.8"
}
},
"node_modules/@ai-sdk/openai": {
"version": "3.0.48",
"resolved": "https://registry.npmjs.org/@ai-sdk/openai/-/openai-3.0.48.tgz",
"integrity": "sha512-ALmj/53EXpcRqMbGpPJPP4UOSWw0q4VGpnDo7YctvsynjkrKDmoneDG/1a7VQnSPYHnJp6tTRMf5ZdxZ5whulg==",
"license": "Apache-2.0",
"dependencies": {
"@ai-sdk/provider": "3.0.8",
"@ai-sdk/provider-utils": "4.0.21"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"zod": "^3.25.76 || ^4.1.8"
}
},
"node_modules/@ai-sdk/provider": {
"version": "3.0.8",
"resolved": "https://registry.npmjs.org/@ai-sdk/provider/-/provider-3.0.8.tgz",
"integrity": "sha512-oGMAgGoQdBXbZqNG0Ze56CHjDZ1IDYOwGYxYjO5KLSlz5HiNQ9udIXsPZ61VWaHGZ5XW/jyjmr6t2xz2jGVwbQ==",
"license": "Apache-2.0",
"dependencies": {
"json-schema": "^0.4.0"
},
"engines": {
"node": ">=18"
}
},
"node_modules/@ai-sdk/provider-utils": {
"version": "4.0.21",
"resolved": "https://registry.npmjs.org/@ai-sdk/provider-utils/-/provider-utils-4.0.21.tgz",
"integrity": "sha512-MtFUYI1/8mgDvRmaBDjbLJPFFrMG777AvSgyIFQtZHIMzm88R/12vYBBpnk7pfiWLFE1DSZzY4WDYzGbKAcmiw==",
"license": "Apache-2.0",
"dependencies": {
"@ai-sdk/provider": "3.0.8",
"@standard-schema/spec": "^1.1.0",
"eventsource-parser": "^3.0.6"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"zod": "^3.25.76 || ^4.1.8"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.27.4.tgz",
"integrity": "sha512-b7xaGIwdJlht8ZFCvMkpDN6uiSmnxxK56N2GDTMYPr2/gzvfdQN8rTfBsvVKmIVY/X7EM+/hJKEIbbHs9oA4tQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@opentelemetry/api": {
"version": "1.9.0",
"resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.0.tgz",
"integrity": "sha512-3giAOQvZiH5F9bMlMiv8+GSPMeqg0dbaeo58/0SlA9sxSqZhnUtxzX9/2FzyhS9sWQf5S0GJE0AKBrFqjpeYcg==",
"license": "Apache-2.0",
"engines": {
"node": ">=8.0.0"
}
},
"node_modules/@standard-schema/spec": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz",
"integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==",
"license": "MIT"
},
"node_modules/@vercel/oidc": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/@vercel/oidc/-/oidc-3.1.0.tgz",
"integrity": "sha512-Fw28YZpRnA3cAHHDlkt7xQHiJ0fcL+NRcIqsocZQUSmbzeIKRpwttJjik5ZGanXP+vlA4SbTg+AbA3bP363l+w==",
"license": "Apache-2.0",
"engines": {
"node": ">= 20"
}
},
"node_modules/ai": {
"version": "6.0.138",
"resolved": "https://registry.npmjs.org/ai/-/ai-6.0.138.tgz",
"integrity": "sha512-49OfPe0f5uxJ6jUdA5BBXjIinP6+ZdYfAtpF2aEH64GA5wPcxH2rf/TBUQQ0bbamBz/D+TLMV18xilZqOC+zaA==",
"license": "Apache-2.0",
"dependencies": {
"@ai-sdk/gateway": "3.0.80",
"@ai-sdk/provider": "3.0.8",
"@ai-sdk/provider-utils": "4.0.21",
"@opentelemetry/api": "1.9.0"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"zod": "^3.25.76 || ^4.1.8"
}
},
"node_modules/dotenv": {
"version": "17.3.1",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-17.3.1.tgz",
"integrity": "sha512-IO8C/dzEb6O3F9/twg6ZLXz164a2fhTnEWb95H23Dm4OuN+92NmEAlTrupP9VW6Jm3sO26tQlqyvyi4CsnY9GA==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://dotenvx.com"
}
},
"node_modules/esbuild": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.4.tgz",
"integrity": "sha512-Rq4vbHnYkK5fws5NF7MYTU68FPRE1ajX7heQ/8QXXWqNgqqJ/GkmmyxIzUnf2Sr/bakf8l54716CcMGHYhMrrQ==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.27.4",
"@esbuild/android-arm": "0.27.4",
"@esbuild/android-arm64": "0.27.4",
"@esbuild/android-x64": "0.27.4",
"@esbuild/darwin-arm64": "0.27.4",
"@esbuild/darwin-x64": "0.27.4",
"@esbuild/freebsd-arm64": "0.27.4",
"@esbuild/freebsd-x64": "0.27.4",
"@esbuild/linux-arm": "0.27.4",
"@esbuild/linux-arm64": "0.27.4",
"@esbuild/linux-ia32": "0.27.4",
"@esbuild/linux-loong64": "0.27.4",
"@esbuild/linux-mips64el": "0.27.4",
"@esbuild/linux-ppc64": "0.27.4",
"@esbuild/linux-riscv64": "0.27.4",
"@esbuild/linux-s390x": "0.27.4",
"@esbuild/linux-x64": "0.27.4",
"@esbuild/netbsd-arm64": "0.27.4",
"@esbuild/netbsd-x64": "0.27.4",
"@esbuild/openbsd-arm64": "0.27.4",
"@esbuild/openbsd-x64": "0.27.4",
"@esbuild/openharmony-arm64": "0.27.4",
"@esbuild/sunos-x64": "0.27.4",
"@esbuild/win32-arm64": "0.27.4",
"@esbuild/win32-ia32": "0.27.4",
"@esbuild/win32-x64": "0.27.4"
}
},
"node_modules/eventsource-parser": {
"version": "3.0.6",
"resolved": "https://registry.npmjs.org/eventsource-parser/-/eventsource-parser-3.0.6.tgz",
"integrity": "sha512-Vo1ab+QXPzZ4tCa8SwIHJFaSzy4R6SHf7BY79rFBDf0idraZWAkYrDjDj8uWaSm3S2TK+hJ7/t1CEmZ7jXw+pg==",
"license": "MIT",
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/fsevents": {
"version": "2.3.3",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/get-tsconfig": {
"version": "4.13.7",
"resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.13.7.tgz",
"integrity": "sha512-7tN6rFgBlMgpBML5j8typ92BKFi2sFQvIdpAqLA2beia5avZDrMs0FLZiM5etShWq5irVyGcGMEA1jcDaK7A/Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"resolve-pkg-maps": "^1.0.0"
},
"funding": {
"url": "https://github.com/privatenumber/get-tsconfig?sponsor=1"
}
},
"node_modules/headroom-ai": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/headroom-ai/-/headroom-ai-0.1.0.tgz",
"integrity": "sha512-SM5i2kptwsO+KnQqpaLThZVx1ZrmobVw5JRXwF5OvnMvhV/3WbwUI+VvXHTwTCvVq+AAyhkhuVcNXod1piAVuQ==",
"license": "Apache-2.0",
"engines": {
"node": ">=18.0.0"
},
"peerDependencies": {
"@ai-sdk/provider": ">=1.0.0",
"@anthropic-ai/sdk": ">=0.30.0",
"ai": ">=6.0.0",
"openai": ">=4.0.0"
},
"peerDependenciesMeta": {
"@ai-sdk/provider": {
"optional": true
},
"@anthropic-ai/sdk": {
"optional": true
},
"ai": {
"optional": true
},
"openai": {
"optional": true
}
}
},
"node_modules/json-schema": {
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/json-schema/-/json-schema-0.4.0.tgz",
"integrity": "sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA==",
"license": "(AFL-2.1 OR BSD-3-Clause)"
},
"node_modules/resolve-pkg-maps": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz",
"integrity": "sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==",
"dev": true,
"license": "MIT",
"funding": {
"url": "https://github.com/privatenumber/resolve-pkg-maps?sponsor=1"
}
},
"node_modules/tsx": {
"version": "4.21.0",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz",
"integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==",
"dev": true,
"license": "MIT",
"dependencies": {
"esbuild": "~0.27.0",
"get-tsconfig": "^4.7.5"
},
"bin": {
"tsx": "dist/cli.mjs"
},
"engines": {
"node": ">=18.0.0"
},
"optionalDependencies": {
"fsevents": "~2.3.3"
}
},
"node_modules/typescript": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.2.tgz",
"integrity": "sha512-bGdAIrZ0wiGDo5l8c++HWtbaNCWTS4UTv7RaTH/ThVIgjkveJt83m74bBHMJkuCbslY8ixgLBVZJIOiQlQTjfQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"node_modules/zod": {
"version": "4.3.6",
"resolved": "https://registry.npmjs.org/zod/-/zod-4.3.6.tgz",
"integrity": "sha512-rftlrkhHZOcjDwkGlnUtZZkvaPHCsDATp4pGpuOOMDaTdDDXF91wuVDJoWoPsKX/3YPQ5fHuF3STjcYyKr+Qhg==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
}
}
}

2560
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/CHANGELOG.md generated vendored
View file

File diff suppressed because it is too large Load diff

13
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/LICENSE generated vendored
View file

@ -1,13 +0,0 @@
Copyright 2023 Vercel, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

43
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/README.md generated vendored
View file

@ -1,43 +0,0 @@
# AI SDK - Anthropic Provider
The **[Anthropic provider](https://ai-sdk.dev/providers/ai-sdk-providers/anthropic)** for the [AI SDK](https://ai-sdk.dev/docs) contains language model support for the [Anthropic Messages API](https://docs.anthropic.com/claude/reference/messages_post).
## Setup
The Anthropic provider is available in the `@ai-sdk/anthropic` module. You can install it with
```bash
npm i @ai-sdk/anthropic
```
## Skill for Coding Agents
If you use coding agents such as Claude Code or Cursor, we highly recommend adding the AI SDK skill to your repository:
```shell
npx skills add vercel/ai
```
## Provider Instance
You can import the default provider instance `anthropic` from `@ai-sdk/anthropic`:
```ts
import { anthropic } from '@ai-sdk/anthropic';
```
## Example
```ts
import { anthropic } from '@ai-sdk/anthropic';
import { generateText } from 'ai';
const { text } = await generateText({
model: anthropic('claude-3-haiku-20240307'),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```
## Documentation
Please check out the **[Anthropic provider documentation](https://ai-sdk.dev/providers/ai-sdk-providers/anthropic)** for more information.

1368
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/docs/05-anthropic.mdx generated vendored
View file

File diff suppressed because it is too large Load diff

1
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/internal.d.ts generated vendored
View file

@ -1 +0,0 @@
export * from './dist/internal';

81
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/package.json generated vendored
View file

@ -1,81 +0,0 @@
{
"name": "@ai-sdk/anthropic",
"version": "3.0.64",
"license": "Apache-2.0",
"sideEffects": false,
"main": "./dist/index.js",
"module": "./dist/index.mjs",
"types": "./dist/index.d.ts",
"files": [
"dist/**/*",
"docs/**/*",
"src",
"!src/**/*.test.ts",
"!src/**/*.test-d.ts",
"!src/**/__snapshots__",
"!src/**/__fixtures__",
"CHANGELOG.md",
"README.md",
"internal.d.ts"
],
"directories": {
"doc": "./docs"
},
"exports": {
"./package.json": "./package.json",
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.mjs",
"require": "./dist/index.js"
},
"./internal": {
"types": "./dist/internal/index.d.ts",
"import": "./dist/internal/index.mjs",
"module": "./dist/internal/index.mjs",
"require": "./dist/internal/index.js"
}
},
"dependencies": {
"@ai-sdk/provider": "3.0.8",
"@ai-sdk/provider-utils": "4.0.21"
},
"devDependencies": {
"@types/node": "20.17.24",
"tsup": "^8",
"typescript": "5.8.3",
"zod": "3.25.76",
"@ai-sdk/test-server": "1.0.3",
"@vercel/ai-tsconfig": "0.0.0"
},
"peerDependencies": {
"zod": "^3.25.76 || ^4.1.8"
},
"engines": {
"node": ">=18"
},
"publishConfig": {
"access": "public"
},
"homepage": "https://ai-sdk.dev/docs",
"repository": {
"type": "git",
"url": "git+https://github.com/vercel/ai.git"
},
"bugs": {
"url": "https://github.com/vercel/ai/issues"
},
"keywords": [
"ai"
],
"scripts": {
"build": "pnpm clean && tsup --tsconfig tsconfig.build.json",
"build:watch": "pnpm clean && tsup --watch --tsconfig tsconfig.build.json",
"clean": "del-cli dist docs *.tsbuildinfo",
"type-check": "tsc --build",
"test": "pnpm test:node && pnpm test:edge",
"test:update": "pnpm test:node -u",
"test:watch": "vitest --config vitest.node.config.js",
"test:edge": "vitest --config vitest.edge.config.js --run",
"test:node": "vitest --config vitest.node.config.js --run"
}
}

26
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/anthropic-error.ts generated vendored
View file

@ -1,26 +0,0 @@
import {
createJsonErrorResponseHandler,
InferSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
export const anthropicErrorDataSchema = lazySchema(() =>
zodSchema(
z.object({
type: z.literal('error'),
error: z.object({
type: z.string(),
message: z.string(),
}),
}),
),
);
export type AnthropicErrorData = InferSchema<typeof anthropicErrorDataSchema>;
export const anthropicFailedResponseHandler = createJsonErrorResponseHandler({
errorSchema: anthropicErrorDataSchema,
errorToMessage: data => data.error.message,
});

143
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/anthropic-message-metadata.ts generated vendored
View file

@ -1,143 +0,0 @@
import { JSONObject } from '@ai-sdk/provider';
/**
* Represents a single iteration in the usage breakdown.
* When compaction occurs, the API returns an iterations array showing
* usage for each sampling iteration (compaction + message).
*/
export interface AnthropicUsageIteration {
type: 'compaction' | 'message';
/**
* Number of input tokens consumed in this iteration.
*/
inputTokens: number;
/**
* Number of output tokens generated in this iteration.
*/
outputTokens: number;
}
export interface AnthropicMessageMetadata {
usage: JSONObject;
// TODO remove cacheCreationInputTokens in AI SDK 6
// (use value in usage object instead)
cacheCreationInputTokens: number | null;
stopSequence: string | null;
/**
* Usage breakdown by iteration when compaction is triggered.
*
* When compaction occurs, this array contains usage for each sampling iteration.
* The first iteration is typically the compaction step, followed by the main
* message iteration.
*/
iterations: AnthropicUsageIteration[] | null;
/**
* Information about the container used in this request.
*
* This will be non-null if a container tool (e.g., code execution) was used.
* Information about the container used in the request (for the code execution tool).
*/
container: {
/**
* The time at which the container will expire (RFC3339 timestamp).
*/
expiresAt: string;
/**
* Identifier for the container used in this request.
*/
id: string;
/**
* Skills loaded in the container.
*/
skills: Array<{
/**
* Type of skill: either 'anthropic' (built-in) or 'custom' (user-defined).
*/
type: 'anthropic' | 'custom';
/**
* Skill ID (1-64 characters).
*/
skillId: string;
/**
* Skill version or 'latest' for most recent version (1-64 characters).
*/
version: string;
}> | null;
} | null;
/**
* Context management response.
*
* Information about context management strategies applied during the request.
*/
contextManagement: {
/**
* List of context management edits that were applied.
* Each item in the array is a specific type of context management edit.
*/
appliedEdits: Array<
/**
* Represents an edit where a certain number of tool uses and input tokens were cleared.
*/
| {
/**
* The type of context management edit applied.
* Possible value: 'clear_tool_uses_20250919'
*/
type: 'clear_tool_uses_20250919';
/**
* Number of tool uses that were cleared by this edit.
* Minimum: 0
*/
clearedToolUses: number;
/**
* Number of input tokens cleared by this edit.
* Minimum: 0
*/
clearedInputTokens: number;
}
/**
* Represents an edit where a certain number of thinking turns and input tokens were cleared.
*/
| {
/**
* The type of context management edit applied.
* Possible value: 'clear_thinking_20251015'
*/
type: 'clear_thinking_20251015';
/**
* Number of thinking turns that were cleared by this edit.
* Minimum: 0
*/
clearedThinkingTurns: number;
/**
* Number of input tokens cleared by this edit.
* Minimum: 0
*/
clearedInputTokens: number;
}
/**
* Represents a compaction edit where the conversation context was summarized.
*/
| {
/**
* The type of context management edit applied.
* Possible value: 'compact_20260112'
*/
type: 'compact_20260112';
}
>;
} | null;
}

1344
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/anthropic-messages-api.ts generated vendored
View file

File diff suppressed because it is too large Load diff

2392
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/anthropic-messages-language-model.ts generated vendored
View file

File diff suppressed because it is too large Load diff

263
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/anthropic-messages-options.ts generated vendored
View file

@ -1,263 +0,0 @@
import { z } from 'zod/v4';
// https://docs.claude.com/en/docs/about-claude/models/overview
export type AnthropicMessagesModelId =
| 'claude-3-haiku-20240307'
| 'claude-haiku-4-5-20251001'
| 'claude-haiku-4-5'
| 'claude-opus-4-0'
| 'claude-opus-4-20250514'
| 'claude-opus-4-1-20250805'
| 'claude-opus-4-1'
| 'claude-opus-4-5'
| 'claude-opus-4-5-20251101'
| 'claude-sonnet-4-0'
| 'claude-sonnet-4-20250514'
| 'claude-sonnet-4-5-20250929'
| 'claude-sonnet-4-5'
| 'claude-sonnet-4-6'
| 'claude-opus-4-6'
| (string & {});
/**
* Anthropic file part provider options for document-specific features.
* These options apply to individual file parts (documents).
*/
export const anthropicFilePartProviderOptions = z.object({
/**
* Citation configuration for this document.
* When enabled, this document will generate citations in the response.
*/
citations: z
.object({
/**
* Enable citations for this document
*/
enabled: z.boolean(),
})
.optional(),
/**
* Custom title for the document.
* If not provided, the filename will be used.
*/
title: z.string().optional(),
/**
* Context about the document that will be passed to the model
* but not used towards cited content.
* Useful for storing document metadata as text or stringified JSON.
*/
context: z.string().optional(),
});
export type AnthropicFilePartProviderOptions = z.infer<
typeof anthropicFilePartProviderOptions
>;
export const anthropicLanguageModelOptions = z.object({
/**
* Whether to send reasoning to the model.
*
* This allows you to deactivate reasoning inputs for models that do not support them.
*/
sendReasoning: z.boolean().optional(),
/**
* Determines how structured outputs are generated.
*
* - `outputFormat`: Use the `output_config.format` parameter to specify the structured output format.
* - `jsonTool`: Use a special 'json' tool to specify the structured output format.
* - `auto`: Use 'outputFormat' when supported, otherwise use 'jsonTool' (default).
*/
structuredOutputMode: z.enum(['outputFormat', 'jsonTool', 'auto']).optional(),
/**
* Configuration for enabling Claude's extended thinking.
*
* When enabled, responses include thinking content blocks showing Claude's thinking process before the final answer.
* Requires a minimum budget of 1,024 tokens and counts towards the `max_tokens` limit.
*/
thinking: z
.discriminatedUnion('type', [
z.object({
/** for Sonnet 4.6, Opus 4.6, and newer models */
type: z.literal('adaptive'),
}),
z.object({
/** for models before Opus 4.6, except Sonnet 4.6 still supports it */
type: z.literal('enabled'),
budgetTokens: z.number().optional(),
}),
z.object({
type: z.literal('disabled'),
}),
])
.optional(),
/**
* Whether to disable parallel function calling during tool use. Default is false.
* When set to true, Claude will use at most one tool per response.
*/
disableParallelToolUse: z.boolean().optional(),
/**
* Cache control settings for this message.
* See https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
*/
cacheControl: z
.object({
type: z.literal('ephemeral'),
ttl: z.union([z.literal('5m'), z.literal('1h')]).optional(),
})
.optional(),
/**
* Metadata to include with the request.
*
* See https://platform.claude.com/docs/en/api/messages/create for details.
*/
metadata: z
.object({
/**
* An external identifier for the user associated with the request.
*
* Should be a UUID, hash value, or other opaque identifier.
* Must not contain PII (name, email, phone number, etc.).
*/
userId: z.string().optional(),
})
.optional(),
/**
* MCP servers to be utilized in this request.
*/
mcpServers: z
.array(
z.object({
type: z.literal('url'),
name: z.string(),
url: z.string(),
authorizationToken: z.string().nullish(),
toolConfiguration: z
.object({
enabled: z.boolean().nullish(),
allowedTools: z.array(z.string()).nullish(),
})
.nullish(),
}),
)
.optional(),
/**
* Agent Skills configuration. Skills enable Claude to perform specialized tasks
* like document processing (PPTX, DOCX, PDF, XLSX) and data analysis.
* Requires code execution tool to be enabled.
*/
container: z
.object({
id: z.string().optional(),
skills: z
.array(
z.object({
type: z.union([z.literal('anthropic'), z.literal('custom')]),
skillId: z.string(),
version: z.string().optional(),
}),
)
.optional(),
})
.optional(),
/**
* Whether to enable tool streaming (and structured output streaming).
*
* When set to false, the model will return all tool calls and results
* at once after a delay.
*
* @default true
*/
toolStreaming: z.boolean().optional(),
/**
* @default 'high'
*/
effort: z.enum(['low', 'medium', 'high', 'max']).optional(),
/**
* Enable fast mode for faster inference (2.5x faster output token speeds).
* Only supported with claude-opus-4-6.
*/
speed: z.enum(['fast', 'standard']).optional(),
/**
* A set of beta features to enable.
* Allow a provider to receive the full `betas` set if it needs it.
*/
anthropicBeta: z.array(z.string()).optional(),
contextManagement: z
.object({
edits: z.array(
z.discriminatedUnion('type', [
z.object({
type: z.literal('clear_tool_uses_20250919'),
trigger: z
.discriminatedUnion('type', [
z.object({
type: z.literal('input_tokens'),
value: z.number(),
}),
z.object({
type: z.literal('tool_uses'),
value: z.number(),
}),
])
.optional(),
keep: z
.object({
type: z.literal('tool_uses'),
value: z.number(),
})
.optional(),
clearAtLeast: z
.object({
type: z.literal('input_tokens'),
value: z.number(),
})
.optional(),
clearToolInputs: z.boolean().optional(),
excludeTools: z.array(z.string()).optional(),
}),
z.object({
type: z.literal('clear_thinking_20251015'),
keep: z
.union([
z.literal('all'),
z.object({
type: z.literal('thinking_turns'),
value: z.number(),
}),
])
.optional(),
}),
z.object({
type: z.literal('compact_20260112'),
trigger: z
.object({
type: z.literal('input_tokens'),
value: z.number(),
})
.optional(),
pauseAfterCompaction: z.boolean().optional(),
instructions: z.string().optional(),
}),
]),
),
})
.optional(),
});
export type AnthropicLanguageModelOptions = z.infer<
typeof anthropicLanguageModelOptions
>;

416
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/anthropic-prepare-tools.ts generated vendored
View file

@ -1,416 +0,0 @@
import {
LanguageModelV3CallOptions,
SharedV3Warning,
UnsupportedFunctionalityError,
} from '@ai-sdk/provider';
import { AnthropicTool, AnthropicToolChoice } from './anthropic-messages-api';
import { CacheControlValidator } from './get-cache-control';
import { textEditor_20250728ArgsSchema } from './tool/text-editor_20250728';
import { webSearch_20260209ArgsSchema } from './tool/web-search_20260209';
import { webSearch_20250305ArgsSchema } from './tool/web-search_20250305';
import { webFetch_20260209ArgsSchema } from './tool/web-fetch-20260209';
import { webFetch_20250910ArgsSchema } from './tool/web-fetch-20250910';
import { validateTypes } from '@ai-sdk/provider-utils';
export interface AnthropicToolOptions {
deferLoading?: boolean;
allowedCallers?: Array<
'direct' | 'code_execution_20250825' | 'code_execution_20260120'
>;
eagerInputStreaming?: boolean;
}
export async function prepareTools({
tools,
toolChoice,
disableParallelToolUse,
cacheControlValidator,
supportsStructuredOutput,
supportsStrictTools,
}: {
tools: LanguageModelV3CallOptions['tools'];
toolChoice: LanguageModelV3CallOptions['toolChoice'] | undefined;
disableParallelToolUse?: boolean;
cacheControlValidator?: CacheControlValidator;
/**
* Whether the model supports native structured output response format.
*/
supportsStructuredOutput: boolean;
/**
* Whether the model supports strict mode on tool definitions.
*/
supportsStrictTools: boolean;
}): Promise<{
tools: Array<AnthropicTool> | undefined;
toolChoice: AnthropicToolChoice | undefined;
toolWarnings: SharedV3Warning[];
betas: Set<string>;
}> {
// when the tools array is empty, change it to undefined to prevent errors:
tools = tools?.length ? tools : undefined;
const toolWarnings: SharedV3Warning[] = [];
const betas = new Set<string>();
const validator = cacheControlValidator || new CacheControlValidator();
if (tools == null) {
return { tools: undefined, toolChoice: undefined, toolWarnings, betas };
}
const anthropicTools: AnthropicTool[] = [];
for (const tool of tools) {
switch (tool.type) {
case 'function': {
const cacheControl = validator.getCacheControl(tool.providerOptions, {
type: 'tool definition',
canCache: true,
});
// Read Anthropic-specific provider options
const anthropicOptions = tool.providerOptions?.anthropic as
| AnthropicToolOptions
| undefined;
// eager_input_streaming is only supported on custom (function) tools
const eagerInputStreaming = anthropicOptions?.eagerInputStreaming;
const deferLoading = anthropicOptions?.deferLoading;
const allowedCallers = anthropicOptions?.allowedCallers;
if (!supportsStrictTools && tool.strict != null) {
toolWarnings.push({
type: 'unsupported',
feature: 'strict',
details: `Tool '${tool.name}' has strict: ${tool.strict}, but strict mode is not supported by this provider. The strict property will be ignored.`,
});
}
anthropicTools.push({
name: tool.name,
description: tool.description,
input_schema: tool.inputSchema,
cache_control: cacheControl,
...(eagerInputStreaming ? { eager_input_streaming: true } : {}),
...(supportsStrictTools === true && tool.strict != null
? { strict: tool.strict }
: {}),
...(deferLoading != null ? { defer_loading: deferLoading } : {}),
...(allowedCallers != null
? { allowed_callers: allowedCallers }
: {}),
...(tool.inputExamples != null
? {
input_examples: tool.inputExamples.map(
example => example.input,
),
}
: {}),
});
if (supportsStructuredOutput === true) {
betas.add('structured-outputs-2025-11-13');
}
if (tool.inputExamples != null || allowedCallers != null) {
betas.add('advanced-tool-use-2025-11-20');
}
break;
}
case 'provider': {
// Note: Provider-defined tools don't currently support providerOptions in the SDK,
// so cache_control cannot be set on them. The Anthropic API supports caching all tools,
// but the SDK would need to be updated to expose providerOptions on provider-defined tools.
switch (tool.id) {
case 'anthropic.code_execution_20250522': {
betas.add('code-execution-2025-05-22');
anthropicTools.push({
type: 'code_execution_20250522',
name: 'code_execution',
cache_control: undefined,
});
break;
}
case 'anthropic.code_execution_20250825': {
betas.add('code-execution-2025-08-25');
anthropicTools.push({
type: 'code_execution_20250825',
name: 'code_execution',
});
break;
}
case 'anthropic.code_execution_20260120': {
anthropicTools.push({
type: 'code_execution_20260120',
name: 'code_execution',
});
break;
}
case 'anthropic.computer_20250124': {
betas.add('computer-use-2025-01-24');
anthropicTools.push({
name: 'computer',
type: 'computer_20250124',
display_width_px: tool.args.displayWidthPx as number,
display_height_px: tool.args.displayHeightPx as number,
display_number: tool.args.displayNumber as number,
cache_control: undefined,
});
break;
}
case 'anthropic.computer_20251124': {
betas.add('computer-use-2025-11-24');
anthropicTools.push({
name: 'computer',
type: 'computer_20251124',
display_width_px: tool.args.displayWidthPx as number,
display_height_px: tool.args.displayHeightPx as number,
display_number: tool.args.displayNumber as number,
enable_zoom: tool.args.enableZoom as boolean,
cache_control: undefined,
});
break;
}
case 'anthropic.computer_20241022': {
betas.add('computer-use-2024-10-22');
anthropicTools.push({
name: 'computer',
type: 'computer_20241022',
display_width_px: tool.args.displayWidthPx as number,
display_height_px: tool.args.displayHeightPx as number,
display_number: tool.args.displayNumber as number,
cache_control: undefined,
});
break;
}
case 'anthropic.text_editor_20250124': {
betas.add('computer-use-2025-01-24');
anthropicTools.push({
name: 'str_replace_editor',
type: 'text_editor_20250124',
cache_control: undefined,
});
break;
}
case 'anthropic.text_editor_20241022': {
betas.add('computer-use-2024-10-22');
anthropicTools.push({
name: 'str_replace_editor',
type: 'text_editor_20241022',
cache_control: undefined,
});
break;
}
case 'anthropic.text_editor_20250429': {
betas.add('computer-use-2025-01-24');
anthropicTools.push({
name: 'str_replace_based_edit_tool',
type: 'text_editor_20250429',
cache_control: undefined,
});
break;
}
case 'anthropic.text_editor_20250728': {
const args = await validateTypes({
value: tool.args,
schema: textEditor_20250728ArgsSchema,
});
anthropicTools.push({
name: 'str_replace_based_edit_tool',
type: 'text_editor_20250728',
max_characters: args.maxCharacters,
cache_control: undefined,
});
break;
}
case 'anthropic.bash_20250124': {
betas.add('computer-use-2025-01-24');
anthropicTools.push({
name: 'bash',
type: 'bash_20250124',
cache_control: undefined,
});
break;
}
case 'anthropic.bash_20241022': {
betas.add('computer-use-2024-10-22');
anthropicTools.push({
name: 'bash',
type: 'bash_20241022',
cache_control: undefined,
});
break;
}
case 'anthropic.memory_20250818': {
betas.add('context-management-2025-06-27');
anthropicTools.push({
name: 'memory',
type: 'memory_20250818',
});
break;
}
case 'anthropic.web_fetch_20250910': {
betas.add('web-fetch-2025-09-10');
const args = await validateTypes({
value: tool.args,
schema: webFetch_20250910ArgsSchema,
});
anthropicTools.push({
type: 'web_fetch_20250910',
name: 'web_fetch',
max_uses: args.maxUses,
allowed_domains: args.allowedDomains,
blocked_domains: args.blockedDomains,
citations: args.citations,
max_content_tokens: args.maxContentTokens,
cache_control: undefined,
});
break;
}
case 'anthropic.web_fetch_20260209': {
betas.add('code-execution-web-tools-2026-02-09');
const args = await validateTypes({
value: tool.args,
schema: webFetch_20260209ArgsSchema,
});
anthropicTools.push({
type: 'web_fetch_20260209',
name: 'web_fetch',
max_uses: args.maxUses,
allowed_domains: args.allowedDomains,
blocked_domains: args.blockedDomains,
citations: args.citations,
max_content_tokens: args.maxContentTokens,
cache_control: undefined,
});
break;
}
case 'anthropic.web_search_20250305': {
const args = await validateTypes({
value: tool.args,
schema: webSearch_20250305ArgsSchema,
});
anthropicTools.push({
type: 'web_search_20250305',
name: 'web_search',
max_uses: args.maxUses,
allowed_domains: args.allowedDomains,
blocked_domains: args.blockedDomains,
user_location: args.userLocation,
cache_control: undefined,
});
break;
}
case 'anthropic.web_search_20260209': {
betas.add('code-execution-web-tools-2026-02-09');
const args = await validateTypes({
value: tool.args,
schema: webSearch_20260209ArgsSchema,
});
anthropicTools.push({
type: 'web_search_20260209',
name: 'web_search',
max_uses: args.maxUses,
allowed_domains: args.allowedDomains,
blocked_domains: args.blockedDomains,
user_location: args.userLocation,
cache_control: undefined,
});
break;
}
case 'anthropic.tool_search_regex_20251119': {
anthropicTools.push({
type: 'tool_search_tool_regex_20251119',
name: 'tool_search_tool_regex',
});
break;
}
case 'anthropic.tool_search_bm25_20251119': {
anthropicTools.push({
type: 'tool_search_tool_bm25_20251119',
name: 'tool_search_tool_bm25',
});
break;
}
default: {
toolWarnings.push({
type: 'unsupported',
feature: `provider-defined tool ${tool.id}`,
});
break;
}
}
break;
}
default: {
toolWarnings.push({
type: 'unsupported',
feature: `tool ${tool}`,
});
break;
}
}
}
if (toolChoice == null) {
return {
tools: anthropicTools,
toolChoice: disableParallelToolUse
? { type: 'auto', disable_parallel_tool_use: disableParallelToolUse }
: undefined,
toolWarnings,
betas,
};
}
const type = toolChoice.type;
switch (type) {
case 'auto':
return {
tools: anthropicTools,
toolChoice: {
type: 'auto',
disable_parallel_tool_use: disableParallelToolUse,
},
toolWarnings,
betas,
};
case 'required':
return {
tools: anthropicTools,
toolChoice: {
type: 'any',
disable_parallel_tool_use: disableParallelToolUse,
},
toolWarnings,
betas,
};
case 'none':
// Anthropic does not support 'none' tool choice, so we remove the tools:
return { tools: undefined, toolChoice: undefined, toolWarnings, betas };
case 'tool':
return {
tools: anthropicTools,
toolChoice: {
type: 'tool',
name: toolChoice.toolName,
disable_parallel_tool_use: disableParallelToolUse,
},
toolWarnings,
betas,
};
default: {
const _exhaustiveCheck: never = type;
throw new UnsupportedFunctionalityError({
functionality: `tool choice type: ${_exhaustiveCheck}`,
});
}
}
}

177
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/anthropic-provider.ts generated vendored
View file

@ -1,177 +0,0 @@
import {
InvalidArgumentError,
LanguageModelV3,
NoSuchModelError,
ProviderV3,
} from '@ai-sdk/provider';
import {
FetchFunction,
generateId,
loadApiKey,
loadOptionalSetting,
withoutTrailingSlash,
withUserAgentSuffix,
} from '@ai-sdk/provider-utils';
import { VERSION } from './version';
import { AnthropicMessagesLanguageModel } from './anthropic-messages-language-model';
import { AnthropicMessagesModelId } from './anthropic-messages-options';
import { anthropicTools } from './anthropic-tools';
export interface AnthropicProvider extends ProviderV3 {
/**
* Creates a model for text generation.
*/
(modelId: AnthropicMessagesModelId): LanguageModelV3;
/**
* Creates a model for text generation.
*/
languageModel(modelId: AnthropicMessagesModelId): LanguageModelV3;
chat(modelId: AnthropicMessagesModelId): LanguageModelV3;
messages(modelId: AnthropicMessagesModelId): LanguageModelV3;
/**
* @deprecated Use `embeddingModel` instead.
*/
textEmbeddingModel(modelId: string): never;
/**
* Anthropic-specific computer use tool.
*/
tools: typeof anthropicTools;
}
export interface AnthropicProviderSettings {
/**
* Use a different URL prefix for API calls, e.g. to use proxy servers.
* The default prefix is `https://api.anthropic.com/v1`.
*/
baseURL?: string;
/**
* API key that is being send using the `x-api-key` header.
* It defaults to the `ANTHROPIC_API_KEY` environment variable.
* Only one of `apiKey` or `authToken` is required.
*/
apiKey?: string;
/**
* Auth token that is being sent using the `Authorization: Bearer` header.
* It defaults to the `ANTHROPIC_AUTH_TOKEN` environment variable.
* Only one of `apiKey` or `authToken` is required.
*/
authToken?: string;
/**
* Custom headers to include in the requests.
*/
headers?: Record<string, string>;
/**
* Custom fetch implementation. You can use it as a middleware to intercept requests,
* or to provide a custom fetch implementation for e.g. testing.
*/
fetch?: FetchFunction;
generateId?: () => string;
/**
* Custom provider name
* Defaults to 'anthropic.messages'.
*/
name?: string;
}
/**
* Create an Anthropic provider instance.
*/
export function createAnthropic(
options: AnthropicProviderSettings = {},
): AnthropicProvider {
const baseURL =
withoutTrailingSlash(
loadOptionalSetting({
settingValue: options.baseURL,
environmentVariableName: 'ANTHROPIC_BASE_URL',
}),
) ?? 'https://api.anthropic.com/v1';
const providerName = options.name ?? 'anthropic.messages';
// Only error if both are explicitly provided in options
if (options.apiKey && options.authToken) {
throw new InvalidArgumentError({
argument: 'apiKey/authToken',
message:
'Both apiKey and authToken were provided. Please use only one authentication method.',
});
}
const getHeaders = () => {
const authHeaders: Record<string, string> = options.authToken
? { Authorization: `Bearer ${options.authToken}` }
: {
'x-api-key': loadApiKey({
apiKey: options.apiKey,
environmentVariableName: 'ANTHROPIC_API_KEY',
description: 'Anthropic',
}),
};
return withUserAgentSuffix(
{
'anthropic-version': '2023-06-01',
...authHeaders,
...options.headers,
},
`ai-sdk/anthropic/${VERSION}`,
);
};
const createChatModel = (modelId: AnthropicMessagesModelId) =>
new AnthropicMessagesLanguageModel(modelId, {
provider: providerName,
baseURL,
headers: getHeaders,
fetch: options.fetch,
generateId: options.generateId ?? generateId,
supportedUrls: () => ({
'image/*': [/^https?:\/\/.*$/],
'application/pdf': [/^https?:\/\/.*$/],
}),
});
const provider = function (modelId: AnthropicMessagesModelId) {
if (new.target) {
throw new Error(
'The Anthropic model function cannot be called with the new keyword.',
);
}
return createChatModel(modelId);
};
provider.specificationVersion = 'v3' as const;
provider.languageModel = createChatModel;
provider.chat = createChatModel;
provider.messages = createChatModel;
provider.embeddingModel = (modelId: string) => {
throw new NoSuchModelError({ modelId, modelType: 'embeddingModel' });
};
provider.textEmbeddingModel = provider.embeddingModel;
provider.imageModel = (modelId: string) => {
throw new NoSuchModelError({ modelId, modelType: 'imageModel' });
};
provider.tools = anthropicTools;
return provider;
}
/**
* Default Anthropic provider instance.
*/
export const anthropic = createAnthropic();

238
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/anthropic-tools.ts generated vendored
View file

@ -1,238 +0,0 @@
import { bash_20241022 } from './tool/bash_20241022';
import { bash_20250124 } from './tool/bash_20250124';
import { codeExecution_20250522 } from './tool/code-execution_20250522';
import { codeExecution_20250825 } from './tool/code-execution_20250825';
import { codeExecution_20260120 } from './tool/code-execution_20260120';
import { computer_20241022 } from './tool/computer_20241022';
import { computer_20250124 } from './tool/computer_20250124';
import { computer_20251124 } from './tool/computer_20251124';
import { memory_20250818 } from './tool/memory_20250818';
import { textEditor_20241022 } from './tool/text-editor_20241022';
import { textEditor_20250124 } from './tool/text-editor_20250124';
import { textEditor_20250429 } from './tool/text-editor_20250429';
import { textEditor_20250728 } from './tool/text-editor_20250728';
import { toolSearchBm25_20251119 } from './tool/tool-search-bm25_20251119';
import { toolSearchRegex_20251119 } from './tool/tool-search-regex_20251119';
import { webFetch_20260209 } from './tool/web-fetch-20260209';
import { webFetch_20250910 } from './tool/web-fetch-20250910';
import { webSearch_20260209 } from './tool/web-search_20260209';
import { webSearch_20250305 } from './tool/web-search_20250305';
export const anthropicTools = {
/**
* The bash tool enables Claude to execute shell commands in a persistent bash session,
* allowing system operations, script execution, and command-line automation.
*
* Image results are supported.
*/
bash_20241022,
/**
* The bash tool enables Claude to execute shell commands in a persistent bash session,
* allowing system operations, script execution, and command-line automation.
*
* Image results are supported.
*/
bash_20250124,
/**
* Claude can analyze data, create visualizations, perform complex calculations,
* run system commands, create and edit files, and process uploaded files directly within
* the API conversation.
*
* The code execution tool allows Claude to run Bash commands and manipulate files,
* including writing code, in a secure, sandboxed environment.
*/
codeExecution_20250522,
/**
* Claude can analyze data, create visualizations, perform complex calculations,
* run system commands, create and edit files, and process uploaded files directly within
* the API conversation.
*
* The code execution tool allows Claude to run both Python and Bash commands and manipulate files,
* including writing code, in a secure, sandboxed environment.
*
* This is the latest version with enhanced Bash support and file operations.
*/
codeExecution_20250825,
/**
* Claude can analyze data, create visualizations, perform complex calculations,
* run system commands, create and edit files, and process uploaded files directly within
* the API conversation.
*
* The code execution tool allows Claude to run both Python and Bash commands and manipulate files,
* including writing code, in a secure, sandboxed environment.
*
* This is the recommended version. Does not require a beta header.
*
* Supported models: Claude Opus 4.6, Sonnet 4.6, Sonnet 4.5, Opus 4.5
*/
codeExecution_20260120,
/**
* Claude can interact with computer environments through the computer use tool, which
* provides screenshot capabilities and mouse/keyboard control for autonomous desktop interaction.
*
* Image results are supported.
*
* @param displayWidthPx - The width of the display being controlled by the model in pixels.
* @param displayHeightPx - The height of the display being controlled by the model in pixels.
* @param displayNumber - The display number to control (only relevant for X11 environments). If specified, the tool will be provided a display number in the tool definition.
*/
computer_20241022,
/**
* Claude can interact with computer environments through the computer use tool, which
* provides screenshot capabilities and mouse/keyboard control for autonomous desktop interaction.
*
* Image results are supported.
*
* @param displayWidthPx - The width of the display being controlled by the model in pixels.
* @param displayHeightPx - The height of the display being controlled by the model in pixels.
* @param displayNumber - The display number to control (only relevant for X11 environments). If specified, the tool will be provided a display number in the tool definition.
*/
computer_20250124,
/**
* Claude can interact with computer environments through the computer use tool, which
* provides screenshot capabilities and mouse/keyboard control for autonomous desktop interaction.
*
* This version adds the zoom action for detailed screen region inspection.
*
* Image results are supported.
*
* Supported models: Claude Opus 4.5
*
* @param displayWidthPx - The width of the display being controlled by the model in pixels.
* @param displayHeightPx - The height of the display being controlled by the model in pixels.
* @param displayNumber - The display number to control (only relevant for X11 environments). If specified, the tool will be provided a display number in the tool definition.
* @param enableZoom - Enable zoom action. Set to true to allow Claude to zoom into specific screen regions. Default: false.
*/
computer_20251124,
/**
* The memory tool enables Claude to store and retrieve information across conversations through a memory file directory.
* Claude can create, read, update, and delete files that persist between sessions,
* allowing it to build knowledge over time without keeping everything in the context window.
* The memory tool operates client-side—you control where and how the data is stored through your own infrastructure.
*
* Supported models: Claude Sonnet 4.5, Claude Sonnet 4, Claude Opus 4.1, Claude Opus 4.
*/
memory_20250818,
/**
* Claude can use an Anthropic-defined text editor tool to view and modify text files,
* helping you debug, fix, and improve your code or other text documents. This allows Claude
* to directly interact with your files, providing hands-on assistance rather than just suggesting changes.
*
* Supported models: Claude Sonnet 3.5
*/
textEditor_20241022,
/**
* Claude can use an Anthropic-defined text editor tool to view and modify text files,
* helping you debug, fix, and improve your code or other text documents. This allows Claude
* to directly interact with your files, providing hands-on assistance rather than just suggesting changes.
*
* Supported models: Claude Sonnet 3.7
*/
textEditor_20250124,
/**
* Claude can use an Anthropic-defined text editor tool to view and modify text files,
* helping you debug, fix, and improve your code or other text documents. This allows Claude
* to directly interact with your files, providing hands-on assistance rather than just suggesting changes.
*
* Note: This version does not support the "undo_edit" command.
*
* @deprecated Use textEditor_20250728 instead
*/
textEditor_20250429,
/**
* Claude can use an Anthropic-defined text editor tool to view and modify text files,
* helping you debug, fix, and improve your code or other text documents. This allows Claude
* to directly interact with your files, providing hands-on assistance rather than just suggesting changes.
*
* Note: This version does not support the "undo_edit" command and adds optional max_characters parameter.
*
* Supported models: Claude Sonnet 4, Opus 4, and Opus 4.1
*
* @param maxCharacters - Optional maximum number of characters to view in the file
*/
textEditor_20250728,
/**
* Creates a web fetch tool that gives Claude direct access to real-time web content.
*
* @param maxUses - The max_uses parameter limits the number of web fetches performed
* @param allowedDomains - Only fetch from these domains
* @param blockedDomains - Never fetch from these domains
* @param citations - Unlike web search where citations are always enabled, citations are optional for web fetch. Set "citations": {"enabled": true} to enable Claude to cite specific passages from fetched documents.
* @param maxContentTokens - The max_content_tokens parameter limits the amount of content that will be included in the context.
*/
webFetch_20250910,
/**
* Creates a web fetch tool that gives Claude direct access to real-time web content.
*
* @param maxUses - The max_uses parameter limits the number of web fetches performed
* @param allowedDomains - Only fetch from these domains
* @param blockedDomains - Never fetch from these domains
* @param citations - Unlike web search where citations are always enabled, citations are optional for web fetch. Set "citations": {"enabled": true} to enable Claude to cite specific passages from fetched documents.
* @param maxContentTokens - The max_content_tokens parameter limits the amount of content that will be included in the context.
*/
webFetch_20260209,
/**
* Creates a web search tool that gives Claude direct access to real-time web content.
*
* @param maxUses - Maximum number of web searches Claude can perform during the conversation.
* @param allowedDomains - Optional list of domains that Claude is allowed to search.
* @param blockedDomains - Optional list of domains that Claude should avoid when searching.
* @param userLocation - Optional user location information to provide geographically relevant search results.
*/
webSearch_20250305,
/**
* Creates a web search tool that gives Claude direct access to real-time web content.
*
* @param maxUses - Maximum number of web searches Claude can perform during the conversation.
* @param allowedDomains - Optional list of domains that Claude is allowed to search.
* @param blockedDomains - Optional list of domains that Claude should avoid when searching.
* @param userLocation - Optional user location information to provide geographically relevant search results.
*/
webSearch_20260209,
/**
* Creates a tool search tool that uses regex patterns to find tools.
*
* The tool search tool enables Claude to work with hundreds or thousands of tools
* by dynamically discovering and loading them on-demand. Instead of loading all
* tool definitions into the context window upfront, Claude searches your tool
* catalog and loads only the tools it needs.
*
* Use `providerOptions: { anthropic: { deferLoading: true } }` on other tools
* to mark them for deferred loading.
*
* Supported models: Claude Opus 4.5, Claude Sonnet 4.5
*/
toolSearchRegex_20251119,
/**
* Creates a tool search tool that uses BM25 (natural language) to find tools.
*
* The tool search tool enables Claude to work with hundreds or thousands of tools
* by dynamically discovering and loading them on-demand. Instead of loading all
* tool definitions into the context window upfront, Claude searches your tool
* catalog and loads only the tools it needs.
*
* Use `providerOptions: { anthropic: { deferLoading: true } }` on other tools
* to mark them for deferred loading.
*
* Supported models: Claude Opus 4.5, Claude Sonnet 4.5
*/
toolSearchBm25_20251119,
};

73
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/convert-anthropic-messages-usage.ts generated vendored
View file

@ -1,73 +0,0 @@
import { JSONObject, LanguageModelV3Usage } from '@ai-sdk/provider';
/**
* Represents a single iteration in the usage breakdown.
* When compaction occurs, the API returns an iterations array showing
* usage for each sampling iteration (compaction + message).
*/
export type AnthropicUsageIteration = {
type: 'compaction' | 'message';
input_tokens: number;
output_tokens: number;
};
export type AnthropicMessagesUsage = {
input_tokens: number;
output_tokens: number;
cache_creation_input_tokens?: number | null;
cache_read_input_tokens?: number | null;
/**
* When compaction is triggered, this array contains usage for each
* sampling iteration. The top-level input_tokens and output_tokens
* do NOT include compaction iteration usage - to get total tokens
* consumed and billed, sum across all entries in this array.
*/
iterations?: AnthropicUsageIteration[] | null;
};
export function convertAnthropicMessagesUsage({
usage,
rawUsage,
}: {
usage: AnthropicMessagesUsage;
rawUsage?: JSONObject;
}): LanguageModelV3Usage {
const cacheCreationTokens = usage.cache_creation_input_tokens ?? 0;
const cacheReadTokens = usage.cache_read_input_tokens ?? 0;
// When iterations is present (compaction occurred), sum across all iterations
// to get the true total tokens consumed/billed. The top-level input_tokens
// and output_tokens exclude compaction iteration usage.
let inputTokens: number;
let outputTokens: number;
if (usage.iterations && usage.iterations.length > 0) {
const totals = usage.iterations.reduce(
(acc, iter) => ({
input: acc.input + iter.input_tokens,
output: acc.output + iter.output_tokens,
}),
{ input: 0, output: 0 },
);
inputTokens = totals.input;
outputTokens = totals.output;
} else {
inputTokens = usage.input_tokens;
outputTokens = usage.output_tokens;
}
return {
inputTokens: {
total: inputTokens + cacheCreationTokens + cacheReadTokens,
noCache: inputTokens,
cacheRead: cacheReadTokens,
cacheWrite: cacheCreationTokens,
},
outputTokens: {
total: outputTokens,
text: undefined,
reasoning: undefined,
},
raw: rawUsage ?? usage,
};
}

1119
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/convert-to-anthropic-messages-prompt.ts generated vendored
View file

File diff suppressed because it is too large Load diff

38
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/forward-anthropic-container-id-from-last-step.ts generated vendored
View file

@ -1,38 +0,0 @@
import { JSONObject } from '@ai-sdk/provider';
import { AnthropicMessageMetadata } from './anthropic-message-metadata';
/**
* Sets the Anthropic container ID in the provider options based on
* any previous step's provider metadata.
*
* Searches backwards through steps to find the most recent container ID.
* You can use this function in `prepareStep` to forward the container ID between steps.
*/
export function forwardAnthropicContainerIdFromLastStep({
steps,
}: {
steps: Array<{
providerMetadata?: Record<string, JSONObject>;
}>;
}): undefined | { providerOptions?: Record<string, JSONObject> } {
// Search backwards through steps to find the most recent container ID
for (let i = steps.length - 1; i >= 0; i--) {
const containerId = (
steps[i].providerMetadata?.anthropic as
| AnthropicMessageMetadata
| undefined
)?.container?.id;
if (containerId) {
return {
providerOptions: {
anthropic: {
container: { id: containerId },
},
},
};
}
}
return undefined;
}

63
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/get-cache-control.ts generated vendored
View file

@ -1,63 +0,0 @@
import { SharedV3Warning, SharedV3ProviderMetadata } from '@ai-sdk/provider';
import { AnthropicCacheControl } from './anthropic-messages-api';
// Anthropic allows a maximum of 4 cache breakpoints per request
const MAX_CACHE_BREAKPOINTS = 4;
// Helper function to extract cache_control from provider metadata
// Allows both cacheControl and cache_control for flexibility
function getCacheControl(
providerMetadata: SharedV3ProviderMetadata | undefined,
): AnthropicCacheControl | undefined {
const anthropic = providerMetadata?.anthropic;
// allow both cacheControl and cache_control:
const cacheControlValue = anthropic?.cacheControl ?? anthropic?.cache_control;
// Pass through value assuming it is of the correct type.
// The Anthropic API will validate the value.
return cacheControlValue as AnthropicCacheControl | undefined;
}
export class CacheControlValidator {
private breakpointCount = 0;
private warnings: SharedV3Warning[] = [];
getCacheControl(
providerMetadata: SharedV3ProviderMetadata | undefined,
context: { type: string; canCache: boolean },
): AnthropicCacheControl | undefined {
const cacheControlValue = getCacheControl(providerMetadata);
if (!cacheControlValue) {
return undefined;
}
// Validate that cache_control is allowed in this context
if (!context.canCache) {
this.warnings.push({
type: 'unsupported',
feature: 'cache_control on non-cacheable context',
details: `cache_control cannot be set on ${context.type}. It will be ignored.`,
});
return undefined;
}
// Validate cache breakpoint limit
this.breakpointCount++;
if (this.breakpointCount > MAX_CACHE_BREAKPOINTS) {
this.warnings.push({
type: 'unsupported',
feature: 'cacheControl breakpoint limit',
details: `Maximum ${MAX_CACHE_BREAKPOINTS} cache breakpoints exceeded (found ${this.breakpointCount}). This breakpoint will be ignored.`,
});
return undefined;
}
return cacheControlValue;
}
getWarnings(): SharedV3Warning[] {
return this.warnings;
}
}

17
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/index.ts generated vendored
View file

@ -1,17 +0,0 @@
export type {
AnthropicMessageMetadata,
AnthropicUsageIteration,
} from './anthropic-message-metadata';
export type {
AnthropicLanguageModelOptions,
/** @deprecated Use `AnthropicLanguageModelOptions` instead. */
AnthropicLanguageModelOptions as AnthropicProviderOptions,
} from './anthropic-messages-options';
export type { AnthropicToolOptions } from './anthropic-prepare-tools';
export { anthropic, createAnthropic } from './anthropic-provider';
export type {
AnthropicProvider,
AnthropicProviderSettings,
} from './anthropic-provider';
export { forwardAnthropicContainerIdFromLastStep } from './forward-anthropic-container-id-from-last-step';
export { VERSION } from './version';

4
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/internal/index.ts generated vendored
View file

@ -1,4 +0,0 @@
export { AnthropicMessagesLanguageModel } from '../anthropic-messages-language-model';
export { anthropicTools } from '../anthropic-tools';
export type { AnthropicMessagesModelId } from '../anthropic-messages-options';
export { prepareTools } from '../anthropic-prepare-tools';

30
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/map-anthropic-stop-reason.ts generated vendored
View file

@ -1,30 +0,0 @@
import { LanguageModelV3FinishReason } from '@ai-sdk/provider';
/**
* @see https://docs.anthropic.com/en/api/messages#response-stop-reason
*/
export function mapAnthropicStopReason({
finishReason,
isJsonResponseFromTool,
}: {
finishReason: string | null | undefined;
isJsonResponseFromTool?: boolean;
}): LanguageModelV3FinishReason['unified'] {
switch (finishReason) {
case 'pause_turn':
case 'end_turn':
case 'stop_sequence':
return 'stop';
case 'refusal':
return 'content-filter';
case 'tool_use':
return isJsonResponseFromTool ? 'stop' : 'tool-calls';
case 'max_tokens':
case 'model_context_window_exceeded':
return 'length';
case 'compaction':
return 'other';
default:
return 'other';
}
}

33
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/bash_20241022.ts generated vendored
View file

@ -1,33 +0,0 @@
import {
createProviderToolFactory,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
const bash_20241022InputSchema = lazySchema(() =>
zodSchema(
z.object({
command: z.string(),
restart: z.boolean().optional(),
}),
),
);
export const bash_20241022 = createProviderToolFactory<
{
/**
* The bash command to run. Required unless the tool is being restarted.
*/
command: string;
/**
* Specifying true will restart this tool. Otherwise, leave this unspecified.
*/
restart?: boolean;
},
{}
>({
id: 'anthropic.bash_20241022',
inputSchema: bash_20241022InputSchema,
});

33
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/bash_20250124.ts generated vendored
View file

@ -1,33 +0,0 @@
import {
createProviderToolFactory,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
const bash_20250124InputSchema = lazySchema(() =>
zodSchema(
z.object({
command: z.string(),
restart: z.boolean().optional(),
}),
),
);
export const bash_20250124 = createProviderToolFactory<
{
/**
* The bash command to run. Required unless the tool is being restarted.
*/
command: string;
/**
* Specifying true will restart this tool. Otherwise, leave this unspecified.
*/
restart?: boolean;
},
{}
>({
id: 'anthropic.bash_20250124',
inputSchema: bash_20250124InputSchema,
});

61
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/code-execution_20250522.ts generated vendored
View file

@ -1,61 +0,0 @@
import {
createProviderToolFactoryWithOutputSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
export const codeExecution_20250522OutputSchema = lazySchema(() =>
zodSchema(
z.object({
type: z.literal('code_execution_result'),
stdout: z.string(),
stderr: z.string(),
return_code: z.number(),
content: z
.array(
z.object({
type: z.literal('code_execution_output'),
file_id: z.string(),
}),
)
.optional()
.default([]),
}),
),
);
const codeExecution_20250522InputSchema = lazySchema(() =>
zodSchema(
z.object({
code: z.string(),
}),
),
);
const factory = createProviderToolFactoryWithOutputSchema<
{
/**
* The Python code to execute.
*/
code: string;
},
{
type: 'code_execution_result';
stdout: string;
stderr: string;
return_code: number;
content: Array<{ type: 'code_execution_output'; file_id: string }>;
},
{}
>({
id: 'anthropic.code_execution_20250522',
inputSchema: codeExecution_20250522InputSchema,
outputSchema: codeExecution_20250522OutputSchema,
});
export const codeExecution_20250522 = (
args: Parameters<typeof factory>[0] = {},
) => {
return factory(args);
};

281
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/code-execution_20250825.ts generated vendored
View file

@ -1,281 +0,0 @@
import {
createProviderToolFactoryWithOutputSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
export const codeExecution_20250825OutputSchema = lazySchema(() =>
zodSchema(
z.discriminatedUnion('type', [
z.object({
type: z.literal('code_execution_result'),
stdout: z.string(),
stderr: z.string(),
return_code: z.number(),
content: z
.array(
z.object({
type: z.literal('code_execution_output'),
file_id: z.string(),
}),
)
.optional()
.default([]),
}),
z.object({
type: z.literal('bash_code_execution_result'),
content: z.array(
z.object({
type: z.literal('bash_code_execution_output'),
file_id: z.string(),
}),
),
stdout: z.string(),
stderr: z.string(),
return_code: z.number(),
}),
z.object({
type: z.literal('bash_code_execution_tool_result_error'),
error_code: z.string(),
}),
z.object({
type: z.literal('text_editor_code_execution_tool_result_error'),
error_code: z.string(),
}),
z.object({
type: z.literal('text_editor_code_execution_view_result'),
content: z.string(),
file_type: z.string(),
num_lines: z.number().nullable(),
start_line: z.number().nullable(),
total_lines: z.number().nullable(),
}),
z.object({
type: z.literal('text_editor_code_execution_create_result'),
is_file_update: z.boolean(),
}),
z.object({
type: z.literal('text_editor_code_execution_str_replace_result'),
lines: z.array(z.string()).nullable(),
new_lines: z.number().nullable(),
new_start: z.number().nullable(),
old_lines: z.number().nullable(),
old_start: z.number().nullable(),
}),
]),
),
);
export const codeExecution_20250825InputSchema = lazySchema(() =>
zodSchema(
z.discriminatedUnion('type', [
// Programmatic tool calling format (mapped from { code } by AI SDK)
z.object({
type: z.literal('programmatic-tool-call'),
code: z.string(),
}),
z.object({
type: z.literal('bash_code_execution'),
command: z.string(),
}),
z.discriminatedUnion('command', [
z.object({
type: z.literal('text_editor_code_execution'),
command: z.literal('view'),
path: z.string(),
}),
z.object({
type: z.literal('text_editor_code_execution'),
command: z.literal('create'),
path: z.string(),
file_text: z.string().nullish(),
}),
z.object({
type: z.literal('text_editor_code_execution'),
command: z.literal('str_replace'),
path: z.string(),
old_str: z.string(),
new_str: z.string(),
}),
]),
]),
),
);
const factory = createProviderToolFactoryWithOutputSchema<
| {
type: 'programmatic-tool-call';
/**
* Programmatic tool calling: Python code to execute when code_execution
* is used with allowedCallers to trigger client-executed tools.
*/
code: string;
}
| {
type: 'bash_code_execution';
/**
* Shell command to execute.
*/
command: string;
}
| {
type: 'text_editor_code_execution';
command: 'view';
/**
* The path to the file to view.
*/
path: string;
}
| {
type: 'text_editor_code_execution';
command: 'create';
/**
* The path to the file to edit.
*/
path: string;
/**
* The text of the file to edit.
*/
file_text?: string | null;
}
| {
type: 'text_editor_code_execution';
command: 'str_replace';
/**
* The path to the file to edit.
*/
path: string;
/**
* The string to replace.
*/
old_str: string;
/**
* The new string to replace the old string with.
*/
new_str: string;
},
| {
/**
* Programmatic tool calling result: returned when code_execution runs code
* that calls client-executed tools via allowedCallers.
*/
type: 'code_execution_result';
/**
* Output from successful execution
*/
stdout: string;
/**
* Error messages if execution fails
*/
stderr: string;
/**
* 0 for success, non-zero for failure
*/
return_code: number;
/**
* Output file Id list
*/
content: Array<{ type: 'code_execution_output'; file_id: string }>;
}
| {
type: 'bash_code_execution_result';
/**
* Output file Id list
*/
content: Array<{
type: 'bash_code_execution_output';
file_id: string;
}>;
/**
* Output from successful execution
*/
stdout: string;
/**
* Error messages if execution fails
*/
stderr: string;
/**
* 0 for success, non-zero for failure
*/
return_code: number;
}
| {
type: 'bash_code_execution_tool_result_error';
/**
* Available options: invalid_tool_input, unavailable, too_many_requests,
* execution_time_exceeded, output_file_too_large.
*/
error_code: string;
}
| {
type: 'text_editor_code_execution_tool_result_error';
/**
* Available options: invalid_tool_input, unavailable, too_many_requests,
* execution_time_exceeded, file_not_found.
*/
error_code: string;
}
| {
type: 'text_editor_code_execution_view_result';
content: string;
/**
* The type of the file. Available options: text, image, pdf.
*/
file_type: string;
num_lines: number | null;
start_line: number | null;
total_lines: number | null;
}
| {
type: 'text_editor_code_execution_create_result';
is_file_update: boolean;
}
| {
type: 'text_editor_code_execution_str_replace_result';
lines: string[] | null;
new_lines: number | null;
new_start: number | null;
old_lines: number | null;
old_start: number | null;
},
{
// no arguments
}
>({
id: 'anthropic.code_execution_20250825',
inputSchema: codeExecution_20250825InputSchema,
outputSchema: codeExecution_20250825OutputSchema,
// Programmatic tool calling: tool results may be deferred to a later turn
// when code execution triggers a client-executed tool that needs to be
// resolved before the code execution result can be returned.
supportsDeferredResults: true,
});
export const codeExecution_20250825 = (
args: Parameters<typeof factory>[0] = {},
) => {
return factory(args);
};

315
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/code-execution_20260120.ts generated vendored
View file

@ -1,315 +0,0 @@
import {
createProviderToolFactoryWithOutputSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
export const codeExecution_20260120OutputSchema = lazySchema(() =>
zodSchema(
z.discriminatedUnion('type', [
z.object({
type: z.literal('code_execution_result'),
stdout: z.string(),
stderr: z.string(),
return_code: z.number(),
content: z
.array(
z.object({
type: z.literal('code_execution_output'),
file_id: z.string(),
}),
)
.optional()
.default([]),
}),
z.object({
type: z.literal('encrypted_code_execution_result'),
encrypted_stdout: z.string(),
stderr: z.string(),
return_code: z.number(),
content: z
.array(
z.object({
type: z.literal('code_execution_output'),
file_id: z.string(),
}),
)
.optional()
.default([]),
}),
z.object({
type: z.literal('bash_code_execution_result'),
content: z.array(
z.object({
type: z.literal('bash_code_execution_output'),
file_id: z.string(),
}),
),
stdout: z.string(),
stderr: z.string(),
return_code: z.number(),
}),
z.object({
type: z.literal('bash_code_execution_tool_result_error'),
error_code: z.string(),
}),
z.object({
type: z.literal('text_editor_code_execution_tool_result_error'),
error_code: z.string(),
}),
z.object({
type: z.literal('text_editor_code_execution_view_result'),
content: z.string(),
file_type: z.string(),
num_lines: z.number().nullable(),
start_line: z.number().nullable(),
total_lines: z.number().nullable(),
}),
z.object({
type: z.literal('text_editor_code_execution_create_result'),
is_file_update: z.boolean(),
}),
z.object({
type: z.literal('text_editor_code_execution_str_replace_result'),
lines: z.array(z.string()).nullable(),
new_lines: z.number().nullable(),
new_start: z.number().nullable(),
old_lines: z.number().nullable(),
old_start: z.number().nullable(),
}),
]),
),
);
export const codeExecution_20260120InputSchema = lazySchema(() =>
zodSchema(
z.discriminatedUnion('type', [
z.object({
type: z.literal('programmatic-tool-call'),
code: z.string(),
}),
z.object({
type: z.literal('bash_code_execution'),
command: z.string(),
}),
z.discriminatedUnion('command', [
z.object({
type: z.literal('text_editor_code_execution'),
command: z.literal('view'),
path: z.string(),
}),
z.object({
type: z.literal('text_editor_code_execution'),
command: z.literal('create'),
path: z.string(),
file_text: z.string().nullish(),
}),
z.object({
type: z.literal('text_editor_code_execution'),
command: z.literal('str_replace'),
path: z.string(),
old_str: z.string(),
new_str: z.string(),
}),
]),
]),
),
);
const factory = createProviderToolFactoryWithOutputSchema<
| {
type: 'programmatic-tool-call';
/**
* Programmatic tool calling: Python code to execute when code_execution
* is used with allowedCallers to trigger client-executed tools.
*/
code: string;
}
| {
type: 'bash_code_execution';
/**
* Shell command to execute.
*/
command: string;
}
| {
type: 'text_editor_code_execution';
command: 'view';
/**
* The path to the file to view.
*/
path: string;
}
| {
type: 'text_editor_code_execution';
command: 'create';
/**
* The path to the file to edit.
*/
path: string;
/**
* The text of the file to edit.
*/
file_text?: string | null;
}
| {
type: 'text_editor_code_execution';
command: 'str_replace';
/**
* The path to the file to edit.
*/
path: string;
/**
* The string to replace.
*/
old_str: string;
/**
* The new string to replace the old string with.
*/
new_str: string;
},
| {
/**
* Programmatic tool calling result: returned when code_execution runs code
* that calls client-executed tools via allowedCallers.
*/
type: 'code_execution_result';
/**
* Output from successful execution
*/
stdout: string;
/**
* Error messages if execution fails
*/
stderr: string;
/**
* 0 for success, non-zero for failure
*/
return_code: number;
/**
* Output file Id list
*/
content: Array<{ type: 'code_execution_output'; file_id: string }>;
}
| {
type: 'encrypted_code_execution_result';
/**
* Encrypted output from successful execution
*/
encrypted_stdout: string;
/**
* Error messages if execution fails
*/
stderr: string;
/**
* 0 for success, non-zero for failure
*/
return_code: number;
/**
* Output file Id list
*/
content: Array<{ type: 'code_execution_output'; file_id: string }>;
}
| {
type: 'bash_code_execution_result';
/**
* Output file Id list
*/
content: Array<{
type: 'bash_code_execution_output';
file_id: string;
}>;
/**
* Output from successful execution
*/
stdout: string;
/**
* Error messages if execution fails
*/
stderr: string;
/**
* 0 for success, non-zero for failure
*/
return_code: number;
}
| {
type: 'bash_code_execution_tool_result_error';
/**
* Available options: invalid_tool_input, unavailable, too_many_requests,
* execution_time_exceeded, output_file_too_large.
*/
error_code: string;
}
| {
type: 'text_editor_code_execution_tool_result_error';
/**
* Available options: invalid_tool_input, unavailable, too_many_requests,
* execution_time_exceeded, file_not_found.
*/
error_code: string;
}
| {
type: 'text_editor_code_execution_view_result';
content: string;
/**
* The type of the file. Available options: text, image, pdf.
*/
file_type: string;
num_lines: number | null;
start_line: number | null;
total_lines: number | null;
}
| {
type: 'text_editor_code_execution_create_result';
is_file_update: boolean;
}
| {
type: 'text_editor_code_execution_str_replace_result';
lines: string[] | null;
new_lines: number | null;
new_start: number | null;
old_lines: number | null;
old_start: number | null;
},
{
// no arguments
}
>({
id: 'anthropic.code_execution_20260120',
inputSchema: codeExecution_20260120InputSchema,
outputSchema: codeExecution_20260120OutputSchema,
supportsDeferredResults: true,
});
export const codeExecution_20260120 = (
args: Parameters<typeof factory>[0] = {},
) => {
return factory(args);
};

87
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/computer_20241022.ts generated vendored
View file

@ -1,87 +0,0 @@
import {
createProviderToolFactory,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
const computer_20241022InputSchema = lazySchema(() =>
zodSchema(
z.object({
action: z.enum([
'key',
'type',
'mouse_move',
'left_click',
'left_click_drag',
'right_click',
'middle_click',
'double_click',
'screenshot',
'cursor_position',
]),
coordinate: z.array(z.number().int()).optional(),
text: z.string().optional(),
}),
),
);
export const computer_20241022 = createProviderToolFactory<
{
/**
* The action to perform. The available actions are:
* - `key`: Press a key or key-combination on the keyboard.
* - This supports xdotool's `key` syntax.
* - Examples: "a", "Return", "alt+Tab", "ctrl+s", "Up", "KP_0" (for the numpad 0 key).
* - `type`: Type a string of text on the keyboard.
* - `cursor_position`: Get the current (x, y) pixel coordinate of the cursor on the screen.
* - `mouse_move`: Move the cursor to a specified (x, y) pixel coordinate on the screen.
* - `left_click`: Click the left mouse button.
* - `left_click_drag`: Click and drag the cursor to a specified (x, y) pixel coordinate on the screen.
* - `right_click`: Click the right mouse button.
* - `middle_click`: Click the middle mouse button.
* - `double_click`: Double-click the left mouse button.
* - `screenshot`: Take a screenshot of the screen.
*/
action:
| 'key'
| 'type'
| 'mouse_move'
| 'left_click'
| 'left_click_drag'
| 'right_click'
| 'middle_click'
| 'double_click'
| 'screenshot'
| 'cursor_position';
/**
* (x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to move the mouse to. Required only by `action=mouse_move` and `action=left_click_drag`.
*/
coordinate?: number[];
/**
* Required only by `action=type` and `action=key`.
*/
text?: string;
},
{
/**
* The width of the display being controlled by the model in pixels.
*/
displayWidthPx: number;
/**
* The height of the display being controlled by the model in pixels.
*/
displayHeightPx: number;
/**
* The display number to control (only relevant for X11 environments). If specified, the tool will be provided a display number in the tool definition.
*/
displayNumber?: number;
}
>({
id: 'anthropic.computer_20241022',
inputSchema: computer_20241022InputSchema,
});

130
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/computer_20250124.ts generated vendored
View file

@ -1,130 +0,0 @@
import {
createProviderToolFactory,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
const computer_20250124InputSchema = lazySchema(() =>
zodSchema(
z.object({
action: z.enum([
'key',
'hold_key',
'type',
'cursor_position',
'mouse_move',
'left_mouse_down',
'left_mouse_up',
'left_click',
'left_click_drag',
'right_click',
'middle_click',
'double_click',
'triple_click',
'scroll',
'wait',
'screenshot',
]),
coordinate: z.tuple([z.number().int(), z.number().int()]).optional(),
duration: z.number().optional(),
scroll_amount: z.number().optional(),
scroll_direction: z.enum(['up', 'down', 'left', 'right']).optional(),
start_coordinate: z
.tuple([z.number().int(), z.number().int()])
.optional(),
text: z.string().optional(),
}),
),
);
export const computer_20250124 = createProviderToolFactory<
{
/**
* - `key`: Press a key or key-combination on the keyboard.
* - This supports xdotool's `key` syntax.
* - Examples: "a", "Return", "alt+Tab", "ctrl+s", "Up", "KP_0" (for the numpad 0 key).
* - `hold_key`: Hold down a key or multiple keys for a specified duration (in seconds). Supports the same syntax as `key`.
* - `type`: Type a string of text on the keyboard.
* - `cursor_position`: Get the current (x, y) pixel coordinate of the cursor on the screen.
* - `mouse_move`: Move the cursor to a specified (x, y) pixel coordinate on the screen.
* - `left_mouse_down`: Press the left mouse button.
* - `left_mouse_up`: Release the left mouse button.
* - `left_click`: Click the left mouse button at the specified (x, y) pixel coordinate on the screen. You can also include a key combination to hold down while clicking using the `text` parameter.
* - `left_click_drag`: Click and drag the cursor from `start_coordinate` to a specified (x, y) pixel coordinate on the screen.
* - `right_click`: Click the right mouse button at the specified (x, y) pixel coordinate on the screen.
* - `middle_click`: Click the middle mouse button at the specified (x, y) pixel coordinate on the screen.
* - `double_click`: Double-click the left mouse button at the specified (x, y) pixel coordinate on the screen.
* - `triple_click`: Triple-click the left mouse button at the specified (x, y) pixel coordinate on the screen.
* - `scroll`: Scroll the screen in a specified direction by a specified amount of clicks of the scroll wheel, at the specified (x, y) pixel coordinate. DO NOT use PageUp/PageDown to scroll.
* - `wait`: Wait for a specified duration (in seconds).
* - `screenshot`: Take a screenshot of the screen.
*/
action:
| 'key'
| 'hold_key'
| 'type'
| 'cursor_position'
| 'mouse_move'
| 'left_mouse_down'
| 'left_mouse_up'
| 'left_click'
| 'left_click_drag'
| 'right_click'
| 'middle_click'
| 'double_click'
| 'triple_click'
| 'scroll'
| 'wait'
| 'screenshot';
/**
* (x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to move the mouse to. Required only by `action=mouse_move` and `action=left_click_drag`.
*/
coordinate?: [number, number];
/**
* The duration to hold the key down for. Required only by `action=hold_key` and `action=wait`.
*/
duration?: number;
/**
* The number of 'clicks' to scroll. Required only by `action=scroll`.
*/
scroll_amount?: number;
/**
* The direction to scroll the screen. Required only by `action=scroll`.
*/
scroll_direction?: 'up' | 'down' | 'left' | 'right';
/**
* (x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to start the drag from. Required only by `action=left_click_drag`.
*/
start_coordinate?: [number, number];
/**
* Required only by `action=type`, `action=key`, and `action=hold_key`. Can also be used by click or scroll actions to hold down keys while clicking or scrolling.
*/
text?: string;
},
{
/**
* The width of the display being controlled by the model in pixels.
*/
displayWidthPx: number;
/**
* The height of the display being controlled by the model in pixels.
*/
displayHeightPx: number;
/**
* The display number to control (only relevant for X11 environments). If specified, the tool will be provided a display number in the tool definition.
*/
displayNumber?: number;
}
>({
id: 'anthropic.computer_20250124',
inputSchema: computer_20250124InputSchema,
});

151
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/computer_20251124.ts generated vendored
View file

@ -1,151 +0,0 @@
import {
createProviderToolFactory,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
const computer_20251124InputSchema = lazySchema(() =>
zodSchema(
z.object({
action: z.enum([
'key',
'hold_key',
'type',
'cursor_position',
'mouse_move',
'left_mouse_down',
'left_mouse_up',
'left_click',
'left_click_drag',
'right_click',
'middle_click',
'double_click',
'triple_click',
'scroll',
'wait',
'screenshot',
'zoom',
]),
coordinate: z.tuple([z.number().int(), z.number().int()]).optional(),
duration: z.number().optional(),
region: z
.tuple([
z.number().int(),
z.number().int(),
z.number().int(),
z.number().int(),
])
.optional(),
scroll_amount: z.number().optional(),
scroll_direction: z.enum(['up', 'down', 'left', 'right']).optional(),
start_coordinate: z
.tuple([z.number().int(), z.number().int()])
.optional(),
text: z.string().optional(),
}),
),
);
export const computer_20251124 = createProviderToolFactory<
{
/**
* - `key`: Press a key or key-combination on the keyboard.
* - This supports xdotool's `key` syntax.
* - Examples: "a", "Return", "alt+Tab", "ctrl+s", "Up", "KP_0" (for the numpad 0 key).
* - `hold_key`: Hold down a key or multiple keys for a specified duration (in seconds). Supports the same syntax as `key`.
* - `type`: Type a string of text on the keyboard.
* - `cursor_position`: Get the current (x, y) pixel coordinate of the cursor on the screen.
* - `mouse_move`: Move the cursor to a specified (x, y) pixel coordinate on the screen.
* - `left_mouse_down`: Press the left mouse button.
* - `left_mouse_up`: Release the left mouse button.
* - `left_click`: Click the left mouse button at the specified (x, y) pixel coordinate on the screen. You can also include a key combination to hold down while clicking using the `text` parameter.
* - `left_click_drag`: Click and drag the cursor from `start_coordinate` to a specified (x, y) pixel coordinate on the screen.
* - `right_click`: Click the right mouse button at the specified (x, y) pixel coordinate on the screen.
* - `middle_click`: Click the middle mouse button at the specified (x, y) pixel coordinate on the screen.
* - `double_click`: Double-click the left mouse button at the specified (x, y) pixel coordinate on the screen.
* - `triple_click`: Triple-click the left mouse button at the specified (x, y) pixel coordinate on the screen.
* - `scroll`: Scroll the screen in a specified direction by a specified amount of clicks of the scroll wheel, at the specified (x, y) pixel coordinate. DO NOT use PageUp/PageDown to scroll.
* - `wait`: Wait for a specified duration (in seconds).
* - `screenshot`: Take a screenshot of the screen.
* - `zoom`: View a specific region of the screen at full resolution. Requires `enableZoom: true` in tool definition. Takes a `region` parameter with coordinates `[x1, y1, x2, y2]` defining top-left and bottom-right corners of the area to inspect.
*/
action:
| 'key'
| 'hold_key'
| 'type'
| 'cursor_position'
| 'mouse_move'
| 'left_mouse_down'
| 'left_mouse_up'
| 'left_click'
| 'left_click_drag'
| 'right_click'
| 'middle_click'
| 'double_click'
| 'triple_click'
| 'scroll'
| 'wait'
| 'screenshot'
| 'zoom';
/**
* (x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to move the mouse to. Required only by `action=mouse_move` and `action=left_click_drag`.
*/
coordinate?: [number, number];
/**
* The duration to hold the key down for. Required only by `action=hold_key` and `action=wait`.
*/
duration?: number;
/**
* [x1, y1, x2, y2]: The coordinates defining the region to zoom into. x1, y1 is the top-left corner and x2, y2 is the bottom-right corner. Required only by `action=zoom`.
*/
region?: [number, number, number, number];
/**
* The number of 'clicks' to scroll. Required only by `action=scroll`.
*/
scroll_amount?: number;
/**
* The direction to scroll the screen. Required only by `action=scroll`.
*/
scroll_direction?: 'up' | 'down' | 'left' | 'right';
/**
* (x, y): The x (pixels from the left edge) and y (pixels from the top edge) coordinates to start the drag from. Required only by `action=left_click_drag`.
*/
start_coordinate?: [number, number];
/**
* Required only by `action=type`, `action=key`, and `action=hold_key`. Can also be used by click or scroll actions to hold down keys while clicking or scrolling.
*/
text?: string;
},
{
/**
* The width of the display being controlled by the model in pixels.
*/
displayWidthPx: number;
/**
* The height of the display being controlled by the model in pixels.
*/
displayHeightPx: number;
/**
* The display number to control (only relevant for X11 environments). If specified, the tool will be provided a display number in the tool definition.
*/
displayNumber?: number;
/**
* Enable zoom action. Set to true to allow Claude to zoom into specific screen regions. Default: false.
*/
enableZoom?: boolean;
}
>({
id: 'anthropic.computer_20251124',
inputSchema: computer_20251124InputSchema,
});

62
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/memory_20250818.ts generated vendored
View file

@ -1,62 +0,0 @@
import {
createProviderToolFactory,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
const memory_20250818InputSchema = lazySchema(() =>
zodSchema(
z.discriminatedUnion('command', [
z.object({
command: z.literal('view'),
path: z.string(),
view_range: z.tuple([z.number(), z.number()]).optional(),
}),
z.object({
command: z.literal('create'),
path: z.string(),
file_text: z.string(),
}),
z.object({
command: z.literal('str_replace'),
path: z.string(),
old_str: z.string(),
new_str: z.string(),
}),
z.object({
command: z.literal('insert'),
path: z.string(),
insert_line: z.number(),
insert_text: z.string(),
}),
z.object({
command: z.literal('delete'),
path: z.string(),
}),
z.object({
command: z.literal('rename'),
old_path: z.string(),
new_path: z.string(),
}),
]),
),
);
export const memory_20250818 = createProviderToolFactory<
| { command: 'view'; path: string; view_range?: [number, number] }
| { command: 'create'; path: string; file_text: string }
| { command: 'str_replace'; path: string; old_str: string; new_str: string }
| {
command: 'insert';
path: string;
insert_line: number;
insert_text: string;
}
| { command: 'delete'; path: string }
| { command: 'rename'; old_path: string; new_path: string },
{}
>({
id: 'anthropic.memory_20250818',
inputSchema: memory_20250818InputSchema,
});

69
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/text-editor_20241022.ts generated vendored
View file

@ -1,69 +0,0 @@
import {
createProviderToolFactory,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
const textEditor_20241022InputSchema = lazySchema(() =>
zodSchema(
z.object({
command: z.enum(['view', 'create', 'str_replace', 'insert', 'undo_edit']),
path: z.string(),
file_text: z.string().optional(),
insert_line: z.number().int().optional(),
new_str: z.string().optional(),
insert_text: z.string().optional(),
old_str: z.string().optional(),
view_range: z.array(z.number().int()).optional(),
}),
),
);
export const textEditor_20241022 = createProviderToolFactory<
{
/**
* The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`, `undo_edit`.
*/
command: 'view' | 'create' | 'str_replace' | 'insert' | 'undo_edit';
/**
* Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`.
*/
path: string;
/**
* Required parameter of `create` command, with the content of the file to be created.
*/
file_text?: string;
/**
* Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`.
*/
insert_line?: number;
/**
* Optional parameter of `str_replace` command containing the new string (if not given, no string will be added).
*/
new_str?: string;
/**
* Required parameter of `insert` command containing the text to insert.
*/
insert_text?: string;
/**
* Required parameter of `str_replace` command containing the string in `path` to replace.
*/
old_str?: string;
/**
* Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.
*/
view_range?: number[];
},
{}
>({
id: 'anthropic.text_editor_20241022',
inputSchema: textEditor_20241022InputSchema,
});

69
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/text-editor_20250124.ts generated vendored
View file

@ -1,69 +0,0 @@
import {
createProviderToolFactory,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
const textEditor_20250124InputSchema = lazySchema(() =>
zodSchema(
z.object({
command: z.enum(['view', 'create', 'str_replace', 'insert', 'undo_edit']),
path: z.string(),
file_text: z.string().optional(),
insert_line: z.number().int().optional(),
new_str: z.string().optional(),
insert_text: z.string().optional(),
old_str: z.string().optional(),
view_range: z.array(z.number().int()).optional(),
}),
),
);
export const textEditor_20250124 = createProviderToolFactory<
{
/**
* The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`, `undo_edit`.
*/
command: 'view' | 'create' | 'str_replace' | 'insert' | 'undo_edit';
/**
* Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`.
*/
path: string;
/**
* Required parameter of `create` command, with the content of the file to be created.
*/
file_text?: string;
/**
* Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`.
*/
insert_line?: number;
/**
* Optional parameter of `str_replace` command containing the new string (if not given, no string will be added).
*/
new_str?: string;
/**
* Required parameter of `insert` command containing the text to insert.
*/
insert_text?: string;
/**
* Required parameter of `str_replace` command containing the string in `path` to replace.
*/
old_str?: string;
/**
* Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.
*/
view_range?: number[];
},
{}
>({
id: 'anthropic.text_editor_20250124',
inputSchema: textEditor_20250124InputSchema,
});

70
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/text-editor_20250429.ts generated vendored
View file

@ -1,70 +0,0 @@
import {
createProviderToolFactory,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
const textEditor_20250429InputSchema = lazySchema(() =>
zodSchema(
z.object({
command: z.enum(['view', 'create', 'str_replace', 'insert']),
path: z.string(),
file_text: z.string().optional(),
insert_line: z.number().int().optional(),
new_str: z.string().optional(),
insert_text: z.string().optional(),
old_str: z.string().optional(),
view_range: z.array(z.number().int()).optional(),
}),
),
);
export const textEditor_20250429 = createProviderToolFactory<
{
/**
* The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.
* Note: `undo_edit` is not supported in Claude 4 models.
*/
command: 'view' | 'create' | 'str_replace' | 'insert';
/**
* Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`.
*/
path: string;
/**
* Required parameter of `create` command, with the content of the file to be created.
*/
file_text?: string;
/**
* Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`.
*/
insert_line?: number;
/**
* Optional parameter of `str_replace` command containing the new string (if not given, no string will be added).
*/
new_str?: string;
/**
* Required parameter of `insert` command containing the text to insert.
*/
insert_text?: string;
/**
* Required parameter of `str_replace` command containing the string in `path` to replace.
*/
old_str?: string;
/**
* Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.
*/
view_range?: number[];
},
{}
>({
id: 'anthropic.text_editor_20250429',
inputSchema: textEditor_20250429InputSchema,
});

86
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/text-editor_20250728.ts generated vendored
View file

@ -1,86 +0,0 @@
import { createProviderToolFactory } from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
import { lazySchema, zodSchema } from '@ai-sdk/provider-utils';
export const textEditor_20250728ArgsSchema = lazySchema(() =>
zodSchema(
z.object({
maxCharacters: z.number().optional(),
}),
),
);
const textEditor_20250728InputSchema = lazySchema(() =>
zodSchema(
z.object({
command: z.enum(['view', 'create', 'str_replace', 'insert']),
path: z.string(),
file_text: z.string().optional(),
insert_line: z.number().int().optional(),
new_str: z.string().optional(),
insert_text: z.string().optional(),
old_str: z.string().optional(),
view_range: z.array(z.number().int()).optional(),
}),
),
);
const factory = createProviderToolFactory<
{
/**
* The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.
* Note: `undo_edit` is not supported in Claude 4 models.
*/
command: 'view' | 'create' | 'str_replace' | 'insert';
/**
* Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`.
*/
path: string;
/**
* Required parameter of `create` command, with the content of the file to be created.
*/
file_text?: string;
/**
* Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`.
*/
insert_line?: number;
/**
* Optional parameter of `str_replace` command containing the new string (if not given, no string will be added).
*/
new_str?: string;
/**
* Required parameter of `insert` command containing the text to insert.
*/
insert_text?: string;
/**
* Required parameter of `str_replace` command containing the string in `path` to replace.
*/
old_str?: string;
/**
* Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.
*/
view_range?: number[];
},
{
/**
* Optional parameter to control truncation when viewing large files. Only compatible with text_editor_20250728 and later versions.
*/
maxCharacters?: number;
}
>({
id: 'anthropic.text_editor_20250728',
inputSchema: textEditor_20250728InputSchema,
});
export const textEditor_20250728 = (
args: Parameters<typeof factory>[0] = {}, // default
) => {
return factory(args);
};

99
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/tool-search-bm25_20251119.ts generated vendored
View file

@ -1,99 +0,0 @@
import {
createProviderToolFactoryWithOutputSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
/**
* Output schema for tool search results - returns tool references
* that are automatically expanded into full tool definitions by the API.
*/
export const toolSearchBm25_20251119OutputSchema = lazySchema(() =>
zodSchema(
z.array(
z.object({
type: z.literal('tool_reference'),
toolName: z.string(),
}),
),
),
);
/**
* Input schema for BM25-based tool search.
* Claude uses natural language queries to search for tools.
*/
const toolSearchBm25_20251119InputSchema = lazySchema(() =>
zodSchema(
z.object({
/**
* A natural language query to search for tools.
* Claude will use BM25 text search to find relevant tools.
*/
query: z.string(),
/**
* Maximum number of tools to return. Optional.
*/
limit: z.number().optional(),
}),
),
);
const factory = createProviderToolFactoryWithOutputSchema<
{
/**
* A natural language query to search for tools.
* Claude will use BM25 text search to find relevant tools.
*/
query: string;
/**
* Maximum number of tools to return. Optional.
*/
limit?: number;
},
Array<{
type: 'tool_reference';
/**
* The name of the discovered tool.
*/
toolName: string;
}>,
{}
>({
id: 'anthropic.tool_search_bm25_20251119',
inputSchema: toolSearchBm25_20251119InputSchema,
outputSchema: toolSearchBm25_20251119OutputSchema,
supportsDeferredResults: true,
});
/**
* Creates a tool search tool that uses BM25 (natural language) to find tools.
*
* The tool search tool enables Claude to work with hundreds or thousands of tools
* by dynamically discovering and loading them on-demand. Instead of loading all
* tool definitions into the context window upfront, Claude searches your tool
* catalog and loads only the tools it needs.
*
* When Claude uses this tool, it uses natural language queries (NOT regex patterns)
* to search for tools using BM25 text search.
*
* **Important**: This tool should never have `deferLoading: true` in providerOptions.
*
* @example
* ```ts
* import { anthropicTools } from '@ai-sdk/anthropic';
*
* const tools = {
* toolSearch: anthropicTools.toolSearchBm25_20251119(),
* // Other tools with deferLoading...
* };
* ```
*
* @see https://docs.anthropic.com/en/docs/agents-and-tools/tool-search-tool
*/
export const toolSearchBm25_20251119 = (
args: Parameters<typeof factory>[0] = {},
) => {
return factory(args);
};

111
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/tool-search-regex_20251119.ts generated vendored
View file

@ -1,111 +0,0 @@
import {
createProviderToolFactoryWithOutputSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
/**
* Output schema for tool search results - returns tool references
* that are automatically expanded into full tool definitions by the API.
*/
export const toolSearchRegex_20251119OutputSchema = lazySchema(() =>
zodSchema(
z.array(
z.object({
type: z.literal('tool_reference'),
toolName: z.string(),
}),
),
),
);
/**
* Input schema for regex-based tool search.
* Claude constructs regex patterns using Python's re.search() syntax.
*/
const toolSearchRegex_20251119InputSchema = lazySchema(() =>
zodSchema(
z.object({
/**
* A regex pattern to search for tools.
* Uses Python re.search() syntax. Maximum 200 characters.
*
* Examples:
* - "weather" - matches tool names/descriptions containing "weather"
* - "get_.*_data" - matches tools like get_user_data, get_weather_data
* - "database.*query|query.*database" - OR patterns for flexibility
* - "(?i)slack" - case-insensitive search
*/
pattern: z.string(),
/**
* Maximum number of tools to return. Optional.
*/
limit: z.number().optional(),
}),
),
);
const factory = createProviderToolFactoryWithOutputSchema<
{
/**
* A regex pattern to search for tools.
* Uses Python re.search() syntax. Maximum 200 characters.
*
* Examples:
* - "weather" - matches tool names/descriptions containing "weather"
* - "get_.*_data" - matches tools like get_user_data, get_weather_data
* - "database.*query|query.*database" - OR patterns for flexibility
* - "(?i)slack" - case-insensitive search
*/
pattern: string;
/**
* Maximum number of tools to return. Optional.
*/
limit?: number;
},
Array<{
type: 'tool_reference';
/**
* The name of the discovered tool.
*/
toolName: string;
}>,
{}
>({
id: 'anthropic.tool_search_regex_20251119',
inputSchema: toolSearchRegex_20251119InputSchema,
outputSchema: toolSearchRegex_20251119OutputSchema,
supportsDeferredResults: true,
});
/**
* Creates a tool search tool that uses regex patterns to find tools.
*
* The tool search tool enables Claude to work with hundreds or thousands of tools
* by dynamically discovering and loading them on-demand. Instead of loading all
* tool definitions into the context window upfront, Claude searches your tool
* catalog and loads only the tools it needs.
*
* When Claude uses this tool, it constructs regex patterns using Python's
* re.search() syntax (NOT natural language queries).
*
* **Important**: This tool should never have `deferLoading: true` in providerOptions.
*
* @example
* ```ts
* import { anthropicTools } from '@ai-sdk/anthropic';
*
* const tools = {
* toolSearch: anthropicTools.toolSearchRegex_20251119(),
* // Other tools with deferLoading...
* };
* ```
*
* @see https://docs.anthropic.com/en/docs/agents-and-tools/tool-search-tool
*/
export const toolSearchRegex_20251119 = (
args: Parameters<typeof factory>[0] = {},
) => {
return factory(args);
};

145
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/web-fetch-20250910.ts generated vendored
View file

@ -1,145 +0,0 @@
import {
createProviderToolFactoryWithOutputSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
export const webFetch_20250910ArgsSchema = lazySchema(() =>
zodSchema(
z.object({
maxUses: z.number().optional(),
allowedDomains: z.array(z.string()).optional(),
blockedDomains: z.array(z.string()).optional(),
citations: z.object({ enabled: z.boolean() }).optional(),
maxContentTokens: z.number().optional(),
}),
),
);
export const webFetch_20250910OutputSchema = lazySchema(() =>
zodSchema(
z.object({
type: z.literal('web_fetch_result'),
url: z.string(),
content: z.object({
type: z.literal('document'),
title: z.string().nullable(),
citations: z.object({ enabled: z.boolean() }).optional(),
source: z.union([
z.object({
type: z.literal('base64'),
mediaType: z.literal('application/pdf'),
data: z.string(),
}),
z.object({
type: z.literal('text'),
mediaType: z.literal('text/plain'),
data: z.string(),
}),
]),
}),
retrievedAt: z.string().nullable(),
}),
),
);
const webFetch_20250910InputSchema = lazySchema(() =>
zodSchema(
z.object({
url: z.string(),
}),
),
);
const factory = createProviderToolFactoryWithOutputSchema<
{
/**
* The URL to fetch.
*/
url: string;
},
{
type: 'web_fetch_result';
/**
* Fetched content URL
*/
url: string;
/**
* Fetched content.
*/
content: {
type: 'document';
/**
* Title of the document
*/
title: string | null;
/**
* Citation configuration for the document
*/
citations?: { enabled: boolean };
source:
| {
type: 'base64';
mediaType: 'application/pdf';
data: string;
}
| {
type: 'text';
mediaType: 'text/plain';
data: string;
};
};
/**
* ISO 8601 timestamp when the content was retrieved
*/
retrievedAt: string | null;
},
{
/**
* The maxUses parameter limits the number of web fetches performed
*/
maxUses?: number;
/**
* Only fetch from these domains
*/
allowedDomains?: string[];
/**
* Never fetch from these domains
*/
blockedDomains?: string[];
/**
* Unlike web search where citations are always enabled, citations are optional for
* web fetch. Set "citations": {"enabled": true} to enable Claude to cite specific passages
* from fetched documents.
*/
citations?: {
enabled: boolean;
};
/**
* The maxContentTokens parameter limits the amount of content that will be included in the context.
*/
maxContentTokens?: number;
}
>({
id: 'anthropic.web_fetch_20250910',
inputSchema: webFetch_20250910InputSchema,
outputSchema: webFetch_20250910OutputSchema,
supportsDeferredResults: true,
});
export const webFetch_20250910 = (
args: Parameters<typeof factory>[0] = {}, // default
) => {
return factory(args);
};

145
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/web-fetch-20260209.ts generated vendored
View file

@ -1,145 +0,0 @@
import {
createProviderToolFactoryWithOutputSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
export const webFetch_20260209ArgsSchema = lazySchema(() =>
zodSchema(
z.object({
maxUses: z.number().optional(),
allowedDomains: z.array(z.string()).optional(),
blockedDomains: z.array(z.string()).optional(),
citations: z.object({ enabled: z.boolean() }).optional(),
maxContentTokens: z.number().optional(),
}),
),
);
export const webFetch_20260209OutputSchema = lazySchema(() =>
zodSchema(
z.object({
type: z.literal('web_fetch_result'),
url: z.string(),
content: z.object({
type: z.literal('document'),
title: z.string().nullable(),
citations: z.object({ enabled: z.boolean() }).optional(),
source: z.union([
z.object({
type: z.literal('base64'),
mediaType: z.literal('application/pdf'),
data: z.string(),
}),
z.object({
type: z.literal('text'),
mediaType: z.literal('text/plain'),
data: z.string(),
}),
]),
}),
retrievedAt: z.string().nullable(),
}),
),
);
const webFetch_20260209InputSchema = lazySchema(() =>
zodSchema(
z.object({
url: z.string(),
}),
),
);
const factory = createProviderToolFactoryWithOutputSchema<
{
/**
* The URL to fetch.
*/
url: string;
},
{
type: 'web_fetch_result';
/**
* Fetched content URL
*/
url: string;
/**
* Fetched content.
*/
content: {
type: 'document';
/**
* Title of the document
*/
title: string | null;
/**
* Citation configuration for the document
*/
citations?: { enabled: boolean };
source:
| {
type: 'base64';
mediaType: 'application/pdf';
data: string;
}
| {
type: 'text';
mediaType: 'text/plain';
data: string;
};
};
/**
* ISO 8601 timestamp when the content was retrieved
*/
retrievedAt: string | null;
},
{
/**
* The maxUses parameter limits the number of web fetches performed
*/
maxUses?: number;
/**
* Only fetch from these domains
*/
allowedDomains?: string[];
/**
* Never fetch from these domains
*/
blockedDomains?: string[];
/**
* Unlike web search where citations are always enabled, citations are optional for
* web fetch. Set "citations": {"enabled": true} to enable Claude to cite specific passages
* from fetched documents.
*/
citations?: {
enabled: boolean;
};
/**
* The maxContentTokens parameter limits the amount of content that will be included in the context.
*/
maxContentTokens?: number;
}
>({
id: 'anthropic.web_fetch_20260209',
inputSchema: webFetch_20260209InputSchema,
outputSchema: webFetch_20260209OutputSchema,
supportsDeferredResults: true,
});
export const webFetch_20260209 = (
args: Parameters<typeof factory>[0] = {}, // default
) => {
return factory(args);
};

136
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/web-search_20250305.ts generated vendored
View file

@ -1,136 +0,0 @@
import {
createProviderToolFactoryWithOutputSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
export const webSearch_20250305ArgsSchema = lazySchema(() =>
zodSchema(
z.object({
maxUses: z.number().optional(),
allowedDomains: z.array(z.string()).optional(),
blockedDomains: z.array(z.string()).optional(),
userLocation: z
.object({
type: z.literal('approximate'),
city: z.string().optional(),
region: z.string().optional(),
country: z.string().optional(),
timezone: z.string().optional(),
})
.optional(),
}),
),
);
export const webSearch_20250305OutputSchema = lazySchema(() =>
zodSchema(
z.array(
z.object({
url: z.string(),
title: z.string().nullable(),
pageAge: z.string().nullable(),
encryptedContent: z.string(),
type: z.literal('web_search_result'),
}),
),
),
);
const webSearch_20250305InputSchema = lazySchema(() =>
zodSchema(
z.object({
query: z.string(),
}),
),
);
const factory = createProviderToolFactoryWithOutputSchema<
{
/**
* The search query to execute.
*/
query: string;
},
Array<{
type: 'web_search_result';
/**
* The URL of the source page.
*/
url: string;
/**
* The title of the source page.
*/
title: string | null;
/**
* When the site was last updated
*/
pageAge: string | null;
/**
* Encrypted content that must be passed back in multi-turn conversations for citations
*/
encryptedContent: string;
}>,
{
/**
* Maximum number of web searches Claude can perform during the conversation.
*/
maxUses?: number;
/**
* Optional list of domains that Claude is allowed to search.
*/
allowedDomains?: string[];
/**
* Optional list of domains that Claude should avoid when searching.
*/
blockedDomains?: string[];
/**
* Optional user location information to provide geographically relevant search results.
*/
userLocation?: {
/**
* The type of location (must be approximate)
*/
type: 'approximate';
/**
* The city name
*/
city?: string;
/**
* The region or state
*/
region?: string;
/**
* The country
*/
country?: string;
/**
* The IANA timezone ID.
*/
timezone?: string;
};
}
>({
id: 'anthropic.web_search_20250305',
inputSchema: webSearch_20250305InputSchema,
outputSchema: webSearch_20250305OutputSchema,
supportsDeferredResults: true,
});
export const webSearch_20250305 = (
args: Parameters<typeof factory>[0] = {}, // default
) => {
return factory(args);
};

136
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/tool/web-search_20260209.ts generated vendored
View file

@ -1,136 +0,0 @@
import {
createProviderToolFactoryWithOutputSchema,
lazySchema,
zodSchema,
} from '@ai-sdk/provider-utils';
import { z } from 'zod/v4';
export const webSearch_20260209ArgsSchema = lazySchema(() =>
zodSchema(
z.object({
maxUses: z.number().optional(),
allowedDomains: z.array(z.string()).optional(),
blockedDomains: z.array(z.string()).optional(),
userLocation: z
.object({
type: z.literal('approximate'),
city: z.string().optional(),
region: z.string().optional(),
country: z.string().optional(),
timezone: z.string().optional(),
})
.optional(),
}),
),
);
export const webSearch_20260209OutputSchema = lazySchema(() =>
zodSchema(
z.array(
z.object({
url: z.string(),
title: z.string().nullable(),
pageAge: z.string().nullable(),
encryptedContent: z.string(),
type: z.literal('web_search_result'),
}),
),
),
);
const webSearch_20260209InputSchema = lazySchema(() =>
zodSchema(
z.object({
query: z.string(),
}),
),
);
const factory = createProviderToolFactoryWithOutputSchema<
{
/**
* The search query to execute.
*/
query: string;
},
Array<{
type: 'web_search_result';
/**
* The URL of the source page.
*/
url: string;
/**
* The title of the source page.
*/
title: string | null;
/**
* When the site was last updated
*/
pageAge: string | null;
/**
* Encrypted content that must be passed back in multi-turn conversations for citations
*/
encryptedContent: string;
}>,
{
/**
* Maximum number of web searches Claude can perform during the conversation.
*/
maxUses?: number;
/**
* Optional list of domains that Claude is allowed to search.
*/
allowedDomains?: string[];
/**
* Optional list of domains that Claude should avoid when searching.
*/
blockedDomains?: string[];
/**
* Optional user location information to provide geographically relevant search results.
*/
userLocation?: {
/**
* The type of location (must be approximate)
*/
type: 'approximate';
/**
* The city name
*/
city?: string;
/**
* The region or state
*/
region?: string;
/**
* The country
*/
country?: string;
/**
* The IANA timezone ID.
*/
timezone?: string;
};
}
>({
id: 'anthropic.web_search_20260209',
inputSchema: webSearch_20260209InputSchema,
outputSchema: webSearch_20260209OutputSchema,
supportsDeferredResults: true,
});
export const webSearch_20260209 = (
args: Parameters<typeof factory>[0] = {}, // default
) => {
return factory(args);
};

6
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/anthropic/src/version.ts generated vendored
View file

@ -1,6 +0,0 @@
// Version string of this package injected at build time.
declare const __PACKAGE_VERSION__: string | undefined;
export const VERSION: string =
typeof __PACKAGE_VERSION__ !== 'undefined'
? __PACKAGE_VERSION__
: '0.0.0-test';

1718
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/CHANGELOG.md generated vendored
View file

File diff suppressed because it is too large Load diff

13
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/LICENSE generated vendored
View file

@ -1,13 +0,0 @@
Copyright 2023 Vercel, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

44
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/README.md generated vendored
View file

@ -1,44 +0,0 @@
# AI SDK - Gateway Provider
The Gateway provider for the [AI SDK](https://ai-sdk.dev/docs) allows the use of a wide variety of AI models and providers.
## Setup
The Gateway provider is available in the `@ai-sdk/gateway` module. You can install it with
```bash
npm i @ai-sdk/gateway
```
## Skill for Coding Agents
If you use coding agents such as Claude Code or Cursor, we highly recommend adding the AI SDK skill to your repository:
```shell
npx skills add vercel/ai
```
## Provider Instance
You can import the default provider instance `gateway` from `@ai-sdk/gateway`:
```ts
import { gateway } from '@ai-sdk/gateway';
```
## Example
```ts
import { gateway } from '@ai-sdk/gateway';
import { generateText } from 'ai';
const { text } = await generateText({
model: gateway('xai/grok-3-beta'),
prompt:
'Tell me about the history of the San Francisco Mission-style burrito.',
});
```
## Documentation
Please check out the [AI SDK documentation](https://ai-sdk.dev/docs) for more information.

742
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/docs/00-ai-gateway.mdx generated vendored
View file

@ -1,742 +0,0 @@
---
title: AI Gateway
description: Learn how to use the AI Gateway provider with the AI SDK.
---
# AI Gateway Provider
The [AI Gateway](https://vercel.com/docs/ai-gateway) provider connects you to models from multiple AI providers through a single interface. Instead of integrating with each provider separately, you can access OpenAI, Anthropic, Google, Meta, xAI, and other providers and their models.
## Features
- Access models from multiple providers without having to install additional provider modules/dependencies
- Use the same code structure across different AI providers
- Switch between models and providers easily
- Automatic authentication when deployed on Vercel
- View pricing information across providers
- Observability for AI model usage through the Vercel dashboard
## Setup
The Vercel AI Gateway provider is part of the AI SDK.
## Basic Usage
For most use cases, you can use the AI Gateway directly with a model string:
```ts
// use plain model string with global provider
import { generateText } from 'ai';
const { text } = await generateText({
model: 'openai/gpt-5',
prompt: 'Hello world',
});
```
```ts
// use provider instance (requires version 5.0.36 or later)
import { generateText, gateway } from 'ai';
const { text } = await generateText({
model: gateway('openai/gpt-5'),
prompt: 'Hello world',
});
```
The AI SDK automatically uses the AI Gateway when you pass a model string in the `creator/model-name` format.
## Provider Instance
<Note>
The `gateway` provider instance is available from the `ai` package in version
5.0.36 and later.
</Note>
You can also import the default provider instance `gateway` from `ai`:
```ts
import { gateway } from 'ai';
```
You may want to create a custom provider instance when you need to:
- Set custom configuration options (API key, base URL, headers)
- Use the provider in a [provider registry](/docs/ai-sdk-core/provider-management)
- Wrap the provider with [middleware](/docs/ai-sdk-core/middleware)
- Use different settings for different parts of your application
To create a custom provider instance, import `createGateway` from `ai`:
```ts
import { createGateway } from 'ai';
const gateway = createGateway({
apiKey: process.env.AI_GATEWAY_API_KEY ?? '',
});
```
You can use the following optional settings to customize the AI Gateway provider instance:
- **baseURL** _string_
Use a different URL prefix for API calls. The default prefix is `https://ai-gateway.vercel.sh/v3/ai`.
- **apiKey** _string_
API key that is being sent using the `Authorization` header. It defaults to
the `AI_GATEWAY_API_KEY` environment variable.
- **headers** _Record&lt;string,string&gt;_
Custom headers to include in the requests.
- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_
Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
Defaults to the global `fetch` function.
You can use it as a middleware to intercept requests,
or to provide a custom fetch implementation for e.g. testing.
- **metadataCacheRefreshMillis** _number_
How frequently to refresh the metadata cache in milliseconds. Defaults to 5 minutes (300,000ms).
## Authentication
The Gateway provider supports two authentication methods:
### API Key Authentication
Set your API key via environment variable:
```bash
AI_GATEWAY_API_KEY=your_api_key_here
```
Or pass it directly to the provider:
```ts
import { createGateway } from 'ai';
const gateway = createGateway({
apiKey: 'your_api_key_here',
});
```
### OIDC Authentication (Vercel Deployments)
When deployed to Vercel, the AI Gateway provider supports authenticating using [OIDC (OpenID Connect)
tokens](https://vercel.com/docs/oidc) without API Keys.
#### How OIDC Authentication Works
1. **In Production/Preview Deployments**:
- OIDC authentication is automatically handled
- No manual configuration needed
- Tokens are automatically obtained and refreshed
2. **In Local Development**:
- First, install and authenticate with the [Vercel CLI](https://vercel.com/docs/cli)
- Run `vercel env pull` to download your project's OIDC token locally
- For automatic token management:
- Use `vercel dev` to start your development server - this will handle token refreshing automatically
- For manual token management:
- If not using `vercel dev`, note that OIDC tokens expire after 12 hours
- You'll need to run `vercel env pull` again to refresh the token before it expires
<Note>
If an API Key is present (either passed directly or via environment), it will
always be used, even if invalid.
</Note>
Read more about using OIDC tokens in the [Vercel AI Gateway docs](https://vercel.com/docs/ai-gateway#using-the-ai-gateway-with-a-vercel-oidc-token).
## Bring Your Own Key (BYOK)
You can connect your own provider credentials to use with Vercel AI Gateway. This lets you use your existing provider accounts and access private resources.
To set up BYOK, add your provider credentials in your Vercel team's AI Gateway settings. Once configured, AI Gateway automatically uses your credentials. No code changes are needed.
Learn more in the [BYOK documentation](https://vercel.com/docs/ai-gateway/byok).
## Language Models
You can create language models using a provider instance. The first argument is the model ID in the format `creator/model-name`:
```ts
import { generateText } from 'ai';
const { text } = await generateText({
model: 'openai/gpt-5',
prompt: 'Explain quantum computing in simple terms',
});
```
AI Gateway language models can also be used in the `streamText` function and support structured data generation with [`Output`](/docs/reference/ai-sdk-core/output) (see [AI SDK Core](/docs/ai-sdk-core)).
## Available Models
The AI Gateway supports models from OpenAI, Anthropic, Google, Meta, xAI, Mistral, DeepSeek, Amazon Bedrock, Cohere, Perplexity, Alibaba, and other providers.
For the complete list of available models, see the [AI Gateway documentation](https://vercel.com/docs/ai-gateway).
## Dynamic Model Discovery
You can discover available models programmatically:
```ts
import { gateway, generateText } from 'ai';
const availableModels = await gateway.getAvailableModels();
// List all available models
availableModels.models.forEach(model => {
console.log(`${model.id}: ${model.name}`);
if (model.description) {
console.log(` Description: ${model.description}`);
}
if (model.pricing) {
console.log(` Input: $${model.pricing.input}/token`);
console.log(` Output: $${model.pricing.output}/token`);
if (model.pricing.cachedInputTokens) {
console.log(
` Cached input (read): $${model.pricing.cachedInputTokens}/token`,
);
}
if (model.pricing.cacheCreationInputTokens) {
console.log(
` Cache creation (write): $${model.pricing.cacheCreationInputTokens}/token`,
);
}
}
});
// Use any discovered model with plain string
const { text } = await generateText({
model: availableModels.models[0].id, // e.g., 'openai/gpt-4o'
prompt: 'Hello world',
});
```
## Credit Usage
You can check your team's current credit balance and usage:
```ts
import { gateway } from 'ai';
const credits = await gateway.getCredits();
console.log(`Team balance: ${credits.balance} credits`);
console.log(`Team total used: ${credits.total_used} credits`);
```
The `getCredits()` method returns your team's credit information based on the authenticated API key or OIDC token:
- **balance** _number_ - Your team's current available credit balance
- **total_used** _number_ - Total credits consumed by your team
## Examples
### Basic Text Generation
```ts
import { generateText } from 'ai';
const { text } = await generateText({
model: 'anthropic/claude-sonnet-4',
prompt: 'Write a haiku about programming',
});
console.log(text);
```
### Streaming
```ts
import { streamText } from 'ai';
const { textStream } = await streamText({
model: 'openai/gpt-5',
prompt: 'Explain the benefits of serverless architecture',
});
for await (const textPart of textStream) {
process.stdout.write(textPart);
}
```
### Tool Usage
```ts
import { generateText, tool } from 'ai';
import { z } from 'zod';
const { text } = await generateText({
model: 'xai/grok-4',
prompt: 'What is the weather like in San Francisco?',
tools: {
getWeather: tool({
description: 'Get the current weather for a location',
parameters: z.object({
location: z.string().describe('The location to get weather for'),
}),
execute: async ({ location }) => {
// Your weather API call here
return `It's sunny in ${location}`;
},
}),
},
});
```
### Provider-Executed Tools
Some providers offer tools that are executed by the provider itself, such as [OpenAI's web search tool](/providers/ai-sdk-providers/openai#web-search-tool). To use these tools through AI Gateway, import the provider to access the tool definitions:
```ts
import { generateText, stepCountIs } from 'ai';
import { openai } from '@ai-sdk/openai';
const result = await generateText({
model: 'openai/gpt-5-mini',
prompt: 'What is the Vercel AI Gateway?',
stopWhen: stepCountIs(10),
tools: {
web_search: openai.tools.webSearch({}),
},
});
console.dir(result.text);
```
<Note>
Some provider-executed tools require account-specific configuration (such as
Claude Agent Skills) and may not work through AI Gateway. To use these tools,
you must bring your own key (BYOK) directly to the provider.
</Note>
### Gateway Tools
The AI Gateway provider includes built-in tools that are executed by the gateway itself. These tools can be used with any model through the gateway.
#### Perplexity Search
The Perplexity Search tool enables models to search the web using [Perplexity's search API](https://docs.perplexity.ai/guides/search-quickstart). This tool is executed by the AI Gateway and returns web search results that the model can use to provide up-to-date information.
```ts
import { gateway, generateText } from 'ai';
const result = await generateText({
model: 'openai/gpt-5-nano',
prompt: 'Search for news about AI regulations in January 2025.',
tools: {
perplexity_search: gateway.tools.perplexitySearch(),
},
});
console.log(result.text);
console.log('Tool calls:', JSON.stringify(result.toolCalls, null, 2));
console.log('Tool results:', JSON.stringify(result.toolResults, null, 2));
```
You can also configure the search with optional parameters:
```ts
import { gateway, generateText } from 'ai';
const result = await generateText({
model: 'openai/gpt-5-nano',
prompt:
'Search for news about AI regulations from the first week of January 2025.',
tools: {
perplexity_search: gateway.tools.perplexitySearch({
maxResults: 5,
searchLanguageFilter: ['en'],
country: 'US',
searchDomainFilter: ['reuters.com', 'bbc.com', 'nytimes.com'],
}),
},
});
console.log(result.text);
console.log('Tool calls:', JSON.stringify(result.toolCalls, null, 2));
console.log('Tool results:', JSON.stringify(result.toolResults, null, 2));
```
The Perplexity Search tool supports the following optional configuration options:
- **maxResults** _number_
The maximum number of search results to return (1-20, default: 10).
- **maxTokensPerPage** _number_
The maximum number of tokens to extract per search result page (256-2048, default: 2048).
- **maxTokens** _number_
The maximum total tokens across all search results (default: 25000, max: 1000000).
- **searchLanguageFilter** _string[]_
Filter search results by language using ISO 639-1 language codes (e.g., `['en']` for English, `['en', 'es']` for English and Spanish).
- **country** _string_
Filter search results by country using ISO 3166-1 alpha-2 country codes (e.g., `'US'` for United States, `'GB'` for United Kingdom).
- **searchDomainFilter** _string[]_
Limit search results to specific domains (e.g., `['reuters.com', 'bbc.com']`). This is useful for restricting results to trusted sources.
- **searchRecencyFilter** _'day' | 'week' | 'month' | 'year'_
Filter search results by relative time period. Useful for always getting recent results (e.g., 'week' for results from the last week).
The tool works with both `generateText` and `streamText`:
```ts
import { gateway, streamText } from 'ai';
const result = streamText({
model: 'openai/gpt-5-nano',
prompt: 'Search for the latest news about AI regulations.',
tools: {
perplexity_search: gateway.tools.perplexitySearch(),
},
});
for await (const part of result.fullStream) {
switch (part.type) {
case 'text-delta':
process.stdout.write(part.text);
break;
case 'tool-call':
console.log('\nTool call:', JSON.stringify(part, null, 2));
break;
case 'tool-result':
console.log('\nTool result:', JSON.stringify(part, null, 2));
break;
}
}
```
#### Parallel Search
The Parallel Search tool enables models to search the web using [Parallel AI's Search API](https://docs.parallel.ai/api-reference/search-beta/search). This tool is optimized for LLM consumption, returning relevant excerpts from web pages that can replace multiple keyword searches with a single call.
```ts
import { gateway, generateText } from 'ai';
const result = await generateText({
model: 'openai/gpt-5-nano',
prompt: 'Research the latest developments in quantum computing.',
tools: {
parallel_search: gateway.tools.parallelSearch(),
},
});
console.log(result.text);
console.log('Tool calls:', JSON.stringify(result.toolCalls, null, 2));
console.log('Tool results:', JSON.stringify(result.toolResults, null, 2));
```
You can also configure the search with optional parameters:
```ts
import { gateway, generateText } from 'ai';
const result = await generateText({
model: 'openai/gpt-5-nano',
prompt: 'Find detailed information about TypeScript 5.0 features.',
tools: {
parallel_search: gateway.tools.parallelSearch({
mode: 'agentic',
maxResults: 5,
sourcePolicy: {
includeDomains: ['typescriptlang.org', 'github.com'],
},
excerpts: {
maxCharsPerResult: 8000,
},
}),
},
});
console.log(result.text);
console.log('Tool calls:', JSON.stringify(result.toolCalls, null, 2));
console.log('Tool results:', JSON.stringify(result.toolResults, null, 2));
```
The Parallel Search tool supports the following optional configuration options:
- **mode** _'one-shot' | 'agentic'_
Mode preset for different use cases:
- `'one-shot'` - Comprehensive results with longer excerpts for single-response answers (default)
- `'agentic'` - Concise, token-efficient results optimized for multi-step agentic workflows
- **maxResults** _number_
Maximum number of results to return (1-20). Defaults to 10 if not specified.
- **sourcePolicy** _object_
Source policy for controlling which domains to include/exclude:
- `includeDomains` - List of domains to include in search results
- `excludeDomains` - List of domains to exclude from search results
- `afterDate` - Only include results published after this date (ISO 8601 format)
- **excerpts** _object_
Excerpt configuration for controlling result length:
- `maxCharsPerResult` - Maximum characters per result
- `maxCharsTotal` - Maximum total characters across all results
- **fetchPolicy** _object_
Fetch policy for controlling content freshness:
- `maxAgeSeconds` - Maximum age in seconds for cached content (set to 0 for always fresh)
The tool works with both `generateText` and `streamText`:
```ts
import { gateway, streamText } from 'ai';
const result = streamText({
model: 'openai/gpt-5-nano',
prompt: 'Research the latest AI safety guidelines.',
tools: {
parallel_search: gateway.tools.parallelSearch(),
},
});
for await (const part of result.fullStream) {
switch (part.type) {
case 'text-delta':
process.stdout.write(part.text);
break;
case 'tool-call':
console.log('\nTool call:', JSON.stringify(part, null, 2));
break;
case 'tool-result':
console.log('\nTool result:', JSON.stringify(part, null, 2));
break;
}
}
```
### Usage Tracking with User and Tags
Track usage per end-user and categorize requests with tags:
```ts
import type { GatewayLanguageModelOptions } from '@ai-sdk/gateway';
import { generateText } from 'ai';
const { text } = await generateText({
model: 'openai/gpt-5',
prompt: 'Summarize this document...',
providerOptions: {
gateway: {
user: 'user-abc-123', // Track usage for this specific end-user
tags: ['document-summary', 'premium-feature'], // Categorize for reporting
} satisfies GatewayLanguageModelOptions,
},
});
```
This allows you to:
- View usage and costs broken down by end-user in your analytics
- Filter and analyze spending by feature or use case using tags
- Track which users or features are driving the most AI usage
## Provider Options
The AI Gateway provider accepts provider options that control routing behavior and provider-specific configurations.
### Gateway Provider Options
You can use the `gateway` key in `providerOptions` to control how AI Gateway routes requests:
```ts
import type { GatewayLanguageModelOptions } from '@ai-sdk/gateway';
import { generateText } from 'ai';
const { text } = await generateText({
model: 'anthropic/claude-sonnet-4',
prompt: 'Explain quantum computing',
providerOptions: {
gateway: {
order: ['vertex', 'anthropic'], // Try Vertex AI first, then Anthropic
only: ['vertex', 'anthropic'], // Only use these providers
} satisfies GatewayLanguageModelOptions,
},
});
```
The following gateway provider options are available:
- **order** _string[]_
Specifies the sequence of providers to attempt when routing requests. The gateway will try providers in the order specified. If a provider fails or is unavailable, it will move to the next provider in the list.
Example: `order: ['bedrock', 'anthropic']` will attempt Amazon Bedrock first, then fall back to Anthropic.
- **only** _string[]_
Restricts routing to only the specified providers. When set, the gateway will never route to providers not in this list, even if they would otherwise be available.
Example: `only: ['anthropic', 'vertex']` will only allow routing to Anthropic or Vertex AI.
- **models** _string[]_
Specifies fallback models to use when the primary model fails or is unavailable. The gateway will try the primary model first (specified in the `model` parameter), then try each model in this array in order until one succeeds.
Example: `models: ['openai/gpt-5-nano', 'gemini-2.0-flash']` will try the fallback models in order if the primary model fails.
- **user** _string_
Optional identifier for the end user on whose behalf the request is being made. This is used for spend tracking and attribution purposes, allowing you to track usage per end-user in your application.
Example: `user: 'user-123'` will associate this request with end-user ID "user-123" in usage reports.
- **tags** _string[]_
Optional array of tags for categorizing and filtering usage in reports. Useful for tracking spend by feature, prompt version, or any other dimension relevant to your application.
Example: `tags: ['chat', 'v2']` will tag this request with "chat" and "v2" for filtering in usage analytics.
- **byok** _Record&lt;string, Array&lt;Record&lt;string, unknown&gt;&gt;&gt;_
Request-scoped BYOK (Bring Your Own Key) credentials to use for this request. When provided, any cached BYOK credentials configured in the gateway system are not considered. Requests may still fall back to use system credentials if the provided credentials fail.
Each provider can have multiple credentials (tried in order). The structure is a record where keys are provider slugs and values are arrays of credential objects.
Examples:
- Single provider: `byok: { 'anthropic': [{ apiKey: 'sk-ant-...' }] }`
- Multiple credentials: `byok: { 'vertex': [{ project: 'proj-1', googleCredentials: { privateKey: '...', clientEmail: '...' } }, { project: 'proj-2', googleCredentials: { privateKey: '...', clientEmail: '...' } }] }`
- Multiple providers: `byok: { 'anthropic': [{ apiKey: '...' }], 'bedrock': [{ accessKeyId: '...', secretAccessKey: '...' }] }`
- **zeroDataRetention** _boolean_
Restricts routing requests to providers that have zero data retention policies.
- **providerTimeouts** _object_
Per-provider timeouts for BYOK credentials in milliseconds. Controls how long to wait for a provider to start responding before falling back to the next available provider.
Example: `providerTimeouts: { byok: { openai: 5000, anthropic: 2000 } }`
For full details, see [Provider Timeouts](https://vercel.com/docs/ai-gateway/models-and-providers/provider-timeouts).
You can combine these options to have fine-grained control over routing and tracking:
```ts
import type { GatewayLanguageModelOptions } from '@ai-sdk/gateway';
import { generateText } from 'ai';
const { text } = await generateText({
model: 'anthropic/claude-sonnet-4',
prompt: 'Write a haiku about programming',
providerOptions: {
gateway: {
order: ['vertex'], // Prefer Vertex AI
only: ['anthropic', 'vertex'], // Only allow these providers
} satisfies GatewayLanguageModelOptions,
},
});
```
#### Model Fallbacks Example
The `models` option enables automatic fallback to alternative models when the primary model fails:
```ts
import type { GatewayLanguageModelOptions } from '@ai-sdk/gateway';
import { generateText } from 'ai';
const { text } = await generateText({
model: 'openai/gpt-4o', // Primary model
prompt: 'Write a TypeScript haiku',
providerOptions: {
gateway: {
models: ['openai/gpt-5-nano', 'gemini-2.0-flash'], // Fallback models
} satisfies GatewayLanguageModelOptions,
},
});
// This will:
// 1. Try openai/gpt-4o first
// 2. If it fails, try openai/gpt-5-nano
// 3. If that fails, try gemini-2.0-flash
// 4. Return the result from the first model that succeeds
```
#### Zero Data Retention Example
Set `zeroDataRetention` to true to ensure requests are only routed to providers
that have zero data retention policies. When `zeroDataRetention` is `false` or not
specified, there is no enforcement of restricting routing.
```ts
import type { GatewayLanguageModelOptions } from '@ai-sdk/gateway';
import { generateText } from 'ai';
const { text } = await generateText({
model: 'anthropic/claude-sonnet-4.5',
prompt: 'Analyze this sensitive document...',
providerOptions: {
gateway: {
zeroDataRetention: true,
} satisfies GatewayLanguageModelOptions,
},
});
```
### Provider-Specific Options
When using provider-specific options through AI Gateway, use the actual provider name (e.g. `anthropic`, `openai`, not `gateway`) as the key:
```ts
import type { AnthropicLanguageModelOptions } from '@ai-sdk/anthropic';
import type { GatewayLanguageModelOptions } from '@ai-sdk/gateway';
import { generateText } from 'ai';
const { text } = await generateText({
model: 'anthropic/claude-sonnet-4',
prompt: 'Explain quantum computing',
providerOptions: {
gateway: {
order: ['vertex', 'anthropic'],
} satisfies GatewayLanguageModelOptions,
anthropic: {
thinking: { type: 'enabled', budgetTokens: 12000 },
} satisfies AnthropicLanguageModelOptions,
},
});
```
This works with any provider supported by AI Gateway. Each provider has its own set of options - see the individual [provider documentation pages](/providers/ai-sdk-providers) for details on provider-specific options.
### Available Providers
AI Gateway supports routing to 20+ providers.
For a complete list of available providers and their slugs, see the [AI Gateway documentation](https://vercel.com/docs/ai-gateway/provider-options#available-providers).
## Model Capabilities
Model capabilities depend on the specific provider and model you're using. For detailed capability information, see:
- [AI Gateway provider options](https://vercel.com/docs/ai-gateway/provider-options#available-providers) for an overview of available providers
- Individual [AI SDK provider pages](/providers/ai-sdk-providers) for specific model capabilities and features

78
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/package.json generated vendored
View file

@ -1,78 +0,0 @@
{
"name": "@ai-sdk/gateway",
"private": false,
"version": "3.0.80",
"license": "Apache-2.0",
"sideEffects": false,
"main": "./dist/index.js",
"module": "./dist/index.mjs",
"types": "./dist/index.d.ts",
"files": [
"dist/**/*",
"docs/**/*",
"src",
"!src/**/*.test.ts",
"!src/**/*.test-d.ts",
"!src/**/__snapshots__",
"!src/**/__fixtures__",
"CHANGELOG.md",
"README.md"
],
"directories": {
"doc": "./docs"
},
"exports": {
"./package.json": "./package.json",
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.mjs",
"require": "./dist/index.js"
}
},
"dependencies": {
"@vercel/oidc": "3.1.0",
"@ai-sdk/provider": "3.0.8",
"@ai-sdk/provider-utils": "4.0.21"
},
"devDependencies": {
"@types/node": "18.15.11",
"tsup": "^8",
"tsx": "4.19.2",
"typescript": "5.8.3",
"zod": "3.25.76",
"@ai-sdk/test-server": "1.0.3",
"@vercel/ai-tsconfig": "0.0.0"
},
"peerDependencies": {
"zod": "^3.25.76 || ^4.1.8"
},
"engines": {
"node": ">=18"
},
"publishConfig": {
"access": "public"
},
"homepage": "https://ai-sdk.dev/docs",
"repository": {
"type": "git",
"url": "git+https://github.com/vercel/ai.git"
},
"bugs": {
"url": "https://github.com/vercel/ai/issues"
},
"keywords": [
"ai"
],
"scripts": {
"build": "pnpm clean && tsup --tsconfig tsconfig.build.json",
"build:watch": "pnpm clean && tsup --watch",
"clean": "del-cli dist docs *.tsbuildinfo",
"generate-model-settings": "tsx scripts/generate-model-settings.ts",
"type-check": "tsc --build",
"test": "pnpm test:node && pnpm test:edge",
"test:update": "pnpm test:node -u",
"test:watch": "vitest --config vitest.node.config.js",
"test:edge": "vitest --config vitest.edge.config.js --run",
"test:node": "vitest --config vitest.node.config.js --run"
}
}

74
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/src/errors/as-gateway-error.ts generated vendored
View file

@ -1,74 +0,0 @@
import { APICallError } from '@ai-sdk/provider';
import { extractApiCallResponse, GatewayError } from '.';
import { createGatewayErrorFromResponse } from './create-gateway-error';
import { GatewayTimeoutError } from './gateway-timeout-error';
/**
* Checks if an error is a timeout error from undici.
* Only checks undici-specific error codes to avoid false positives.
*/
function isTimeoutError(error: unknown): boolean {
if (!(error instanceof Error)) {
return false;
}
// Check for undici-specific timeout error codes
const errorCode = (error as any).code;
if (typeof errorCode === 'string') {
const undiciTimeoutCodes = [
'UND_ERR_HEADERS_TIMEOUT',
'UND_ERR_BODY_TIMEOUT',
'UND_ERR_CONNECT_TIMEOUT',
];
return undiciTimeoutCodes.includes(errorCode);
}
return false;
}
export async function asGatewayError(
error: unknown,
authMethod?: 'api-key' | 'oidc',
) {
if (GatewayError.isInstance(error)) {
return error;
}
// Check if this is a timeout error (or has a timeout error in the cause chain)
if (isTimeoutError(error)) {
return GatewayTimeoutError.createTimeoutError({
originalMessage: error instanceof Error ? error.message : 'Unknown error',
cause: error,
});
}
// Check if this is an APICallError caused by a timeout
if (APICallError.isInstance(error)) {
// Check if the cause is a timeout error
if (error.cause && isTimeoutError(error.cause)) {
return GatewayTimeoutError.createTimeoutError({
originalMessage: error.message,
cause: error,
});
}
return await createGatewayErrorFromResponse({
response: extractApiCallResponse(error),
statusCode: error.statusCode ?? 500,
defaultMessage: 'Gateway request failed',
cause: error,
authMethod,
});
}
return await createGatewayErrorFromResponse({
response: {},
statusCode: 500,
defaultMessage:
error instanceof Error
? `Gateway request failed: ${error.message}`
: 'Unknown Gateway error',
cause: error,
authMethod,
});
}

132
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/src/errors/create-gateway-error.ts generated vendored
View file

@ -1,132 +0,0 @@
import { z } from 'zod/v4';
import type { GatewayError } from './gateway-error';
import { GatewayAuthenticationError } from './gateway-authentication-error';
import { GatewayInvalidRequestError } from './gateway-invalid-request-error';
import { GatewayRateLimitError } from './gateway-rate-limit-error';
import {
GatewayModelNotFoundError,
modelNotFoundParamSchema,
} from './gateway-model-not-found-error';
import { GatewayInternalServerError } from './gateway-internal-server-error';
import { GatewayResponseError } from './gateway-response-error';
import {
InferSchema,
lazySchema,
safeValidateTypes,
validateTypes,
zodSchema,
} from '@ai-sdk/provider-utils';
export async function createGatewayErrorFromResponse({
response,
statusCode,
defaultMessage = 'Gateway request failed',
cause,
authMethod,
}: {
response: unknown;
statusCode: number;
defaultMessage?: string;
cause?: unknown;
authMethod?: 'api-key' | 'oidc';
}): Promise<GatewayError> {
const parseResult = await safeValidateTypes({
value: response,
schema: gatewayErrorResponseSchema,
});
if (!parseResult.success) {
// Try to extract generationId even if validation failed
const rawGenerationId =
typeof response === 'object' &&
response !== null &&
'generationId' in response
? (response as { generationId?: string }).generationId
: undefined;
return new GatewayResponseError({
message: `Invalid error response format: ${defaultMessage}`,
statusCode,
response,
validationError: parseResult.error,
cause,
generationId: rawGenerationId,
});
}
const validatedResponse: GatewayErrorResponse = parseResult.value;
const errorType = validatedResponse.error.type;
const message = validatedResponse.error.message;
const generationId = validatedResponse.generationId ?? undefined;
switch (errorType) {
case 'authentication_error':
return GatewayAuthenticationError.createContextualError({
apiKeyProvided: authMethod === 'api-key',
oidcTokenProvided: authMethod === 'oidc',
statusCode,
cause,
generationId,
});
case 'invalid_request_error':
return new GatewayInvalidRequestError({
message,
statusCode,
cause,
generationId,
});
case 'rate_limit_exceeded':
return new GatewayRateLimitError({
message,
statusCode,
cause,
generationId,
});
case 'model_not_found': {
const modelResult = await safeValidateTypes({
value: validatedResponse.error.param,
schema: modelNotFoundParamSchema,
});
return new GatewayModelNotFoundError({
message,
statusCode,
modelId: modelResult.success ? modelResult.value.modelId : undefined,
cause,
generationId,
});
}
case 'internal_server_error':
return new GatewayInternalServerError({
message,
statusCode,
cause,
generationId,
});
default:
return new GatewayInternalServerError({
message,
statusCode,
cause,
generationId,
});
}
}
const gatewayErrorResponseSchema = lazySchema(() =>
zodSchema(
z.object({
error: z.object({
message: z.string(),
type: z.string().nullish(),
param: z.unknown().nullish(),
code: z.union([z.string(), z.number()]).nullish(),
}),
generationId: z.string().nullish(),
}),
),
);
export type GatewayErrorResponse = InferSchema<
typeof gatewayErrorResponseSchema
>;

15
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/src/errors/extract-api-call-response.ts generated vendored
View file

@ -1,15 +0,0 @@
import type { APICallError } from '@ai-sdk/provider';
export function extractApiCallResponse(error: APICallError): unknown {
if (error.data !== undefined) {
return error.data;
}
if (error.responseBody != null) {
try {
return JSON.parse(error.responseBody);
} catch {
return error.responseBody;
}
}
return {};
}

84
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/src/errors/gateway-authentication-error.ts generated vendored
View file

@ -1,84 +0,0 @@
import { GatewayError } from './gateway-error';
const name = 'GatewayAuthenticationError';
const marker = `vercel.ai.gateway.error.${name}`;
const symbol = Symbol.for(marker);
/**
* Authentication failed - invalid API key or OIDC token
*/
export class GatewayAuthenticationError extends GatewayError {
private readonly [symbol] = true; // used in isInstance
readonly name = name;
readonly type = 'authentication_error';
constructor({
message = 'Authentication failed',
statusCode = 401,
cause,
generationId,
}: {
message?: string;
statusCode?: number;
cause?: unknown;
generationId?: string;
} = {}) {
super({ message, statusCode, cause, generationId });
}
static isInstance(error: unknown): error is GatewayAuthenticationError {
return GatewayError.hasMarker(error) && symbol in error;
}
/**
* Creates a contextual error message when authentication fails
*/
static createContextualError({
apiKeyProvided,
oidcTokenProvided,
message = 'Authentication failed',
statusCode = 401,
cause,
generationId,
}: {
apiKeyProvided: boolean;
oidcTokenProvided: boolean;
message?: string;
statusCode?: number;
cause?: unknown;
generationId?: string;
}): GatewayAuthenticationError {
let contextualMessage: string;
if (apiKeyProvided) {
contextualMessage = `AI Gateway authentication failed: Invalid API key.
Create a new API key: https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fai%2Fapi-keys
Provide via 'apiKey' option or 'AI_GATEWAY_API_KEY' environment variable.`;
} else if (oidcTokenProvided) {
contextualMessage = `AI Gateway authentication failed: Invalid OIDC token.
Run 'npx vercel link' to link your project, then 'vc env pull' to fetch the token.
Alternatively, use an API key: https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fai%2Fapi-keys`;
} else {
contextualMessage = `AI Gateway authentication failed: No authentication provided.
Option 1 - API key:
Create an API key: https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fai%2Fapi-keys
Provide via 'apiKey' option or 'AI_GATEWAY_API_KEY' environment variable.
Option 2 - OIDC token:
Run 'npx vercel link' to link your project, then 'vc env pull' to fetch the token.`;
}
return new GatewayAuthenticationError({
message: contextualMessage,
statusCode,
cause,
generationId,
});
}
}

47
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/src/errors/gateway-error.ts generated vendored
View file

@ -1,47 +0,0 @@
const marker = 'vercel.ai.gateway.error';
const symbol = Symbol.for(marker);
export abstract class GatewayError extends Error {
private readonly [symbol] = true; // used in isInstance
abstract readonly name: string;
abstract readonly type: string;
readonly statusCode: number;
readonly cause?: unknown;
readonly generationId?: string;
constructor({
message,
statusCode = 500,
cause,
generationId,
}: {
message: string;
statusCode?: number;
cause?: unknown;
generationId?: string;
}) {
super(generationId ? `${message} [${generationId}]` : message);
this.statusCode = statusCode;
this.cause = cause;
this.generationId = generationId;
}
/**
* Checks if the given error is a Gateway Error.
* @param {unknown} error - The error to check.
* @returns {boolean} True if the error is a Gateway Error, false otherwise.
*/
static isInstance(error: unknown): error is GatewayError {
return GatewayError.hasMarker(error);
}
static hasMarker(error: unknown): error is GatewayError {
return (
typeof error === 'object' &&
error !== null &&
symbol in error &&
(error as any)[symbol] === true
);
}
}

33
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/src/errors/gateway-internal-server-error.ts generated vendored
View file

@ -1,33 +0,0 @@
import { GatewayError } from './gateway-error';
const name = 'GatewayInternalServerError';
const marker = `vercel.ai.gateway.error.${name}`;
const symbol = Symbol.for(marker);
/**
* Internal server error from the Gateway
*/
export class GatewayInternalServerError extends GatewayError {
private readonly [symbol] = true; // used in isInstance
readonly name = name;
readonly type = 'internal_server_error';
constructor({
message = 'Internal server error',
statusCode = 500,
cause,
generationId,
}: {
message?: string;
statusCode?: number;
cause?: unknown;
generationId?: string;
} = {}) {
super({ message, statusCode, cause, generationId });
}
static isInstance(error: unknown): error is GatewayInternalServerError {
return GatewayError.hasMarker(error) && symbol in error;
}
}

33
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/src/errors/gateway-invalid-request-error.ts generated vendored
View file

@ -1,33 +0,0 @@
import { GatewayError } from './gateway-error';
const name = 'GatewayInvalidRequestError';
const marker = `vercel.ai.gateway.error.${name}`;
const symbol = Symbol.for(marker);
/**
* Invalid request - missing headers, malformed data, etc.
*/
export class GatewayInvalidRequestError extends GatewayError {
private readonly [symbol] = true; // used in isInstance
readonly name = name;
readonly type = 'invalid_request_error';
constructor({
message = 'Invalid request',
statusCode = 400,
cause,
generationId,
}: {
message?: string;
statusCode?: number;
cause?: unknown;
generationId?: string;
} = {}) {
super({ message, statusCode, cause, generationId });
}
static isInstance(error: unknown): error is GatewayInvalidRequestError {
return GatewayError.hasMarker(error) && symbol in error;
}
}

47
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/src/errors/gateway-model-not-found-error.ts generated vendored
View file

@ -1,47 +0,0 @@
import { z } from 'zod/v4';
import { GatewayError } from './gateway-error';
import { lazySchema, zodSchema } from '@ai-sdk/provider-utils';
const name = 'GatewayModelNotFoundError';
const marker = `vercel.ai.gateway.error.${name}`;
const symbol = Symbol.for(marker);
export const modelNotFoundParamSchema = lazySchema(() =>
zodSchema(
z.object({
modelId: z.string(),
}),
),
);
/**
* Model not found or not available
*/
export class GatewayModelNotFoundError extends GatewayError {
private readonly [symbol] = true; // used in isInstance
readonly name = name;
readonly type = 'model_not_found';
readonly modelId?: string;
constructor({
message = 'Model not found',
statusCode = 404,
modelId,
cause,
generationId,
}: {
message?: string;
statusCode?: number;
modelId?: string;
cause?: unknown;
generationId?: string;
} = {}) {
super({ message, statusCode, cause, generationId });
this.modelId = modelId;
}
static isInstance(error: unknown): error is GatewayModelNotFoundError {
return GatewayError.hasMarker(error) && symbol in error;
}
}

33
examples/vercel-ai-sdk-pr/node-headroom-compression/node_modules/@ai-sdk/gateway/src/errors/gateway-rate-limit-error.ts generated vendored
View file

@ -1,33 +0,0 @@
import { GatewayError } from './gateway-error';
const name = 'GatewayRateLimitError';
const marker = `vercel.ai.gateway.error.${name}`;
const symbol = Symbol.for(marker);
/**
* Rate limit exceeded.
*/
export class GatewayRateLimitError extends GatewayError {
private readonly [symbol] = true; // used in isInstance
readonly name = name;
readonly type = 'rate_limit_exceeded';
constructor({
message = 'Rate limit exceeded',
statusCode = 429,
cause,
generationId,
}: {
message?: string;
statusCode?: number;
cause?: unknown;
generationId?: string;
} = {}) {
super({ message, statusCode, cause, generationId });
}
static isInstance(error: unknown): error is GatewayRateLimitError {
return GatewayError.hasMarker(error) && symbol in error;
}
}

Some files were not shown because too many files have changed in this diff Show more