headroom/headroom
Tejas Chopra 2f2950a626
Some checks failed
CI / workflow-validation (push) Failing after 0s
CI / commitlint (push) Failing after 1s
CI / changes (push) Failing after 7s
Dev Containers / validate (.devcontainer/devcontainer.json, default) (push) Failing after 4s
Dev Containers / validate (.devcontainer/memory-stack/devcontainer.json, memory-stack) (push) Failing after 3s
Init Native E2E / init-native (ubuntu-latest, claude) (push) Failing after 4s
OpenCode Plugin / typecheck + build + test (push) Failing after 9s
Dev Containers / validate-worktree (push) Failing after 3s
Init E2E / docker-init-e2e (push) Failing after 3s
Init Native E2E / init-native (ubuntu-latest, codex) (push) Failing after 4s
Init Native E2E / init-native (ubuntu-latest, copilot) (push) Failing after 3s
Install Native E2E / install-native (ubuntu-latest) (push) Failing after 3s
Merge Conflicts / merge-conflicts (push) Failing after 3s
Release Please / release-please (push) Failing after 7s
rust / rust-changes (push) Failing after 7s
Wrap E2E / docker-wrap-e2e (push) Failing after 2s
Wrap Native E2E / wrap-native (ubuntu-latest) (push) Failing after 2s
Security / Dependency audit (pip-audit) (push) Failing after 3s
Security / CodeQL (javascript-typescript) (push) Failing after 2s
Security / CodeQL (python) (push) Failing after 2s
Security / Secret scan (gitleaks) (push) Failing after 2s
CI / lint (push) Failing after 0s
CI / build-wheel (push) Failing after 0s
CI / prefetch-model (push) Failing after 0s
CI / build (push) Failing after 0s
rust / simulator e2e (ubuntu-latest) (push) Failing after 0s
rust / wheels (x86_64-unknown-linux-gnu) (push) Failing after 0s
rust / audit (push) Failing after 0s
rust / parity (push) Failing after 0s
CI / docker-native-e2e (push) Failing after 0s
rust / test (ubuntu) (push) Failing after 0s
CI / test-extras (push) Failing after 0s
CI / test-agno (push) Failing after 0s
CI / test-dashboard-ui (push) Failing after 0s
CI / test (1) (push) Failing after 0s
CI / test (2) (push) Failing after 0s
CI / test (3) (push) Failing after 0s
CI / test (4) (push) Failing after 0s
CI / windows-native-wrapper (push) Has been cancelled
CI / macos-native-wrapper (push) Has been cancelled
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime name:]) (push) Has been cancelled
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-code name:code]) (push) Has been cancelled
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-code-nonroot name:code-nonroot]) (push) Has been cancelled
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-code-slim name:code-slim]) (push) Has been cancelled
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-code-slim-nonroot name:code-slim-nonroot]) (push) Has been cancelled
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-nonroot name:nonroot]) (push) Has been cancelled
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-slim name:slim]) (push) Has been cancelled
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-slim-nonroot name:slim-nonroot]) (push) Has been cancelled
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime name:]) (push) Has been cancelled
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-code name:code]) (push) Has been cancelled
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-code-nonroot name:code-nonroot]) (push) Has been cancelled
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-code-slim name:code-slim]) (push) Has been cancelled
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-code-slim-nonroot name:code-slim-nonroot]) (push) Has been cancelled
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-nonroot name:nonroot]) (push) Has been cancelled
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-slim name:slim]) (push) Has been cancelled
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-slim-nonroot name:slim-nonroot]) (push) Has been cancelled
Init Native E2E / init-native (macos-latest, claude) (push) Has been cancelled
Init Native E2E / init-native (macos-latest, codex) (push) Has been cancelled
Init Native E2E / init-native (macos-latest, copilot) (push) Has been cancelled
rust / simulator e2e (macos-latest) (push) Has been cancelled
Install Native E2E / install-native (macos-latest) (push) Has been cancelled
Wrap Native E2E / wrap-native (macos-latest) (push) Has been cancelled
CI / build-wheel-windows (push) Has been cancelled
rust / simulator e2e (windows-latest) (push) Has been cancelled
rust / wheels (aarch64-apple-darwin) (push) Has been cancelled
rust / wheels (x86_64-apple-darwin) (push) Has been cancelled
Docker / docker-manifest (map[bake_target:runtime name:]) (push) Has been cancelled
Docker / docker-manifest (map[bake_target:runtime-code name:code]) (push) Has been cancelled
Docker / docker-manifest (map[bake_target:runtime-code-nonroot name:code-nonroot]) (push) Has been cancelled
Docker / docker-manifest (map[bake_target:runtime-code-slim name:code-slim]) (push) Has been cancelled
Docker / docker-manifest (map[bake_target:runtime-code-slim-nonroot name:code-slim-nonroot]) (push) Has been cancelled
Docker / docker-manifest (map[bake_target:runtime-nonroot name:nonroot]) (push) Has been cancelled
Docker / docker-manifest (map[bake_target:runtime-slim name:slim]) (push) Has been cancelled
Docker / docker-manifest (map[bake_target:runtime-slim-nonroot name:slim-nonroot]) (push) Has been cancelled
Docker / promote-latest (push) Has been cancelled
fix(litellm): don't forward a caller key the target cannot accept (#2883)
Split out of #2852 at review request: that PR is bounded upstream calls
plus
measured hot-path costs, and this is an authentication/routing change
that
belongs on its own scope. #2852 now carries only the timeout work.

## The bug

A routing extension can rewrite the model across families mid-request
(`claude-opus-5` → `gpt-5-mini`). The caller's key does not travel with
that
rewrite, so the proxy forwards `sk-ant-...` to OpenAI and earns a
guaranteed
401. Downstream that is indistinguishable from *"the cheap model failed
the
task"* — it scores as a quality regression against the router, not as a
bug.

Dropping the `api_key` kwarg instead lets litellm fall back to the
target
provider's own env credential, which is the only key that can work.

## Why this cut is different from the one that was rejected

The first version returned `not provider.startswith("anthropic")`, so
**any**
non-`sk-ant-` credential was dropped against an Anthropic-class target —
a
plain Bearer token against an Anthropic-compatible or custom gateway
lost its
key and fell back to an env credential that may not exist.

That direction is the dangerous one. A false refusal breaks a deployment
that
was working; a missed refusal just leaves today's 401. So this refuses
on
**positive evidence only**:

| credential | target | forwarded? |
|---|---|---|
| `sk-ant-…` | `openai` / `azure` / `gemini` | **no** — cannot possibly
authenticate |
| `sk-ant-…` | anthropic | yes |
| `sk-ant-…` | unrecognised / unclassifiable model | yes — pass-through
|
| anything else | anything | yes — pass-through, unchanged |

`sk-ant-` is Anthropic's documented vendor-specific prefix, which is
what makes
it classifiable. `sk-` is not: a dozen vendors mint that shape.
Everything the
string cannot settle keeps main's behaviour.

The reject list is explicit rather than inverted (`not anthropic`)
because an
unrecognised provider is usually a compatible or self-hosted gateway.
Marked in
the code as a hand-kept tuple with the registry-lookup upgrade path
noted.

Bedrock / Vertex / SageMaker are unaffected — all four dispatch sites
already
skip credential forwarding for them entirely (env-based auth).

## Verification

`tests/test_litellm_caller_key.py`, 12 cases — the refusal, the
Anthropic
target, the unknown provider, `get_llm_provider` raising, and each
unclassifiable credential shape asserted against **both** target
families.
Those last ones fail against the rejected version.

Applied at all four dispatch sites (Anthropic non-stream/stream, OpenAI
non-stream/stream).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 20:12:41 -07:00
..
audit fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
backends fix(litellm): don't forward a caller key the target cannot accept (#2883) 2026-08-09 20:12:41 -07:00
cache fix(ccr): stop persisting retrieval markers as original content (#2694) (#2703) 2026-08-02 13:10:41 -07:00
capture fix(cli): harden all CLI surfaces + fix docs accuracy (#1491) 2026-06-27 14:48:43 -07:00
ccr fix(telemetry): anonymous compression stats — no prompts, no data (#2728) 2026-08-03 05:43:25 -07:00
cli fix(wrap): stop the launch cwd from shadowing the installed package in the proxy subprocess (#2843) 2026-08-08 01:31:40 -05:00
compression fix(code): parse-probe tree-sitter availability in code_handler (#1231) (#1300) 2026-07-09 14:06:29 -05:00
dashboard fix(stats): report one "Tokens Saved" headline across every harness (#2737) 2026-08-03 09:03:54 -07:00
evals feat(evals): weekly HotpotQA answer-recall report on the prose path (#1188) 2026-07-15 21:40:55 +00:00
graph refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) 2026-07-22 20:59:24 -07:00
image fix(image): reuse image models instead of rebuilding them per request (#2513) (#2536) 2026-07-26 07:33:47 -07:00
install fix(install): use --userns=keep-id under Podman so bind-mount writes don't fail (#2846) 2026-08-08 01:30:48 -05:00
integrations feat(beacon): hourly R2 compaction, per-strategy savings, and a stack that reports (#2853) 2026-08-09 15:42:12 -07:00
learn fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
mcp_registry refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) 2026-07-22 20:59:24 -07:00
memory fix(memory): bound the TrafficLearner pending-pattern accumulator (memory leak) (#2579) 2026-08-06 19:21:59 -07:00
models fix(models): version-boundary longest-prefix match in ModelRegistry.get (#1658) 2026-07-10 23:07:31 -05:00
observability test: add fluent Headroom harness (#2650) 2026-07-29 09:17:25 -07:00
perf fix(stats): report one "Tokens Saved" headline across every harness (#2737) 2026-08-03 09:03:54 -07:00
pricing feat(cli,pricing): add CLI extension seam and prompt-cache TTL pricing (#2802) 2026-08-05 12:24:49 -07:00
providers perf: cut hot-path latency 27% (token-count memo, startup preloads, JSON scan memo) (#2838) 2026-08-06 17:47:40 -07:00
proxy perf(proxy): bound upstream calls and hot-path costs (#2852) 2026-08-09 16:24:33 -07:00
relevance feat(security): pilot hardening — stateless guarantee, model pinning, CI security gate (#1515) 2026-06-27 17:44:10 -07:00
reporting feat: attribute reread waste to over-compression via marker check (#901) 2026-06-13 10:43:35 -05:00
storage Fix 19 test failures: missing security attr, nosec inside f-strings, stale test mock 2026-04-07 18:30:29 -07:00
subscription perf(subscription): skip transcripts older than the window in compute_window_tokens (#2861) 2026-08-08 09:23:25 -07:00
telemetry feat(beacon): hourly R2 compaction, per-strategy savings, and a stack that reports (#2853) 2026-08-09 15:42:12 -07:00
testing test: add fluent Headroom harness (#2650) 2026-07-29 09:17:25 -07:00
tokenizers perf: cut hot-path latency 27% (token-count memo, startup preloads, JSON scan memo) (#2838) 2026-08-06 17:47:40 -07:00
transforms feat(beacon): hourly R2 compaction, per-strategy savings, and a stack that reports (#2853) 2026-08-09 15:42:12 -07:00
__init__.py Pin ORT dylib on Windows; init Python logging (#1010) 2026-06-23 07:46:24 -05:00
_ort.py fix(core): load ONNX Runtime dynamically so headroom._core imports on non-AVX2 x86-64 (#1715) 2026-07-14 13:25:41 -04:00
_subprocess.py ci: repair mypy no-any-return in _win32_pid_alive (#1556 follow-up) (#2336) 2026-07-16 20:55:17 -07:00
_version.py fix(version): mark source-checkout builds as -dev (#2072) 2026-07-13 09:38:17 -04:00
agent_savings.py fix(savings): coding profile compresses the recent delta (protect_recent 2->0, min_tokens 25->10) (#2145) 2026-07-14 04:07:25 -04:00
binaries.py fix(windows): pin UTF-8 encoding on text-mode subprocess calls (#1311) 2026-06-23 12:52:49 -05:00
cli.py Add Click-based CLI with memory management commands 2026-01-29 21:30:21 -08:00
client.py refactor: DRY cache logic, add thread safety, fix Bash exclusion (#704) 2026-06-16 14:50:04 -05:00
compress.py feat(compress): expose frozen_message_count in library-mode compress() (#2178) 2026-07-14 16:07:21 -04:00
config.py fix(proxy): unwrap Hermes tool_call bridge in tool name map (#2717) 2026-08-04 22:16:46 -05:00
context_tool_cleanup.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
copilot_auth.py feat(copilot): proxy VS Code models transparently (#2687) 2026-08-03 04:42:48 -07:00
copilot_linux_secret.py fix(windows): pin UTF-8 encoding on text-mode subprocess calls (#1311) 2026-06-23 12:52:49 -05:00
copilot_macos_keychain.py fix(windows): pin UTF-8 encoding on text-mode subprocess calls (#1311) 2026-06-23 12:52:49 -05:00
exceptions.py fix: B1 — retire ICM, RollingWindow, scoring, relevance + dependents 2026-05-02 12:23:17 -07:00
fsutil.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
hooks.py feat: introduce canonical pipeline lifecycle contract 2026-04-21 23:54:28 -05:00
offline.py feat(proxy): pilot hardening — inbound auth, security headers, audit log, air-gap switch (#1537) 2026-06-28 12:09:00 -07:00
onnx_runtime.py fix(onnx): stop ONNX thread pools from spinning idle cores (#2495) (#2540) 2026-08-08 01:33:57 -05:00
parser.py fix(agno): tolerate streaming tool-call SDK objects in parser (#1312) (#1336) 2026-06-24 09:52:15 -05:00
paths.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
pipeline.py feat: introduce canonical pipeline lifecycle contract 2026-04-21 23:54:28 -05:00
py.typed Prepare for OSS release v0.2.0 2026-01-07 11:36:44 -08:00
release_version.py ci(release): publish win_amd64 wheel so Windows installs need no Rust (#1328) (#1335) 2026-06-24 09:48:37 -05:00
savings_ledger.py fix(savings): don't bill free models at the $3/M fallback in the ledger (#2147) 2026-07-14 12:19:30 -04:00
settings_store.py fix(proxy/cost): mark estimated-basis budget records and add an enforcement policy (#2713) (#2725) 2026-08-02 23:05:44 -07:00
shared_context.py fix(shared_context): don't evict an unrelated entry on an update at capacity (#2136) 2026-07-13 19:57:32 -04:00
tokenizer.py Initial commit: Headroom SDK - LLM context optimization toolkit 2026-01-06 23:16:58 -08:00
tools.json fix(review): address PR #210 feedback — race, query-params, SHA logging, frozen prefix, etc 2026-04-20 17:36:44 -07:00
update_check.py feat(proxy): pilot hardening — inbound auth, security headers, audit log, air-gap switch (#1537) 2026-06-28 12:09:00 -07:00
utils.py chore: add nosec B324 annotations to non-cryptographic MD5 usages and update temporary database path to use system temp directory 2026-04-07 13:07:26 +06:00