akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
2696 commits 410 branches 169 tags 142 MiB
Commit graph

9 commits

Author SHA1 Message Date
Abhay Singh
1e448b5503
fix(providers): route Claude requests to Copilot when the OpenAI target is a Copilot host (#3258) 
## Description

Through `headroom wrap vscode` / `wrap copilot --subscription`, GitHub
Copilot **GPT** models work but **Claude** models fail with `Invalid
bearer token` (issue #3247). The logs tell the story:

```text
# GPT — works:
event=outbound_request  path=https://api.githubcopilot.com/chat/completions  status=200

# Claude — fails:
event=outbound_request  path=https://api.anthropic.com/v1/messages           status=401
```

GitHub Copilot serves **both** surfaces from the same host: its OpenAI
surface (`/chat/completions`, `/responses`) and its Anthropic surface
for Claude models (`/v1/messages`) — `build_copilot_upstream_url`
already documents and handles this. But `resolve_api_targets` resolves
each provider target independently: when the Copilot flow points the
**OpenAI** target at a Copilot host (so GPT works), the **Anthropic**
target is left at its default `https://api.anthropic.com`. Claude-model
requests are therefore forwarded to the real Anthropic API carrying the
GitHub Copilot bearer, which Anthropic rejects with `Invalid bearer
token`.

## Fix

In `resolve_api_targets`, when the resolved OpenAI target is a Copilot
upstream host **and no explicit Anthropic target was configured**,
default the Anthropic target to that same Copilot host. Claude requests
then reach `https://api.githubcopilot.com/v1/messages` — the surface
that serves them, where the Copilot bearer is valid. An explicit
`ANTHROPIC_TARGET_API_URL` always wins (only a `None` override is filled
in), and non-Copilot OpenAI targets are untouched, so direct-Anthropic
setups are unaffected.

Reproduction:

```python
resolve_api_targets(ProviderApiOverrides(openai="https://api.githubcopilot.com", anthropic=None, ...))
# BEFORE: targets.anthropic == "https://api.anthropic.com"   -> Copilot bearer 401s there
# AFTER:  targets.anthropic == "https://api.githubcopilot.com"
```

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/providers/registry.py`: `resolve_api_targets` now fills a
`None` Anthropic override with the OpenAI target when that target is a
Copilot host (`is_copilot_upstream_url`). Explicit overrides and
non-Copilot targets are unchanged.
- `tests/test_provider_registry.py`: added three tests — Copilot OpenAI
target routes Anthropic to Copilot; an explicit Anthropic override wins;
a non-Copilot OpenAI target leaves the Anthropic default alone.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality

### Test Output

```text
tests/test_provider_registry.py tests/test_provider_registry_extended.py tests/test_banner_upstream_targets.py  ->  37 passed in 12.11s
(the new Copilot test FAILS on pre-fix code — verified via git stash)
uvx ruff@0.16.2 check headroom/providers/registry.py tests/test_provider_registry.py  ->  All checks passed!
uvx mypy@1.20.2 headroom/providers/registry.py  ->  Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.16.2 and mypy 1.20.2 via uvx.
- Exact command / steps: `resolve_api_targets` with
`openai="https://api.githubcopilot.com"` (and the
`api.business.githubcopilot.com` variant) and `anthropic=None` returned
`anthropic="https://api.anthropic.com"` before the fix and the Copilot
host after; an explicit `anthropic="https://api.anthropic.com"` is
preserved; `openai="https://api.openai.com"` leaves `anthropic` at the
default.
- Observed result: Claude-model requests now resolve to the Copilot host
that serves them; OpenAI/direct-Anthropic behavior is unchanged.
- Not tested: no live macOS/VS Code Copilot round trip
(environment-specific); the target-resolution seam that decides the
upstream host is exercised directly. `is_copilot_upstream_url` already
recognizes the github.com Copilot hosts (verified).

## Runtime Rollout Safety

- Rollout-managed feature(s): none. This is upstream target resolution
in the provider registry, not a rollout-channel-gated runtime feature.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: only the broken case changes — a
Copilot OpenAI target with no Anthropic override now sends Claude to
Copilot instead of 401ing against api.anthropic.com. Explicit Anthropic
targets and non-Copilot OpenAI targets are byte-for-byte unchanged.
- Kill switch / disable path: set `ANTHROPIC_TARGET_API_URL` explicitly
to opt out of the default.
- Unsafe override required: no.
- Qualification impact: none for non-Copilot deployments.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A:
internal behavior)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

Fixes the routing/auth mismatch at the resolution layer so it applies
uniformly across the Copilot config paths (`wrap vscode`, `wrap copilot
--subscription`) that set the OpenAI target to a Copilot host. If a
specific deploy sets neither target to a Copilot host (relying solely on
path-based passthrough routing for OpenAI), configuring
`ANTHROPIC_TARGET_API_URL` to the Copilot host remains the explicit
escape hatch.
 2026-08-26 22:43:11 +05:30
nangsontay
e5b3a634df
fix(proxy): dedupe Codex WS request logging for accurate mixed-provider dashboards (#2189) 
## Description

Running Claude Code (Anthropic) and Codex (OpenAI) against the **same**
Headroom proxy instance on one port produced incorrect, unstable
dashboard data. The proxy core is provider-isolated and
multi-provider-safe by design; the defect was in the observability
layer. The Codex `/v1/responses` **WebSocket** handler was the only path
in the proxy that wrote to the request logger by hand instead of through
the unified `emit_request_outcome` funnel, and it did so twice per
session close: the per-turn funnel record plus an unconditional
cumulative session-summary `RequestLog`. This PR removes the duplicate
summary log so Codex WS emits exactly one request log per turn, matching
the HTTP provider paths.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Dropped the duplicate cumulative session-summary `RequestLog` in the
Codex WS handler while preserving the per-turn `emit_request_outcome`
path.
- Preserved gated `request_messages` and `turn_id` on residual outcomes
so dashboard telemetry keeps the useful attribution without
double-counting tokens.
- Ensured explicit `--anyllm-provider` wins over a leaked
`HEADROOM_ANYLLM_PROVIDER` environment variable.
- Registered retry delay settings that had drifted out of the settings
registry.
- Hardened tests against developer-shell `HEADROOM_*` /
`ANTHROPIC_CUSTOM_HEADERS` leakage and stabilized several focused
proxy/wrap test fixtures.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/pytest tests/ -q -p no:cacheprovider
8529 passed, 537 skipped, 5831 warnings in 276.25s (0:04:36)

$ .venv/bin/ruff check <touched files>
All checks passed!
```

## Real Behavior Proof

- Environment: macOS (Darwin 25.4.0), Python 3.13.14, pytest 9.0.3, ruff
via project venv, branch `fix/multi-provider-runtime`.
- Exact command / steps: Ran the full test suite without pytest cache
provider and Ruff on all touched files; used `git stash` to confirm the
stale fake-config failures pre-existed this change.
- Observed result: Full suite passed with no failures; Ruff passed;
Codex WS now routes end-of-session logging through
`emit_request_outcome`, emitting one request log per turn with the same
accounting model as Anthropic HTTP turns.
- Not tested: Live simultaneous Claude + Codex dashboard run. `mypy
headroom` was not run to completion; a scoped run reported one
pre-existing `settings_store.py:470` coercion error outside this diff.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A - server-side observability fix; no UI markup changed.

## Additional Notes

- The proxy's multi-provider routing, header/auth isolation, and
per-model cache keying are already correct and unchanged here; only the
WS observability write path was double-counting.
- Architectural assessment:
`plans/reports/research-260714-0004-multi-provider-upstream-compression-report.md`;
root-cause + resolution trail:
`plans/reports/debug-assessment-260714-0011-dashboard-instability-mixed-claude-codex-report.md`.
- No live simultaneous Claude + Codex dashboard run was performed;
validation is from test coverage and code review of the WS logging path.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
 2026-07-15 18:18:34 +00:00
Rod Boev
931eed879d
fix(mcp): surface dead proxy state (#1786) 
## Description

When the configured Headroom proxy is down, the MCP server can still
start cleanly and return successful-looking no-op compression or zeroed
stats. That hides the real failure from the client and makes it look
like Headroom is working while compression has stopped. This change
makes proxy-backed MCP tool paths surface unreachable-proxy state
explicitly instead of silently degrading.

Closes #881

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Detect unreachable configured proxy state before returning
proxy-backed MCP tool results.
- Report proxy-unreachable status for compression and stats instead of
presenting no-op output as healthy.
- Preserve local MCP behavior when proxy checking is disabled or a
local-only tool path is intended.
- Keep the short `/livez` health probe isolated from the shared proxy
client used by retrieval and stats calls.

## Testing

- [x] Unit tests pass (`uv run pytest tests/test_ccr_mcp_server.py
tests/test_provider_registry.py -q`)
- [x] Linting passes (`uv run ruff check headroom/ccr/mcp_server.py
headroom/providers/registry.py tests/test_ccr_mcp_server.py
tests/test_provider_registry.py`; `uv run ruff format --check
headroom/ccr/mcp_server.py headroom/providers/registry.py
tests/test_ccr_mcp_server.py tests/test_provider_registry.py`)
- [ ] Type checking passes (`uv run mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
collected 26 items

tests\test_ccr_mcp_server.py ...s..........                              [ 53%]
tests\test_provider_registry.py ............                             [100%]

======================== 25 passed, 1 skipped in 6.64s ========================
All checks passed!
4 files already formatted
```

## Real Behavior Proof

- Environment: Windows, focused local MCP tests.
- Exact command / steps: Run `uv run pytest
.tmp\headroom_t45_regression.py -q` in the base and head worktrees, then
run `uv run pytest tests/test_ccr_mcp_server.py
tests/test_provider_registry.py -q`, `uv run ruff check
headroom/ccr/mcp_server.py headroom/providers/registry.py
tests/test_ccr_mcp_server.py tests/test_provider_registry.py`, and `uv
run ruff format --check headroom/ccr/mcp_server.py
headroom/providers/registry.py tests/test_ccr_mcp_server.py
tests/test_provider_registry.py` in the head worktree.
- Observed result: `base: KeyError: 'proxy'` on the new
proxy-unreachable assertions, `head: .tmp\headroom_t45_regression.py
.... [100%]`, broader head suite `25 passed, 1 skipped in 6.64s`, and
the proxy health probe regression preserved the shared proxy client used
by retrieval and stats.
- Not tested: The reporter's bundled macOS runtime, live Claude Desktop
MCP logs, and the full test suite.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

Documentation and changelog are left unchecked for now; the behavior
change is an error-surfacing fix for existing MCP tools and Headroom
generates changelog entries from conventional commits.
 2026-07-07 23:18:57 -05:00
Matt Haitana
7d87aa2f1c
fix(bedrock): route ARNs via converse, named AWS profiles, and au. re… (#1456) 
## Description

Fix three related gaps in Bedrock support that prevented headroom from
working with Claude Code when `CLAUDE_CODE_USE_BEDROCK=0` and
`ANTHROPIC_BASE_URL` is pointed at the proxy:

1. **ARN passthrough used the wrong LiteLLM route** — application
inference profile ARNs (e.g.
`arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>`)
were forwarded as `bedrock/<arn>`, which LiteLLM rejects with HTTP 400
"Try calling via converse route". Fixed to `bedrock/converse/<arn>`.

2. **Named AWS profile not forwarded to completion calls** —
`--bedrock-profile` was wired through the CLI → config →
`LiteLLMBackend.__init__` and used to fetch the model map at startup,
but never stored on `self`. All four `acompletion()` call sites
(`send_message`, `stream_message`, `send_openai_message`,
`stream_openai_message`) passed only `aws_region_name` — the
actual Bedrock calls used ambient credentials regardless of the flag.
Fixed by storing `self.profile_name` and passing `aws_profile_name=` to
every `acompletion()` call.

3. **`ap-southeast-2` used the wrong region prefix** — Australia should
use `au.` for cross-region inference profile IDs, not `apac.`. Added
`ap-southeast-2 → "au"` to `_BEDROCK_REGION_PREFIXES` and `"au."` to the
strip list in `_normalize_bedrock_profile_id`.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `backends/litellm.py`: route `arn:aws:` model IDs via
`bedrock/converse/<arn>` in `map_model_id`
- `backends/litellm.py`: store `profile_name` as `self.profile_name` in
`LiteLLMBackend.__init__`; pass `aws_profile_name=` to `acompletion()`
in all four call sites; use
`boto3.Session(profile_name=...)` for startup discovery; cache key is
`region:profile_name` to prevent cross-profile collisions
- `backends/litellm.py`: add `ap-southeast-2 → "au"` to
`_BEDROCK_REGION_PREFIXES`; add `"au."` to prefix strip list in
`_normalize_bedrock_profile_id`
- `providers/registry.py`: pass `profile_name=bedrock_profile` to
`LiteLLMBackend`
- `proxy/server.py`: pass `config.bedrock_profile` to
`create_proxy_backend`
- `docs/claude-code-bedrock-headroom.md`: remove false claim that ARNs
in `ANTHROPIC_DEFAULT_*_MODEL` bypass the proxy; fix troubleshooting
table
- `tests/test_bedrock_region.py`: update `test_arn_passthrough` to
expect `bedrock/converse/<arn>`; update cache key format; add
`test_profile_cache_isolation`,
`test_ap_southeast_2_uses_au_prefix`, and
`TestBedrockProfileForwardedToCompletion` (3 async tests asserting
`aws_profile_name` appears in `acompletion()` kwargs for named profiles
and is
absent for the no-profile case)
- `tests/test_provider_registry*.py`,
`test_vertex_claude_compression.py`: update `litellm_backend_cls` stubs
to accept `profile_name=None`

## Testing

- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_bedrock_region.py tests/test_provider_registry.py tests/test_provider_registry_extended.py \
    -k "not test_fallback_when_boto3_import_fails and not test_fallback_when_api_call_fails and not test_successful_fetch" -q
collected 51 items / 3 deselected / 48 selected

tests/test_bedrock_region.py ...........................
tests/test_provider_registry.py ...........
tests/test_provider_registry_extended.py .......

48 passed, 3 deselected in 2.00s
```

Note: 3 deselected tests use patch("builtins.__import__") which hangs
under Python 3.13 — pre-existing issue unrelated to these changes.

## Real Behavior Proof

- Environment: macOS, Python 3.13, Claude Code with
`CLAUDE_CODE_USE_BEDROCK=0`, `ANTHROPIC_BASE_URL=http://127.0.0.1:8787`,
AWS ap-southeast-2, application inference profile ARNs in
`ANTHROPIC_DEFAULT_*_MODEL`
- Exact command / steps: `headroom proxy --port 8787 --backend bedrock
--region ap-southeast-2 --bedrock-profile "my-sso-profile"`
- Observed result: Requests routed correctly to
`bedrock/converse/arn:aws:bedrock:ap-southeast-2:...:application-inference-profile/<id>`
as confirmed in LiteLLM logs
- Not tested: EU/APAC region ARN passthrough (logic is identical);
non-SSO credential flows

```text
15:29:44 - LiteLLM:INFO: utils.py:4090 - 
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
2026-06-26 15:29:44,322 - LiteLLM - INFO - 
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
15:31:09 - LiteLLM:INFO: utils.py:4090 - 
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
2026-06-26 15:31:09,928 - LiteLLM - INFO - 
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
15:34:26 - LiteLLM:INFO: utils.py:4090 - 
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
2026-06-26 15:34:26,811 - LiteLLM - INFO - 
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
```

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes

## Additional Notes

The 3 skipped tests (`test_fallback_when_boto3_import_fails`,
`test_fallback_when_api_call_fails`, `test_successful_fetch`) pre-exist
in the repo and use `patch("builtins.__import__")` which hangs under
Python 3.13. Not affected by these changes.

---------

Co-authored-by: Matt Haitana <mhaitana@costar.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
 2026-07-02 22:51:05 -05:00
JD Davis
3c77e52ce4
feat: add Vertex AI proxy routing (#793) 
## Description

Adds first-class GCP Vertex AI proxy routing for publisher REST
endpoints so Vertex requests are forwarded to a configurable regional
Vertex host instead of falling through to the generic
OpenAI/Anthropic/Gemini passthrough selection.

Fixes #792

## Type of Change

- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update

## Changes Made

- Added a `vertex` provider target with `VERTEX_TARGET_API_URL` and
`--vertex-api-url` support.
- Registered explicit Vertex publisher routes for Google
`generateContent`, `streamGenerateContent`, `countTokens` and Anthropic
publisher `rawPredict`, `streamRawPredict` passthrough.
- Added startup banner/routing output for Vertex AI.
- Added focused tests for provider target resolution, CLI/env config,
banner output, and route delegation.
- Added `wiki/vertex.md` with usage examples and Google Cloud source
links.

## Sources

- Vertex AI Gemini inference reference:
https://docs.cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference
- Google Cloud REST authentication:
https://docs.cloud.google.com/docs/authentication/rest
- Google Application Default Credentials:
https://docs.cloud.google.com/docs/authentication/application-default-credentials

## Testing

- [x] Linting passes (`python -m ruff check .`)
- [x] New tests added for new functionality
- [x] Focused unit tests pass
- [ ] Full unit suite completed locally
- [ ] Rust tests completed locally
- [ ] Type checking passes locally

## Test Output

```text
$ python -m ruff check .
All checks passed!

$ python -m pytest tests/test_provider_registry.py tests/test_provider_proxy_routes.py tests/test_cli_proxy_env.py tests/test_banner_upstream_targets.py -q
57 passed, 1 warning in 13.75s
```

Local limitations:

- `python -m pytest tests scripts/tests -q` timed out after 1 hour on
this Windows machine before completing.
- `cargo test -p headroom-proxy --test integration_vertex_raw_predict`
could not run because `cargo` is not installed on PATH in this
environment.
- The commit hook's `mypy` step fails locally on an existing Windows
`fcntl` typing issue in `headroom/subscription/tracker.py`; `ruff`,
`ruff-format`, and plugin-version hooks passed, and the commit was made
with only `mypy` skipped.

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] I have added tests that prove the feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
 2026-06-09 23:05:30 -07:00
JerrettDavis
4576f9caba test: remove provider diff churn 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-22 22:13:13 -05:00
JerrettDavis
7831620eca test: expand provider slice coverage 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-22 21:58:03 -05:00
JerrettDavis
5413e7af47 chore: normalize provider slice line endings 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-21 23:54:28 -05:00
JerrettDavis
b17c6d81cc refactor: extract provider logic into slices 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-21 23:54:28 -05:00