mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
## Description
Reported from a Copilot CLI session:
```
[CopilotCLISession] Failed to fetch models: Error: 401 "unauthorized:
unable to validate HMAC for the given Copilot-Integration-ID"
[CopilotCLISession] Proxy URL configured (authType=hmac), skipping
client-side token validation
```
GitHub **binds a Copilot API token to the `Copilot-Integration-Id` it
was minted under** and verifies the pairing with an HMAC. Present a
token minted for integration A alongside a header naming integration B,
and you get exactly this error.
`apply_copilot_api_auth` applied the integration ID with *set-default*
semantics — `_set_header_default` returns early when the header is
already present — **before** deciding whose token to use:
```python
for name, value in _copilot_chat_header_defaults().items():
_set_header_default(resolved, name, value) # ← never overwrites
...
if incoming_auth and _is_forwardable_copilot_bearer_token(...):
return resolved # client's token kept
...
token = await get_copilot_token_provider().get_api_token() # ← REPLACED
```
The client always sends an ID, so when Headroom replaced the token — the
common case, logged as `incoming token not suitable (kind=unknown), will
replace` — the request left carrying **the client's integration ID next
to Headroom's token**, minted under `vscode-chat` via
`_copilot_token_exchange_headers`. A Copilot CLI session does not
identify as `vscode-chat`.
The second log line is why nothing caught it sooner: seeing a proxy URL,
the Copilot client reports `authType=hmac` and **skips its own token
validation**, deferring to the proxy. Nobody validates the pairing until
GitHub rejects it.
**Why this matters beyond one 401:** the failing call is *model
discovery*. When it fails the client falls back to its built-in model
list — which is why a user's selected model never appeared in telemetry
and all traffic surfaced as `gpt-4o-mini`.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
Restores one invariant: **the credential and the integration ID leave
together.**
- **Mint under the client's ID** rather than the proxy's default, so
GitHub's usage attribution keeps pointing at the surface that actually
made the call.
- **Overwrite the forwarded header to match what we minted** — but only
on the replace path. The pass-through branch returns earlier and keeps
the client's own ID beside the client's own token, which is equally a
matched pair.
- **Key the token cache by integration ID.** A single slot would hand a
`vscode-chat` token to a CLI session and reproduce the same 401 straight
from cache.
Two existing contracts deliberately preserved:
- Resolution order is **client header > `GITHUB_COPILOT_INTEGRATION_ID`
> built-in default**. The env var configures the *default* this proxy
sends; it does not override a client that stated its own identity.
Pinned by the existing
`test_apply_copilot_api_auth_preserves_existing_copilot_headers` (whose
fixture literally names the value `should-not-override`).
- The overwrite writes through the client's **existing key**, so a
lowercase `copilot-integration-id` does not gain a second capitalised
variant beside it — pinned by the existing
`..._preserves_existing_headers_case_insensitively`.
Existing test stubs for `get_api_token` gained the new keyword — the
same signature-drift hazard this repo just hit in
`RemoteKompressCompressor` (#3162).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ pytest tests/ -q -k copilot
338 passed, 8 skipped
$ pytest tests/ -q # this branch
6 failed, 11386 passed, 587 skipped in 425.40s
All 6 also fail on clean origin/main, same machine — pre-existing, not regressions:
test_graceful_shutdown.py::test_run_server_installs_cancelled_error_filter
test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline
test_release_workflows.py::test_no_native_tls_in_wheel_build_tree
test_providers/test_deepseek.py::... (3 litellm pricing tests)
$ ruff check headroom/
All checks passed!
$ mypy headroom/copilot_auth.py
0 errors
```
12 new tests: the mint/forward pairing, the pass-through branch keeping
the client's pair untouched, no duplicate case-variant header,
resolution order in both directions, blank/absent client values,
non-Copilot upstreams untouched, and per-integration cache isolation.
## Real Behavior Proof
- **Environment:** macOS, Python 3.12.13, branch on `origin/main` @
`a3821378`.
- **Exact command / steps:** drove `apply_copilot_api_auth` with the
reported shape — an unusable client bearer plus `Copilot-Integration-Id:
copilot-cli-chat` against `api.githubcopilot.com` — and compared the ID
the token would be **minted under** (via
`_copilot_token_exchange_headers`) against the ID actually
**forwarded**. Run against the same script before and after the change,
with `PYTHONPATH` pinned to the worktree.
- **Observed result:**
```
########## PRE-FIX ##########
token minted under : vscode-chat
header forwarded : copilot-cli-chat
-> GitHub would REJECT (401 HMAC)
########## POST-FIX ##########
token minted under : copilot-cli-chat
header forwarded : copilot-cli-chat
-> GitHub would ACCEPT
```
- **Not tested:** no live call to GitHub's CAPI — the HMAC is validated
server-side by GitHub and cannot be exercised offline. The claim
verified here is that the two halves now agree; that GitHub accepts a
correctly-paired credential is inferred from its error message, not
observed. **Worth one live Copilot CLI run before shipping to a
reporter.** The `GITHUB_COPILOT_API_TOKEN` path is also unchanged: an
externally-supplied token was minted under an integration this proxy
cannot know, so it is passed through as before.
## Runtime Rollout Safety
- **Rollout-managed feature(s):** none.
- **Minimum rollout channel:** n/a.
- **Stable/default behavior changed:** requests where Headroom replaces
the token now forward the integration ID the replacement was minted
under. For a client sending `vscode-chat` (VS Code, the previous
default) nothing changes at all — the resolved value is identical.
- **Kill switch / disable path:** setting
`GITHUB_COPILOT_INTEGRATION_ID` pins the value used for clients that
send none; clients that send one are unaffected either way.
- **Unsafe override required:** none.
- **Qualification impact:** model discovery should stop 401ing for
non-VS-Code Copilot surfaces, which restores the real model list.
- **Rollback path:** revert the commit; behavior returns to minting
under `vscode-chat` regardless of caller.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
231 lines
8.8 KiB
Python
231 lines
8.8 KiB
Python
"""A Copilot token and its integration ID must leave together.
|
|
|
|
GitHub binds a Copilot API token to the ``Copilot-Integration-Id`` it was
|
|
minted under and verifies the pairing with an HMAC. Presenting a token minted
|
|
for one integration alongside a header naming another fails with:
|
|
|
|
401 unauthorized: unable to validate HMAC for the given
|
|
Copilot-Integration-ID
|
|
|
|
Reported from a Copilot CLI session:
|
|
|
|
[CopilotCLISession] Failed to fetch models: Error: 401 "unauthorized:
|
|
unable to validate HMAC for the given Copilot-Integration-ID"
|
|
[CopilotCLISession] Proxy URL configured (authType=hmac), skipping
|
|
client-side token validation
|
|
|
|
``apply_copilot_api_auth`` applied the integration ID with *set-default*
|
|
semantics (``_set_header_default`` returns early when the header is already
|
|
present) BEFORE deciding whose token to use. The client always sends one, so
|
|
when Headroom replaced the token — the common case, logged as ``incoming token
|
|
not suitable (kind=unknown), will replace`` — the request went out carrying the
|
|
CLIENT's integration ID next to HEADROOM's token, minted under ``vscode-chat``.
|
|
|
|
The second log line is why nothing caught it sooner: seeing a proxy URL, the
|
|
Copilot client reports ``authType=hmac`` and skips its own token validation,
|
|
deferring to the proxy. Nobody checks the pairing until GitHub rejects it.
|
|
|
|
The failing call was model discovery, so the client fell back to its built-in
|
|
model list — which is why a user's selected model never appeared in telemetry.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
|
|
import pytest
|
|
|
|
from headroom import copilot_auth
|
|
from headroom.copilot_auth import (
|
|
CopilotAPIToken,
|
|
apply_copilot_api_auth,
|
|
resolve_copilot_integration_id,
|
|
)
|
|
|
|
CAPI = "https://api.githubcopilot.com/chat/completions"
|
|
CLI_ID = "copilot-cli-chat"
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _clean_env(monkeypatch):
|
|
for var in (
|
|
"GITHUB_COPILOT_INTEGRATION_ID",
|
|
"GITHUB_COPILOT_API_TOKEN",
|
|
"GITHUB_COPILOT_REFRESH_OAUTH_TOKEN",
|
|
):
|
|
monkeypatch.delenv(var, raising=False)
|
|
copilot_auth._provider = None
|
|
yield
|
|
copilot_auth._provider = None
|
|
|
|
|
|
class _RecordingProvider:
|
|
"""Stands in for the token provider; records what it was asked to mint."""
|
|
|
|
def __init__(self) -> None:
|
|
self.asked: list[str | None] = []
|
|
|
|
async def get_api_token(self, *, integration_id: str | None = None):
|
|
self.asked.append(integration_id)
|
|
return CopilotAPIToken(
|
|
token=f"minted-for-{integration_id}",
|
|
expires_at=9_999_999_999.0,
|
|
api_url="https://api.githubcopilot.com",
|
|
)
|
|
|
|
|
|
def _install(monkeypatch) -> _RecordingProvider:
|
|
provider = _RecordingProvider()
|
|
monkeypatch.setattr(copilot_auth, "get_copilot_token_provider", lambda: provider)
|
|
return provider
|
|
|
|
|
|
def _apply(headers: dict, url: str = CAPI) -> dict:
|
|
return asyncio.run(apply_copilot_api_auth(headers, url=url))
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# The reported failure
|
|
# --------------------------------------------------------------------------- #
|
|
def test_token_is_minted_under_the_clients_integration_id(monkeypatch) -> None:
|
|
provider = _install(monkeypatch)
|
|
|
|
_apply({"Authorization": "Bearer unusable", "Copilot-Integration-Id": CLI_ID})
|
|
|
|
assert provider.asked == [CLI_ID], (
|
|
"the replacement token must be minted for the surface that made the "
|
|
"call, not for the proxy's default"
|
|
)
|
|
|
|
|
|
def test_forwarded_header_matches_the_minted_token(monkeypatch) -> None:
|
|
"""The invariant. This is what GitHub HMAC-verifies."""
|
|
provider = _install(monkeypatch)
|
|
|
|
out = _apply({"Authorization": "Bearer unusable", "Copilot-Integration-Id": CLI_ID})
|
|
|
|
minted_for = provider.asked[0]
|
|
assert out["Authorization"] == f"Bearer minted-for-{minted_for}"
|
|
assert out["Copilot-Integration-Id"] == minted_for
|
|
|
|
|
|
def test_no_duplicate_integration_id_header_is_emitted(monkeypatch) -> None:
|
|
"""Overwriting must replace the client's casing, not sit beside it."""
|
|
_install(monkeypatch)
|
|
|
|
out = _apply({"Authorization": "Bearer unusable", "copilot-integration-id": CLI_ID})
|
|
|
|
matching = [k for k in out if k.lower() == "copilot-integration-id"]
|
|
assert len(matching) == 1
|
|
# Written through the client's own key, not beside it.
|
|
assert matching[0] == "copilot-integration-id"
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# The pass-through branch keeps the client's own matched pair
|
|
# --------------------------------------------------------------------------- #
|
|
def test_a_forwardable_client_token_keeps_the_clients_id(monkeypatch) -> None:
|
|
"""When we don't replace the credential, we must not touch its pairing."""
|
|
provider = _install(monkeypatch)
|
|
monkeypatch.setattr(copilot_auth, "_is_forwardable_copilot_bearer_token", lambda _t: True)
|
|
monkeypatch.setattr(copilot_auth, "_is_managed_copilot_seeded_bearer", lambda _t: False)
|
|
|
|
out = _apply({"Authorization": "Bearer tid=real;exp=1", "Copilot-Integration-Id": CLI_ID})
|
|
|
|
assert provider.asked == [], "no token should have been minted"
|
|
assert out["Authorization"] == "Bearer tid=real;exp=1"
|
|
assert out["Copilot-Integration-Id"] == CLI_ID
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Resolution order
|
|
# --------------------------------------------------------------------------- #
|
|
def test_a_client_that_states_its_identity_beats_the_configured_default(
|
|
monkeypatch,
|
|
) -> None:
|
|
"""``GITHUB_COPILOT_INTEGRATION_ID`` configures the DEFAULT, it does not
|
|
override a client that named itself — the long-standing contract pinned by
|
|
``test_apply_copilot_api_auth_preserves_existing_copilot_headers``. What
|
|
matters here is that whichever value wins is used for BOTH halves.
|
|
"""
|
|
monkeypatch.setenv("GITHUB_COPILOT_INTEGRATION_ID", "enterprise-shim")
|
|
provider = _install(monkeypatch)
|
|
|
|
out = _apply({"Authorization": "Bearer unusable", "Copilot-Integration-Id": CLI_ID})
|
|
|
|
assert provider.asked == [CLI_ID]
|
|
assert out["Copilot-Integration-Id"] == CLI_ID
|
|
|
|
|
|
def test_the_configured_default_applies_when_the_client_sends_none(
|
|
monkeypatch,
|
|
) -> None:
|
|
monkeypatch.setenv("GITHUB_COPILOT_INTEGRATION_ID", "enterprise-shim")
|
|
provider = _install(monkeypatch)
|
|
|
|
out = _apply({"Authorization": "Bearer unusable"})
|
|
|
|
assert provider.asked == ["enterprise-shim"]
|
|
assert out["Copilot-Integration-Id"] == "enterprise-shim"
|
|
|
|
|
|
def test_client_value_wins_over_the_default(monkeypatch) -> None:
|
|
assert resolve_copilot_integration_id(CLI_ID) == CLI_ID
|
|
|
|
|
|
def test_default_when_the_client_sends_none(monkeypatch) -> None:
|
|
provider = _install(monkeypatch)
|
|
|
|
out = _apply({"Authorization": "Bearer unusable"})
|
|
|
|
assert provider.asked == [copilot_auth._DEFAULT_COPILOT_INTEGRATION_ID]
|
|
assert out["Copilot-Integration-Id"] == copilot_auth._DEFAULT_COPILOT_INTEGRATION_ID
|
|
|
|
|
|
@pytest.mark.parametrize("blank", ["", " ", None])
|
|
def test_blank_client_values_fall_back(blank) -> None:
|
|
assert resolve_copilot_integration_id(blank) == copilot_auth._DEFAULT_COPILOT_INTEGRATION_ID
|
|
|
|
|
|
def test_non_copilot_upstream_is_untouched(monkeypatch) -> None:
|
|
provider = _install(monkeypatch)
|
|
headers = {"Authorization": "Bearer sk-openai", "Copilot-Integration-Id": CLI_ID}
|
|
|
|
out = _apply(dict(headers), url="https://api.openai.com/v1/chat/completions")
|
|
|
|
assert out == headers
|
|
assert provider.asked == []
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# The cache must not hand one integration another's token
|
|
# --------------------------------------------------------------------------- #
|
|
def test_tokens_are_cached_per_integration_id(monkeypatch) -> None:
|
|
from headroom.copilot_auth import CopilotTokenProvider
|
|
|
|
provider = CopilotTokenProvider()
|
|
exchanged: list[str | None] = []
|
|
|
|
async def _fake_exchange(oauth_token, *, integration_id=None): # noqa: ANN001
|
|
exchanged.append(integration_id)
|
|
return CopilotAPIToken(
|
|
token=f"tok-{integration_id}",
|
|
expires_at=9_999_999_999.0,
|
|
api_url="https://api.githubcopilot.com",
|
|
)
|
|
|
|
monkeypatch.setattr(provider, "_exchange_token", _fake_exchange)
|
|
monkeypatch.setattr(copilot_auth, "read_cached_oauth_token", lambda: "oauth")
|
|
monkeypatch.setattr(copilot_auth, "_should_exchange_oauth_token", lambda: True)
|
|
|
|
a = asyncio.run(provider.get_api_token(integration_id="vscode-chat"))
|
|
b = asyncio.run(provider.get_api_token(integration_id=CLI_ID))
|
|
a_again = asyncio.run(provider.get_api_token(integration_id="vscode-chat"))
|
|
|
|
assert a.token == "tok-vscode-chat"
|
|
assert b.token == f"tok-{CLI_ID}"
|
|
# Distinct integrations must not share a slot...
|
|
assert a.token != b.token
|
|
# ...and the same one must still be served from cache.
|
|
assert a_again.token == a.token
|
|
assert exchanged == ["vscode-chat", CLI_ID]
|