headroom/tests/test_proxy_copilot_auth_hooks.py
Rod Boev 5af5e22862
Some checks failed
CI / commitlint (push) Failing after 3s
CI / changes (push) Failing after 42s
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime name:]) (push) Waiting to run
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-code name:code]) (push) Waiting to run
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-code-nonroot name:code-nonroot]) (push) Waiting to run
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-code-slim name:code-slim]) (push) Waiting to run
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-code-slim-nonroot name:code-slim-nonroot]) (push) Waiting to run
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-nonroot name:nonroot]) (push) Waiting to run
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-slim name:slim]) (push) Waiting to run
Docker / docker-build (map[name:amd64 platform:linux/amd64 runs_on:ubuntu-24.04], map[bake_target:runtime-slim-nonroot name:slim-nonroot]) (push) Waiting to run
CI / lint (push) Failing after 4s
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime name:]) (push) Waiting to run
CI / build-wheel (push) Failing after 3s
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-code name:code]) (push) Waiting to run
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-code-nonroot name:code-nonroot]) (push) Waiting to run
CI / prefetch-model (push) Failing after 3s
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-code-slim name:code-slim]) (push) Waiting to run
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-code-slim-nonroot name:code-slim-nonroot]) (push) Waiting to run
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-nonroot name:nonroot]) (push) Waiting to run
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-slim name:slim]) (push) Waiting to run
Docker / docker-build (map[name:arm64 platform:linux/arm64 runs_on:ubuntu-24.04-arm], map[bake_target:runtime-slim-nonroot name:slim-nonroot]) (push) Waiting to run
Docker / docker-manifest (map[bake_target:runtime name:]) (push) Blocked by required conditions
CI / build (push) Failing after 9s
Docker / docker-manifest (map[bake_target:runtime-code name:code]) (push) Blocked by required conditions
Docker / docker-manifest (map[bake_target:runtime-code-nonroot name:code-nonroot]) (push) Blocked by required conditions
Docker / docker-manifest (map[bake_target:runtime-code-slim name:code-slim]) (push) Blocked by required conditions
Docker / docker-manifest (map[bake_target:runtime-code-slim-nonroot name:code-slim-nonroot]) (push) Blocked by required conditions
Docker / docker-manifest (map[bake_target:runtime-nonroot name:nonroot]) (push) Blocked by required conditions
CI / workflow-validation (push) Failing after 5s
Docker / docker-manifest (map[bake_target:runtime-slim name:slim]) (push) Blocked by required conditions
CI / docker-native-e2e (push) Failing after 4s
Docker / docker-manifest (map[bake_target:runtime-slim-nonroot name:slim-nonroot]) (push) Blocked by required conditions
CI / test-extras (push) Failing after 5s
Docker / promote-latest (push) Blocked by required conditions
CI / test (3) (push) Failing after 4s
CI / test (4) (push) Failing after 3s
Deploy Documentation / validate (push) Failing after 3s
Init E2E / docker-init-e2e (push) Failing after 8s
Init Native E2E / init-native (ubuntu-latest, claude) (push) Failing after 10s
Install Native E2E / install-native (ubuntu-latest) (push) Failing after 8s
Init Native E2E / init-native (ubuntu-latest, copilot) (push) Failing after 8s
Merge Conflicts / merge-conflicts (push) Failing after 6s
Release Please / release-please (push) Failing after 8s
Security / CodeQL (javascript-typescript) (push) Failing after 39s
Security / Dependency audit (pip-audit) (push) Failing after 1m8s
Security / CodeQL (python) (push) Failing after 53s
Wrap E2E / docker-wrap-e2e (push) Failing after 11s
Wrap Native E2E / wrap-native (ubuntu-latest) (push) Failing after 9s
Security / Secret scan (gitleaks) (push) Failing after 43s
Init Native E2E / init-native (ubuntu-latest, codex) (push) Failing after 12m54s
Deploy Documentation / deploy (push) Failing after 13m1s
CI / test (2) (push) Failing after 13m14s
CI / test (1) (push) Failing after 13m19s
CI / test-dashboard-ui (push) Failing after 13m21s
CI / test-agno (push) Failing after 13m23s
CI / windows-native-wrapper (push) Has been cancelled
CI / macos-native-wrapper (push) Has been cancelled
Init Native E2E / init-native (macos-latest, claude) (push) Has been cancelled
Init Native E2E / init-native (macos-latest, codex) (push) Has been cancelled
Init Native E2E / init-native (macos-latest, copilot) (push) Has been cancelled
Install Native E2E / install-native (macos-latest) (push) Has been cancelled
Wrap Native E2E / wrap-native (macos-latest) (push) Has been cancelled
fix(copilot): route mixed-model requests per model (#1785)
## Description

Copilot subscription sessions can mix a chat-completions main model with
a Responses-only internal bootstrap model. The wrapper still seeds one
`COPILOT_PROVIDER_WIRE_API` value for launch-time compatibility, but the
proxy now chooses the Copilot upstream path per request model inside
OpenAI chat dispatch. That keeps `gpt-5.4-mini` on `/responses` while
`claude-sonnet-5` stays on `/chat/completions`.

Closes #1745

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added a narrow OpenAI chat-handler path resolver that reuses the
existing Copilot model heuristic and only switches Copilot-hosted
requests to `/responses` when the model already prefers Responses.
- Threaded that resolved path through the OpenAI chat handler so request
logs, cache hits, and upstream dispatch all reflect the actual
per-request route.
- Added a regression test that captures the upstream URL for
`gpt-5.4-mini`, preserves the `claude-sonnet-5` control case, and keeps
the non-Copilot control case on chat completions through the same path
resolver composition used by the handler.
- Left the Copilot launch wrapper behavior intact, so the existing
subscription env defaults still serialize the same way at launch.
- Preserved the existing invalid/custom upstream base URL fallback
behavior while applying the Copilot-only per-model route switch.

## Testing

- [x] Unit tests pass (`uv run pytest
tests/test_proxy_copilot_auth_hooks.py
tests/test_cli/test_wrap_copilot.py
tests/test_proxy/test_openai_transport_path_prefix.py
tests/test_proxy/test_openai_upstream_header.py -q`)
- [x] Linting passes (`uv run ruff check
headroom/proxy/handlers/openai.py headroom/cli/wrap.py
tests/test_proxy_copilot_auth_hooks.py
tests/test_cli/test_wrap_copilot.py
tests/test_proxy/test_openai_transport_path_prefix.py
tests/test_proxy/test_openai_upstream_header.py`; `uv run ruff format
--check headroom/proxy/handlers/openai.py
tests/test_proxy_copilot_auth_hooks.py`)
- [ ] Type checking passes (`uv run mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
collected 44 items

tests\test_proxy_copilot_auth_hooks.py ...                               [  6%]
tests\test_cli\test_wrap_copilot.py ..............................       [ 75%]
tests\test_proxy\test_openai_transport_path_prefix.py .......            [ 90%]
tests\test_proxy\test_openai_upstream_header.py ....                     [100%]

======================== 44 passed, 1 warning in 1.38s ========================
All checks passed!
2 files already formatted
```

## Real Behavior Proof

- Environment: Windows, Python 3.12.13, focused local proxy tests.
- Exact command / steps: `uv run pytest
tests/test_proxy_copilot_auth_hooks.py
tests/test_cli/test_wrap_copilot.py
tests/test_proxy/test_openai_transport_path_prefix.py
tests/test_proxy/test_openai_upstream_header.py -q`, then `uv run ruff
check headroom/proxy/handlers/openai.py headroom/cli/wrap.py
tests/test_proxy_copilot_auth_hooks.py
tests/test_cli/test_wrap_copilot.py
tests/test_proxy/test_openai_transport_path_prefix.py
tests/test_proxy/test_openai_upstream_header.py`, then `uv run ruff
format --check headroom/proxy/handlers/openai.py
tests/test_proxy_copilot_auth_hooks.py`.
- Observed result: the new proxy regression test saw
`https://api.githubcopilot.com/responses` for `gpt-5.4-mini` and
`https://api.githubcopilot.com/chat/completions` for `claude-sonnet-5`;
the non-Copilot control stayed on `/v1/chat/completions`, invalid base
URL fallbacks kept the configured OpenAI `/v1` route, and the wrap
regression tests still passed unchanged.
- Not tested: live GitHub Copilot subscription traffic and the rest of
the suite.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

`CHANGELOG.md` stays unchecked because Headroom generates release notes
from conventional commits, not manual edits, and this patch preserves
the existing launch flags while changing only the runtime route
decision.

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-07-08 09:42:37 -05:00

269 lines
9.3 KiB
Python

from __future__ import annotations
import asyncio
import importlib.util
import sys
import types
from pathlib import Path
from types import SimpleNamespace
import pytest
ROOT = Path(__file__).resolve().parents[1]
_ISOLATED_MODULE_NAMES = (
"headroom.proxy",
"headroom.proxy.handlers",
"httpx",
"fastapi.responses",
"tests.headroom_proxy_handlers_openai",
"tests.headroom_proxy_handlers_streaming",
)
@pytest.fixture(autouse=True)
def restore_isolated_modules() -> None:
saved_modules = {name: sys.modules.get(name) for name in _ISOLATED_MODULE_NAMES}
try:
yield
finally:
for name in _ISOLATED_MODULE_NAMES:
sys.modules.pop(name, None)
for name, module in saved_modules.items():
if module is not None:
sys.modules[name] = module
def _load_handler_module(monkeypatch: pytest.MonkeyPatch, module_name: str, relative_path: str):
proxy_pkg = types.ModuleType("headroom.proxy")
proxy_pkg.__path__ = [str(ROOT / "headroom" / "proxy")]
monkeypatch.setitem(sys.modules, "headroom.proxy", proxy_pkg)
handlers_pkg = types.ModuleType("headroom.proxy.handlers")
handlers_pkg.__path__ = [str(ROOT / "headroom" / "proxy" / "handlers")]
monkeypatch.setitem(sys.modules, "headroom.proxy.handlers", handlers_pkg)
httpx_mod = types.ModuleType("httpx")
httpx_mod.ConnectError = type("ConnectError", (Exception,), {})
httpx_mod.ConnectTimeout = type("ConnectTimeout", (Exception,), {})
httpx_mod.PoolTimeout = type("PoolTimeout", (Exception,), {})
httpx_mod.ReadTimeout = type("ReadTimeout", (Exception,), {})
monkeypatch.setitem(sys.modules, "httpx", httpx_mod)
responses_mod = types.ModuleType("fastapi.responses")
class Response:
def __init__(self, content=None, status_code: int = 200, headers=None, media_type=None):
self.content = content
self.status_code = status_code
self.headers = headers or {}
self.media_type = media_type
class StreamingResponse(Response):
pass
class JSONResponse(Response):
pass
responses_mod.Response = Response
responses_mod.StreamingResponse = StreamingResponse
responses_mod.JSONResponse = JSONResponse
monkeypatch.setitem(sys.modules, "fastapi.responses", responses_mod)
spec = importlib.util.spec_from_file_location(module_name, ROOT / relative_path)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
monkeypatch.setitem(sys.modules, module_name, module)
spec.loader.exec_module(module)
return module
def test_openai_passthrough_applies_copilot_auth(monkeypatch: pytest.MonkeyPatch) -> None:
openai_mod = _load_handler_module(
monkeypatch,
"tests.headroom_proxy_handlers_openai",
"headroom/proxy/handlers/openai.py",
)
seen: dict[str, object] = {}
async def fake_apply(headers: dict[str, str], *, url: str) -> dict[str, str]:
seen["headers"] = dict(headers)
seen["url"] = url
return {"Authorization": "Bearer upstream-token"}
monkeypatch.setattr(openai_mod, "apply_copilot_api_auth", fake_apply)
class Dummy(openai_mod.OpenAIHandlerMixin):
def __init__(self) -> None:
self.metrics = SimpleNamespace(record_request=self._record_request)
self.http_client = SimpleNamespace(request=self._request)
self.cost_tracker = None
self._counter = 0
async def _record_request(self, **kwargs) -> None: # noqa: ANN003
return None
async def _next_request_id(self) -> str:
# The passthrough handler now allocates a request_id at end-
# of-call because it records via ``_record_request_outcome``,
# which requires one. Pre-refactor the dummy didn't need
# this method because metrics.record_request was called
# directly without a request_id.
self._counter += 1
return f"req-{self._counter}"
async def _record_request_outcome(self, outcome) -> None: # noqa: ANN001
from headroom.proxy.outcome import emit_request_outcome
await emit_request_outcome(self, outcome)
def _extract_tags(self, headers: dict) -> dict[str, str]:
# Mirror of HeadroomProxy._extract_tags. The passthrough
# handler now extracts tags at entry as part of the
# outcome-tag invariant lock (PR #480).
return {
k.lower().replace("x-headroom-", ""): v
for k, v in headers.items()
if k.lower().startswith("x-headroom-")
}
async def _request(self, **kwargs): # noqa: ANN003
seen["request_kwargs"] = kwargs
return SimpleNamespace(headers={}, content=b"{}", status_code=200)
request = SimpleNamespace(
url=SimpleNamespace(path="/v1/models", query=""),
headers={
"authorization": "Bearer downstream",
"host": "localhost",
"accept-encoding": "gzip",
},
method="GET",
body=lambda: None,
)
async def body() -> bytes:
return b""
request.body = body
handler = Dummy()
response = asyncio.run(
handler.handle_passthrough(
request,
"https://api.githubcopilot.com",
"models",
"openai",
)
)
assert seen["url"] == "https://api.githubcopilot.com/models"
assert seen["request_kwargs"]["headers"] == {"Authorization": "Bearer upstream-token"}
assert response.status_code == 200
def test_streaming_response_applies_copilot_auth(monkeypatch: pytest.MonkeyPatch) -> None:
streaming_mod = _load_handler_module(
monkeypatch,
"tests.headroom_proxy_handlers_streaming",
"headroom/proxy/handlers/streaming.py",
)
seen: dict[str, object] = {}
async def fake_apply(headers: dict[str, str], *, url: str) -> dict[str, str]:
seen["headers"] = dict(headers)
seen["url"] = url
return {"Authorization": "Bearer upstream-token"}
monkeypatch.setattr(streaming_mod, "apply_copilot_api_auth", fake_apply)
class Dummy(streaming_mod.StreamingMixin):
def __init__(self) -> None:
self.memory_handler = None
self.config = SimpleNamespace(
retry_max_attempts=1,
retry_base_delay_ms=1,
retry_max_delay_ms=1,
)
self.http_client = SimpleNamespace(
build_request=self._build_request,
send=self._send,
)
def _build_request(self, method: str, url: str, **kwargs): # noqa: ANN003
# PR-A3: streaming forwarder is byte-faithful; it now passes
# ``content=<bytes>`` instead of ``json=<dict>``.
seen["request"] = {
"method": method,
"url": url,
**kwargs,
}
return SimpleNamespace()
async def _send(self, request, stream: bool): # noqa: ANN001, ANN003
return SimpleNamespace(headers={}, status_code=200)
handler = Dummy()
response = asyncio.run(
handler._stream_response(
url="https://api.githubcopilot.com/v1/responses",
headers={"authorization": "Bearer downstream"},
body={"model": "gpt-4o"},
provider="openai",
model="gpt-4o",
request_id="req-test",
original_tokens=0,
optimized_tokens=0,
tokens_saved=0,
transforms_applied=[],
tags={},
optimization_latency=0.0,
)
)
assert seen["url"] == "https://api.githubcopilot.com/v1/responses"
# PR-A3: byte-faithful forwarder always sets ``content-type`` explicitly.
sent_headers = seen["request"]["headers"]
assert sent_headers["Authorization"] == "Bearer upstream-token"
assert sent_headers["content-type"] == "application/json"
assert response.status_code == 200
def test_openai_chat_routes_copilot_requests_per_model(monkeypatch: pytest.MonkeyPatch) -> None:
openai_mod = _load_handler_module(
monkeypatch,
"tests.headroom_proxy_handlers_openai",
"headroom/proxy/handlers/openai.py",
)
copilot_base = "https://api.githubcopilot.com"
gpt54_mini_url = openai_mod.build_copilot_upstream_url(
copilot_base,
openai_mod._resolve_openai_handler_path(
{},
handler_path=openai_mod._resolve_openai_chat_handler_path(copilot_base, "gpt-5.4-mini"),
),
)
claude_url = openai_mod.build_copilot_upstream_url(
copilot_base,
openai_mod._resolve_openai_handler_path(
{},
handler_path=openai_mod._resolve_openai_chat_handler_path(
copilot_base, "claude-sonnet-5"
),
),
)
openai_url = openai_mod.build_copilot_upstream_url(
"https://api.openai.com",
openai_mod._resolve_openai_handler_path(
{},
handler_path=openai_mod._resolve_openai_chat_handler_path(
"https://api.openai.com", "gpt-5.4-mini"
),
),
)
assert gpt54_mini_url == "https://api.githubcopilot.com/responses"
assert claude_url == "https://api.githubcopilot.com/chat/completions"
assert openai_url == "https://api.openai.com/v1/chat/completions"