headroom/tests/test_cli/test_wrap_vscode.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

86 lines
3.1 KiB
Python
Raw Normal View History

feat(copilot): proxy VS Code models transparently (#2687) ## Description Make Headroom a transparent proxy for VS Code GitHub Copilot. Users keep using Copilot's normal model picker—GPT-4.1, Claude Sonnet, Claude Opus, and other models in their entitlement—while Headroom silently forwards the selected model instead of registering or requiring a separate "Headroom" model. This also fixes GitHub's device OAuth exchange by sending form-encoded request bodies, matching the endpoint contract. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom wrap vscode` to start a Copilot-seeded subscription proxy and safely configure VS Code's shipped Copilot proxy override. - Add `headroom unwrap vscode` for reversible cleanup. - Preserve VS Code's selected model by changing only the proxy URL/auth override; no custom model is registered and no model preference is written. - Support stable VS Code settings locations on macOS, Windows, and Linux, plus `--settings-file` for Insiders, portable, and other installations. - Edit JSONC settings with a marker-owned block while preserving unrelated bytes, comments, ordering, and trailing commas. - Refuse malformed markers, invalid JSONC, or unmanaged existing Copilot overrides instead of overwriting user configuration. - Fix SIGINT cleanup so the managed settings block is removed and normal shutdown exits successfully. - Fix Copilot device OAuth start/poll requests to use `application/x-www-form-urlencoded`. - Add a compatibility matrix, setup/removal flow, credential behavior, remote-development guidance, enterprise notes, troubleshooting, and verification documentation. ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ .venv/bin/pytest -q tests/test_provider_copilot_vscode_config.py tests/test_cli/test_wrap_vscode.py tests/test_cli/test_wrap_helpers.py tests/test_copilot_auth.py tests/test_cli/test_copilot_auth.py tests/test_provider_copilot_wrap.py tests/test_cli/test_wrap_copilot.py tests/test_copilot_provider_label.py tests/test_copilot_subscription_smoke.py 244 passed in 0.59s $ .venv/bin/ruff check <changed Python files and tests> All checks passed! $ .venv/bin/mypy headroom/providers/copilot/vscode.py Success: no issues found in 1 source file $ cd docs && npm run types:check fumadocs-mdx && next typegen && tsc --noEmit # exited 0 $ git diff --check # exited 0 ``` The full 10,179-test suite was also sampled through approximately 83%, but was stopped because of its runtime. It exposed existing failures in `test_recover_codex.py`, `test_wrap_stale_marker.py`, and `test_proxy_health.py`; therefore the broad `pytest`, repository-wide Ruff, and repository-wide mypy boxes are intentionally not checked. ## Real Behavior Proof - Environment: macOS arm64, VS Code 1.131.0, built-in GitHub Copilot 0.59.0, Headroom 0.33.1-dev. - Exact command / steps: 1. Completed `headroom copilot login` with GitHub's device flow. 2. Ran `.venv/bin/headroom wrap vscode --port 8788`. 3. Confirmed VS Code retained its ordinary Copilot model catalog and made `GET /models` through Headroom with `GitHubCopilotChat/0.59.0` and `editor-version: vscode/1.131.0`. 4. Sent native Copilot `/p/headroom/chat/completions` requests through the same endpoint using `gpt-4.1`, `claude-sonnet-4.6`, and `claude-opus-4.7`. - Observed result: - All three completion requests returned HTTP 200. - GPT-4.1 resolved upstream to `gpt-4.1-2025-04-14`; Sonnet and Opus retained their exact selected IDs. - All returned the requested exact marker content. - VS Code's settings contained only the Headroom proxy URL and token auth override—no Headroom model or model-selection setting. - The proxy health endpoint remained ready with `openai_api_url` set to `https://api.githubcopilot.com`. - Not tested: - Physical Windows or Linux hosts (their path/config behavior is covered by unit tests). - WSL, dev containers, SSH remotes, VS Code Insiders, or enterprise Copilot deployments end-to-end. - Every model in the live Copilot catalog. - A fully submitted chat from VS Code's UI automation; the real extension's catalog request and native completion paths were verified separately. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing targeted unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable; this integration intentionally has no separate UI or model entry. ## Additional Notes The integration uses VS Code Copilot's shipped advanced/debug proxy endpoint seam. The managed settings block is deliberately narrow and reversible. Remote extension hosts may need their own reachable proxy/configuration as documented. --------- Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
2026-08-03 06:42:48 -05:00
"""CLI coverage for transparent VS Code Copilot setup and undo."""
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
from click.testing import CliRunner
from headroom.cli.main import main
from headroom.copilot_auth import CopilotSubscriptionTokenResolution
def _resolution() -> CopilotSubscriptionTokenResolution:
return CopilotSubscriptionTokenResolution(
token="copilot-token",
source="test",
confidence="test",
api_url="https://api.githubcopilot.com",
token_fingerprint="sha256:test",
)
def test_wrap_vscode_configures_actual_port_and_seeds_subscription(tmp_path: Path) -> None:
path = tmp_path / "settings.json"
captured = {}
def fake_watcher(**kwargs): # noqa: ANN003, ANN202
captured.update(kwargs)
kwargs["print_setup_lines"](9999)
with (
patch(
"headroom.cli.wrap._require_copilot_subscription_resolution", return_value=_resolution()
),
patch("headroom.cli.wrap._run_proxy_only_watcher", side_effect=fake_watcher),
):
result = CliRunner().invoke(main, ["wrap", "vscode", "--settings-file", str(path)])
assert result.exit_code == 0, result.output
settings = path.read_text(encoding="utf-8")
assert "http://127.0.0.1:9999/" in settings
assert "model" not in settings.lower()
assert "normal model picker" in result.output
assert captured["openai_api_url"] == "https://api.githubcopilot.com"
assert captured["copilot_api_token"] == "copilot-token"
def test_wrap_vscode_no_configure_prints_transparent_settings(tmp_path: Path) -> None:
path = tmp_path / "settings.json"
def fake_watcher(**kwargs): # noqa: ANN003, ANN202
kwargs["print_setup_lines"](8787)
with (
patch(
"headroom.cli.wrap._require_copilot_subscription_resolution", return_value=_resolution()
),
patch("headroom.cli.wrap._run_proxy_only_watcher", side_effect=fake_watcher),
):
result = CliRunner().invoke(
main,
["wrap", "vscode", "--no-configure", "--settings-file", str(path)],
)
assert result.exit_code == 0, result.output
assert not path.exists()
assert "overrideProxyUrl" in result.output
fix(vscode): persist compatible Claude modes and route Copilot CAPI (#2986) ## Description Fixes #2492, #2028, and #2827. Claude daemon workers consume project settings rather than reliably inheriting wrapper environment state, while the Claude VS Code webview cannot render deferred-tool response blocks. Separately, recent Copilot Chat versions use the whole CAPI override for generation; the legacy proxy override alone only sends model discovery through Headroom. This PR carries both integrations through to the actual consumers instead of only changing their launch-time surface configuration. ## Type of Change - [x] Bug fix - [ ] New feature - [ ] Breaking change - [ ] Documentation update - [ ] Build / CI ## Changes Made - Persist the resolved Claude ENABLE_TOOL_SEARCH value into project settings for daemon workers and restore it transactionally after wrap exits. - Use compatibility-safe Foundry and Claude VS Code defaults while preserving explicit user choices. - Configure both Copilot overrideProxyUrl and overrideCapiUrl in the reversible managed VS Code settings block. - Route Copilot unprefixed POST /chat/completions and HTTP /responses requests through the real compression handlers. - Keep /responses out of the Codex WebSocket aliases because Copilot and Codex use different WebSocket wire protocols. - Extend wrap E2E assertions for both the Claude webview mode and Copilot CAPI routing. ## Testing - [x] 127 combined Claude, Copilot, route-integration, and MCP dependency-contract tests pass. - [x] Ruff check passes on all changed Python files. - [x] Ruff format check passes. - [x] Python compilation and git diff --check pass. ## Runtime Safety Standalone Claude CLI defaults remain unchanged. Explicit Claude tool-search values retain precedence, and project settings are restored through the existing cleanup path. Copilot model/session helper endpoints continue through generic passthrough, while only validated HTTP generation paths receive explicit compression routes. Existing Codex WebSocket behavior is unchanged. ## Review Readiness - [x] Current main and MCP v1 compatibility retained - [x] Worker-facing Claude persistence covered - [x] Reversible Copilot and Claude settings behavior covered - [x] Copilot generation routes covered at registration and proxy integration layers - [x] Ready for review
2026-08-13 15:06:41 -05:00
assert "overrideCapiUrl" in result.output
fix(copilot): send VS Code inline completions to the host that serves them (#3112) ## Description #3077 stopped Copilot's inline completions being forwarded to `api.openai.com` (the corporate-blocked host in the original report) — but sent them to the **CAPI host**, which does not serve that endpoint. Copilot has two surfaces on two different hosts, and GitHub's own client library keeps them apart: ```js _getCAPIUrl(t) -> t?.endpoints.api || "https://api.githubcopilot.com" _getProxyUrl(t) -> t?.endpoints.proxy || DEFAULT_PROXY_BASE_URL DEFAULT_PROXY_BASE_URL = "https://copilot-proxy.githubusercontent.com" ``` building completions as `${proxyBaseURL}/v1/engines/<engine>/completions` (`@vscode/copilot-api` 0.5.2). Probed unauthenticated against the live hosts: | host | `POST /v1/engines/<e>/completions` | |---|---| | `copilot-proxy.githubusercontent.com` | **401** — exists, needs auth | | `proxy.individual.githubcopilot.com` | **401** — CNAME to the above | | `api.githubcopilot.com` | **404** — does not serve this path | So the destination #3077 chose could not have worked. Three separate defects were in the way, each sufficient on its own to keep completions broken. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `copilot_auth.py`: added `DEFAULT_COMPLETIONS_PROXY_URL` and made it the default in `copilot_completions_base_url()`, replacing the CAPI host. - `copilot_auth.py`: the "custom deployment keeps its own host" rule now excludes public Copilot hosts. Without this, `headroom wrap vscode` — the common setup, and the one that exports `GITHUB_COPILOT_API_URL=<resolved subscription URL>` — resolved straight back to the 404 host. **This was a bug in my own first cut of the fix, found by testing the real `wrap vscode` environment rather than just the routing table.** - `copilot_auth.py`: added `is_copilot_completions_host()` and `is_copilot_upstream_url()` (chat ∪ completions). The completions host was recognised as Copilot **nowhere**, so `apply_copilot_api_auth` attached no credentials (401 — routing correctly to a host we then failed to authenticate against) and `build_copilot_upstream_url` skipped `mark_request_routed_to_copilot()`, mislabelling the provider in telemetry. - The union is applied at exactly those two call sites. `is_copilot_api_url` is left alone, so validation of a token payload's `endpoints.api` and the Responses-API preference check keep their strict chat-only meaning. All six call sites were read before choosing this. - `proxy_targets.py`: the "already a Copilot host" guard now keys on the *completions* host. A CAPI host is not a completions host, so it must still be redirected; a genuine per-SKU completions host or operator override is still left untouched. - `providers/copilot/vscode.py`, `cli/wrap.py`, `docs/…/vscode-copilot.mdx`: stop writing/printing `github.copilot.advanced.debug.overrideAuthType`. No such setting exists in the modern Copilot Chat extension — the only one left after `GitHub.copilot` was deprecated in early 2026. Its full `advanced.*` surface is `authPermissions`, `authProvider`, `debug.overrideCapiUrl`, `debug.overrideProxyUrl`, `debug.use*Fetcher`. It is still *recognised* so a stale hand-written copy is detected, just never emitted. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text tests/test_copilot_vscode_completions_routing.py 59 passed Copilot-related suites 293 passed, 8 skipped Full suite: 3 failed, 11250 passed, 581 skipped in 342.50s ``` The 3 failures are pre-existing and environmental, identical to a plain-`main` baseline on this machine: no `cargo` (`test_no_native_tls_in_wheel_build_tree`), no `codex` CLI (`test_learn/test_integration.py`), and `test_run_server_installs_cancelled_error_filter`, which fails under full-suite ordering on `main` too. ## Real Behavior Proof - Environment: macOS (darwin 25.4.0), Python 3.12.13, worktree off `main` @ `139c7cbd`, `HEADROOM_SKIP_UPSTREAM_CHECK=1` - Exact command / steps: (1) composed the real request path — `select_passthrough_base_url(proxy, headers, path)` → `build_copilot_upstream_url` → `apply_copilot_api_auth` — across 7 deployment shapes (no config, `wrap vscode`, advertised `endpoints.proxy`, operator override, GHE `.ghe.com`, GHE custom domain, target already a completions host); (2) probed the three candidate hosts unauthenticated with `curl -X POST /v1/engines/gpt-4o-copilot/completions`; (3) round-tripped `settings.json` through empty / one-setting / comments+array / CRLF shapes asserting valid JSON, idempotency and clean removal. - Observed result: before — `api.githubcopilot.com/...` (404 host), and with `GITHUB_COPILOT_API_URL` set as `wrap vscode` sets it, `api.business.githubcopilot.com` (also 404); no `Authorization` header on the completions host. After — `copilot-proxy.githubusercontent.com/v1/engines/gpt-41-copilot/completions` with credentials attached in every public-Copilot shape, `endpoints.proxy` and the operator override still winning, and a GHE tenant staying on its own host. `settings.json` stays valid JSON in all four shapes with the dead key gone; the two `restored=False` cases are pre-existing whitespace/CRLF normalisation, identical on `main`. Reverting the source fails 14 of the new tests, including the credential test on the completions host. - Not tested: no live VS Code session and no authenticated completion — the 401 proves the endpoint exists, not that GitHub accepts our forwarded request, which needs a real Copilot token. Confirmation from @rganesh-msys is still wanted. **Enterprise remains unresolved by default**: a GHE tenant stays on its own CAPI host, which is likely still the wrong surface for completions, but staying in-tenant beats forwarding keystrokes to a public GitHub host — `GITHUB_COPILOT_PROXY_URL` is the exact fix and now takes precedence over everything. ## Runtime Rollout Safety - Rollout-managed feature(s): None — no rollout channel gates this. - Minimum rollout channel: n/a - Stable/default behavior changed: Yes, and deliberately — the completions destination moves from a host that answers 404 to the one GitHub's own client defaults to. Only `/v1/engines/<engine>/completions` is affected; every other path keeps its upstream, pinned by tests. Copilot credentials now also reach the completions host, which is the point. - Kill switch / disable path: `GITHUB_COPILOT_PROXY_URL` pins the destination explicitly and beats all inference. - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert this commit; completions return to the CAPI host (404) and the settings block regains the inert `overrideAuthType`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes Two things found while reading the extension source, **not changed here**: 1. `advanced.debug.overrideProxyUrl` is **not** deprecated — the report that Copilot 0.60.0 stopped honouring it does not hold. The current canonical key is `github.copilot.internal.completionsUrl`, and `advanced.debug.overrideProxyUrl` is checked as its explicit legacy fallback (`getEndpointOverrideUrl` in `completions-core/lib/src/networkConfiguration.ts`), so what we write still works. Worth migrating to the `internal.*` keys eventually, since they take precedence. 2. `endpoints.proxy` is still only recorded during a token exchange, which is opt-in via `GITHUB_COPILOT_USE_TOKEN_EXCHANGE`, and the base URL is chosen before auth runs. With the default now correct this is a refinement for per-SKU hosts rather than a correctness requirement, so it is left as-is. Closes #3076 --------- Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 15:21:22 -07:00
# No `overrideAuthType`: the setting does not exist in the modern Copilot
# Chat extension, so printing it told users to add a key VS Code flags as
# unknown and which does nothing (#3076).
assert "overrideAuthType" not in result.output
feat(copilot): proxy VS Code models transparently (#2687) ## Description Make Headroom a transparent proxy for VS Code GitHub Copilot. Users keep using Copilot's normal model picker—GPT-4.1, Claude Sonnet, Claude Opus, and other models in their entitlement—while Headroom silently forwards the selected model instead of registering or requiring a separate "Headroom" model. This also fixes GitHub's device OAuth exchange by sending form-encoded request bodies, matching the endpoint contract. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom wrap vscode` to start a Copilot-seeded subscription proxy and safely configure VS Code's shipped Copilot proxy override. - Add `headroom unwrap vscode` for reversible cleanup. - Preserve VS Code's selected model by changing only the proxy URL/auth override; no custom model is registered and no model preference is written. - Support stable VS Code settings locations on macOS, Windows, and Linux, plus `--settings-file` for Insiders, portable, and other installations. - Edit JSONC settings with a marker-owned block while preserving unrelated bytes, comments, ordering, and trailing commas. - Refuse malformed markers, invalid JSONC, or unmanaged existing Copilot overrides instead of overwriting user configuration. - Fix SIGINT cleanup so the managed settings block is removed and normal shutdown exits successfully. - Fix Copilot device OAuth start/poll requests to use `application/x-www-form-urlencoded`. - Add a compatibility matrix, setup/removal flow, credential behavior, remote-development guidance, enterprise notes, troubleshooting, and verification documentation. ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ .venv/bin/pytest -q tests/test_provider_copilot_vscode_config.py tests/test_cli/test_wrap_vscode.py tests/test_cli/test_wrap_helpers.py tests/test_copilot_auth.py tests/test_cli/test_copilot_auth.py tests/test_provider_copilot_wrap.py tests/test_cli/test_wrap_copilot.py tests/test_copilot_provider_label.py tests/test_copilot_subscription_smoke.py 244 passed in 0.59s $ .venv/bin/ruff check <changed Python files and tests> All checks passed! $ .venv/bin/mypy headroom/providers/copilot/vscode.py Success: no issues found in 1 source file $ cd docs && npm run types:check fumadocs-mdx && next typegen && tsc --noEmit # exited 0 $ git diff --check # exited 0 ``` The full 10,179-test suite was also sampled through approximately 83%, but was stopped because of its runtime. It exposed existing failures in `test_recover_codex.py`, `test_wrap_stale_marker.py`, and `test_proxy_health.py`; therefore the broad `pytest`, repository-wide Ruff, and repository-wide mypy boxes are intentionally not checked. ## Real Behavior Proof - Environment: macOS arm64, VS Code 1.131.0, built-in GitHub Copilot 0.59.0, Headroom 0.33.1-dev. - Exact command / steps: 1. Completed `headroom copilot login` with GitHub's device flow. 2. Ran `.venv/bin/headroom wrap vscode --port 8788`. 3. Confirmed VS Code retained its ordinary Copilot model catalog and made `GET /models` through Headroom with `GitHubCopilotChat/0.59.0` and `editor-version: vscode/1.131.0`. 4. Sent native Copilot `/p/headroom/chat/completions` requests through the same endpoint using `gpt-4.1`, `claude-sonnet-4.6`, and `claude-opus-4.7`. - Observed result: - All three completion requests returned HTTP 200. - GPT-4.1 resolved upstream to `gpt-4.1-2025-04-14`; Sonnet and Opus retained their exact selected IDs. - All returned the requested exact marker content. - VS Code's settings contained only the Headroom proxy URL and token auth override—no Headroom model or model-selection setting. - The proxy health endpoint remained ready with `openai_api_url` set to `https://api.githubcopilot.com`. - Not tested: - Physical Windows or Linux hosts (their path/config behavior is covered by unit tests). - WSL, dev containers, SSH remotes, VS Code Insiders, or enterprise Copilot deployments end-to-end. - Every model in the live Copilot catalog. - A fully submitted chat from VS Code's UI automation; the real extension's catalog request and native completion paths were verified separately. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing targeted unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable; this integration intentionally has no separate UI or model entry. ## Additional Notes The integration uses VS Code Copilot's shipped advanced/debug proxy endpoint seam. The managed settings block is deliberately narrow and reversible. Remote extension hosts may need their own reachable proxy/configuration as documented. --------- Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
2026-08-03 06:42:48 -05:00
def test_unwrap_vscode_removes_only_managed_settings(tmp_path: Path) -> None:
path = tmp_path / "settings.json"
original = '{\n "editor.fontSize": 14\n}\n'
path.write_text(original, encoding="utf-8")
from headroom.providers.copilot.vscode import configure_vscode_proxy_settings
configure_vscode_proxy_settings(path, "http://127.0.0.1:8787")
result = CliRunner().invoke(main, ["unwrap", "vscode", "--settings-file", str(path)])
assert result.exit_code == 0, result.output
assert path.read_text(encoding="utf-8") == original