From 18925b8c6e343c9d593891cd29ac27fee1cb9836 Mon Sep 17 00:00:00 2001 From: Tejas Chopra Date: Thu, 4 Jun 2026 16:27:54 -0700 Subject: [PATCH] fix(copilot): restore generic endpoint for non-subscription OAuth (#610) (#612) * fix(copilot): restore generic endpoint for non-subscription OAuth (#610) 0.23.0 re-pointed the shared Copilot OAuth branch from the generic api.githubcopilot.com host to the account-specific endpoints.api host returned by /copilot_internal/user, and made resolve_copilot_api_url ignore the GITHUB_COPILOT_API_URL override whenever a token resolved. That change was meant to add --subscription, but it also altered the pre-existing non-subscription OAuth flow that worked on 0.22.4. The account host does not serve newer models (e.g. gpt-5.4) on the responses API, so wrapped requests began failing with unsupported-model errors while plain Copilot and 0.22.4 kept working. Restore 0.22.4 routing for non-subscription OAuth (generic host, still overridable) and keep account resolution only for --subscription. resolve_copilot_api_url now honors GITHUB_COPILOT_API_URL first, so the override escape hatch works for every path. BYOK is unaffected. Add a regression suite that mocks a successful user-info response, the real-world path the prior test never exercised (it relied on the network call failing in CI and falling back to the generic host). * fix(copilot): route subscription + OAuth through the generic host (#610) The 0.23.0 endpoint resolution derived the Copilot API host from /copilot_internal/user (endpoints.api), which returns a segmented host (e.g. api.individual.githubcopilot.com) that does not serve newer models on the responses API and is not the host the official Copilot client routes with (that comes from the token-exchange endpoint). --subscription used the identical resolution, so it carried the same latent regression as the non-subscription OAuth path. Make Copilot host resolution override -> generic for BOTH --subscription and the implicit OAuth path, and stop using user-info to route. Accounts that require a dedicated host (enterprise / data residency) pin it via GITHUB_COPILOT_API_URL. resolve_copilot_api_url no longer makes a network call; _fetch_copilot_user_info is retained for token validation. Update the subscription smoke tests that encoded the old account-host assumption, and add wrap-level + unit coverage that --subscription routes to the generic host even when user-info advertises an account host, and that the GITHUB_COPILOT_API_URL override flows through both paths. * docs(copilot): document generic-host routing + enterprise override (#610) Spell out the routing contract introduced by the #610 fix so enterprise users have a supported path. Headroom routes wrapped Copilot hosted traffic (--subscription and OAuth) to the generic api.githubcopilot.com, and accounts on a dedicated host (Enterprise Cloud data residency, egress proxy) pin it via GITHUB_COPILOT_API_URL. - copilot --help: note the generic host + GITHUB_COPILOT_API_URL override. - TESTING-copilot-subscription.md: add "API host & Enterprise / data residency" section; correct the stale api.*.githubcopilot.com claim; and invite enterprise tenants who want token-exchange-based auto-detection to open an issue. - integration-guide.md: short hosted-host + override note in the Copilot section. --- TESTING-copilot-subscription.md | 33 +++- headroom/cli/wrap.py | 16 ++ headroom/copilot_auth.py | 30 ++-- tests/test_cli/test_wrap_copilot.py | 220 +++++++++++++++++++++++ tests/test_copilot_subscription_smoke.py | 25 ++- wiki/integration-guide.md | 2 + 6 files changed, 306 insertions(+), 20 deletions(-) diff --git a/TESTING-copilot-subscription.md b/TESTING-copilot-subscription.md index 7d0b09ddb..69204ca24 100644 --- a/TESTING-copilot-subscription.md +++ b/TESTING-copilot-subscription.md @@ -22,7 +22,38 @@ Mechanically: the Copilot CLI's only interposition hook is its provider-override subscription token** and points back at **GitHub's own Copilot API**. So the CLI may print "BYOK" and require an explicit `--model`, but you are **not** paying a third party — it's your subscription, just compressed. (Proof it's working: the -proxy forwards to `https://api.*.githubcopilot.com` with your token.) +proxy forwards to GitHub's Copilot API — `https://api.githubcopilot.com` by +default — with your token.) + +## API host & Enterprise / data-residency + +Headroom routes wrapped Copilot traffic to GitHub's **generic public host**, +`https://api.githubcopilot.com`, for both `--subscription` and the implicit +OAuth path. That host serves the full model set (including newer models on the +responses API) and matches the routing that worked before 0.23. + +Headroom deliberately does **not** auto-select a per-account host from +`/copilot_internal/user`. That endpoint advertises a segmented host (e.g. +`api.individual.githubcopilot.com`) that does **not** serve newer models on the +responses API and is not the host the official Copilot client routes with — using +it regressed `headroom wrap copilot` after 0.22.4 +([#610](https://github.com/chopratejas/headroom/issues/610)). + +**Enterprise / data-residency:** if your organization is provisioned on a +dedicated Copilot API host (GitHub Enterprise Cloud with data residency, or an +egress proxy), pin it explicitly — the override flows through both +`--subscription` and OAuth, and onward through the proxy to the upstream request: + +```bash +export GITHUB_COPILOT_API_URL=https://api..githubcopilot.com +headroom wrap copilot --subscription -- --model gpt-5.4 +``` + +If you operate such an environment and would like Headroom to **auto-detect** the +correct host instead of pinning it, please [open an issue](https://github.com/chopratejas/headroom/issues/new) — +the intended path is to resolve it from GitHub's token-exchange endpoint (the +source the official Copilot client uses), and we'd want to validate it against a +real enterprise tenant. ## Status diff --git a/headroom/cli/wrap.py b/headroom/cli/wrap.py index 3ab64f3f6..0959a602f 100644 --- a/headroom/cli/wrap.py +++ b/headroom/cli/wrap.py @@ -2439,6 +2439,13 @@ def copilot( headroom wrap copilot --provider-type openai --wire-api responses -- --model gpt-5.4 headroom wrap copilot --subscription -- --model gpt-4.1 headroom wrap copilot --no-context-tool -- --prompt "explain this file" + + \b + Copilot hosted API (--subscription and the implicit OAuth path) routes to the + generic host https://api.githubcopilot.com, which serves the full model set. + Enterprise / data-residency accounts provisioned on a dedicated host pin it + explicitly with GITHUB_COPILOT_API_URL (the override flows through to upstream). + See TESTING-copilot-subscription.md for details. """ copilot_bin = shutil.which("copilot") if not copilot_bin: @@ -2534,6 +2541,15 @@ def copilot( else "COPILOT_AUTH_MODE=github-oauth" ), ] + # Resolve the Copilot API host: an explicit GITHUB_COPILOT_API_URL wins, + # otherwise the generic public host (api.githubcopilot.com). This is the + # same policy for --subscription and the implicit OAuth path. The + # account-specific endpoints.api advertised by /copilot_internal/user is + # deliberately NOT used to route — it returns a segmented host (e.g. + # api.individual.githubcopilot.com) that does not serve newer models on + # the responses API (#610), and it is not the host the official Copilot + # client routes with. Accounts that require a dedicated host (enterprise / + # data residency) set GITHUB_COPILOT_API_URL explicitly. openai_api_url = resolve_copilot_api_url(client_bearer) env["GITHUB_COPILOT_API_URL"] = openai_api_url env["OPENAI_TARGET_API_URL"] = openai_api_url diff --git a/headroom/copilot_auth.py b/headroom/copilot_auth.py index 4b5bec6a0..caf5929f1 100644 --- a/headroom/copilot_auth.py +++ b/headroom/copilot_auth.py @@ -474,21 +474,27 @@ def build_copilot_upstream_url(base_url: str, path: str) -> str: def resolve_copilot_api_url(oauth_token: str | None = None) -> str: - """Return the Copilot API endpoint advertised for the current OAuth token.""" + """Return the Copilot API host to route wrapped requests through. - token = (oauth_token or read_cached_oauth_token() or "").strip() - if not token: - return os.environ.get("GITHUB_COPILOT_API_URL", DEFAULT_API_URL).strip() or DEFAULT_API_URL + Resolution order: - payload = _fetch_copilot_user_info(token) - if payload is None: - return os.environ.get("GITHUB_COPILOT_API_URL", DEFAULT_API_URL).strip() or DEFAULT_API_URL + 1. An explicit ``GITHUB_COPILOT_API_URL`` — the operator's escape hatch + (corporate proxy, enterprise / data-residency host, tests). + 2. The generic public host ``https://api.githubcopilot.com``. - endpoints = payload.get("endpoints") if isinstance(payload, dict) else None - api_url = endpoints.get("api") if isinstance(endpoints, dict) else None - if isinstance(api_url, str) and api_url.strip(): - return api_url.strip() - return os.environ.get("GITHUB_COPILOT_API_URL", DEFAULT_API_URL).strip() or DEFAULT_API_URL + The account-specific ``endpoints.api`` advertised by ``/copilot_internal/user`` + is intentionally NOT used to route. It returns a segmented host (e.g. + ``api.individual.githubcopilot.com``) that does not serve newer models on the + responses API — wrapping such a request regressed after 0.22.4 (#610) — and it + is not the host the official Copilot client routes with (that comes from the + token-exchange endpoint, not user info). Accounts that genuinely require a + dedicated host set ``GITHUB_COPILOT_API_URL`` explicitly. ``oauth_token`` is + accepted for call-site compatibility but no longer triggers a network lookup. + """ + + del oauth_token # reserved; routing no longer depends on a user-info lookup + override = os.environ.get("GITHUB_COPILOT_API_URL", "").strip() + return override or DEFAULT_API_URL def _fetch_copilot_user_info(token: str) -> dict[str, Any] | None: diff --git a/tests/test_cli/test_wrap_copilot.py b/tests/test_cli/test_wrap_copilot.py index 5be89c210..30314f582 100644 --- a/tests/test_cli/test_wrap_copilot.py +++ b/tests/test_cli/test_wrap_copilot.py @@ -486,3 +486,223 @@ def test_wrap_copilot_fails_when_binary_missing( assert result.exit_code == 1 assert "'copilot' not found in PATH" in result.output assert "Install GitHub Copilot CLI" in result.output + + +# --------------------------------------------------------------------------- +# Regression suite for #610 — GitHub Copilot endpoint routing per auth mode. +# +# 0.23.0 (commit f4dff9b) re-pointed the *shared* OAuth branch away from the +# generic https://api.githubcopilot.com to the account-specific endpoints.api +# host returned by /copilot_internal/user, and made resolve_copilot_api_url() +# ignore the GITHUB_COPILOT_API_URL override whenever a token resolves. For +# individual-plan users that broke newer models (gpt-5.4) on the responses API +# that had worked on 0.22.4. The pre-existing oauth test passed only because it +# left _fetch_copilot_user_info unmocked — the network call fails in CI, so +# resolve_copilot_api_url() fell back to the generic host and the real-world +# success path was never exercised. These tests mock a *successful* user-info +# response (the real world) so the routing for every auth mode is locked. +# --------------------------------------------------------------------------- + +_ACCOUNT_USER_INFO = {"endpoints": {"api": "https://api.individual.githubcopilot.com"}} + + +def _clear_copilot_env(monkeypatch: pytest.MonkeyPatch) -> None: + for var in ( + "COPILOT_PROVIDER_API_KEY", + "COPILOT_PROVIDER_BEARER_TOKEN", + "OPENAI_API_KEY", + "ANTHROPIC_API_KEY", + "GITHUB_COPILOT_API_URL", + "GITHUB_COPILOT_TOKEN", + "GITHUB_COPILOT_GITHUB_TOKEN", + ): + monkeypatch.delenv(var, raising=False) + + +def test_wrap_copilot_oauth_keeps_generic_endpoint_when_account_advertised( + runner: CliRunner, + wrap_modules: tuple[types.ModuleType, click.Group], + monkeypatch: pytest.MonkeyPatch, +) -> None: + """#610: non-subscription OAuth must route to the generic Copilot endpoint + even when /copilot_internal/user advertises an account-specific host. The + account host (api.individual.githubcopilot.com) does not serve newer models + such as gpt-5.4 on the responses API — exactly what regressed after 0.22.4. + """ + _wrap_cli, main = wrap_modules + _clear_copilot_env(monkeypatch) + captured: dict[str, object] = {} + + def fake_launch_tool(**kwargs): # noqa: ANN003 + captured.update(kwargs) + + with ( + patch("headroom.cli.wrap.shutil.which", return_value="copilot"), + patch("headroom.cli.wrap.resolve_client_bearer_token", return_value="gho-oauth"), + patch("headroom.cli.wrap.has_oauth_auth", return_value=True), + patch("headroom.copilot_auth._fetch_copilot_user_info", return_value=_ACCOUNT_USER_INFO), + patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool), + ): + result = runner.invoke(main, ["wrap", "copilot", "--no-rtk", "--", "--model", "gpt-5.4"]) + + assert result.exit_code == 0, result.output + env = captured["env"] + assert isinstance(env, dict) + assert env["COPILOT_PROVIDER_BEARER_TOKEN"] == "gho-oauth" + assert captured["openai_api_url"] == DEFAULT_API_URL + assert env["OPENAI_TARGET_API_URL"] == DEFAULT_API_URL + assert env["GITHUB_COPILOT_API_URL"] == DEFAULT_API_URL + + +def test_wrap_copilot_oauth_honors_api_url_override( + runner: CliRunner, + wrap_modules: tuple[types.ModuleType, click.Group], + monkeypatch: pytest.MonkeyPatch, +) -> None: + """The GITHUB_COPILOT_API_URL escape hatch must be honored even when a token + resolves and user-info advertises a different host (it was silently lost).""" + _wrap_cli, main = wrap_modules + _clear_copilot_env(monkeypatch) + monkeypatch.setenv("GITHUB_COPILOT_API_URL", "https://proxy.internal.example.com") + captured: dict[str, object] = {} + + def fake_launch_tool(**kwargs): # noqa: ANN003 + captured.update(kwargs) + + with ( + patch("headroom.cli.wrap.shutil.which", return_value="copilot"), + patch("headroom.cli.wrap.resolve_client_bearer_token", return_value="gho-oauth"), + patch("headroom.cli.wrap.has_oauth_auth", return_value=True), + patch("headroom.copilot_auth._fetch_copilot_user_info", return_value=_ACCOUNT_USER_INFO), + patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool), + ): + result = runner.invoke(main, ["wrap", "copilot", "--no-rtk", "--", "--model", "gpt-5.4"]) + + assert result.exit_code == 0, result.output + env = captured["env"] + assert isinstance(env, dict) + assert captured["openai_api_url"] == "https://proxy.internal.example.com" + assert env["OPENAI_TARGET_API_URL"] == "https://proxy.internal.example.com" + + +def test_wrap_copilot_byok_never_resolves_copilot_endpoint( + runner: CliRunner, + wrap_modules: tuple[types.ModuleType, click.Group], + monkeypatch: pytest.MonkeyPatch, +) -> None: + """BYOK (provider key, no OAuth) routes to the model provider through the + proxy and must never resolve the Copilot hosted endpoint. It was unaffected + by #610 — this pins that independence so a future change can't entangle it. + """ + _wrap_cli, main = wrap_modules + _clear_copilot_env(monkeypatch) + monkeypatch.setenv("COPILOT_PROVIDER_API_KEY", "sk-test-dummy") + captured: dict[str, object] = {} + + def fake_launch_tool(**kwargs): # noqa: ANN003 + captured.update(kwargs) + + def tripwire(*_args, **_kwargs): # noqa: ANN002,ANN003 + raise AssertionError("BYOK must not resolve the Copilot hosted endpoint") + + with ( + patch("headroom.cli.wrap.shutil.which", return_value="copilot"), + patch("headroom.cli.wrap.has_oauth_auth", return_value=False), + patch("headroom.cli.wrap.resolve_copilot_api_url", side_effect=tripwire), + patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool), + ): + result = runner.invoke( + main, + ["wrap", "copilot", "--no-rtk", "--provider-type", "openai", "--", "--model", "gpt-4o"], + ) + + assert result.exit_code == 0, result.output + env = captured["env"] + assert isinstance(env, dict) + assert captured["openai_api_url"] is None + assert env["COPILOT_PROVIDER_TYPE"] == "openai" + + +def test_wrap_copilot_subscription_uses_generic_endpoint_not_account( + runner: CliRunner, + wrap_modules: tuple[types.ModuleType, click.Group], + monkeypatch: pytest.MonkeyPatch, +) -> None: + """#610 (subscription has the same latent bug): --subscription must route to + the generic host too, even when /copilot_internal/user advertises an + account-specific host. The segmented host does not serve newer models on the + responses API, and it is not the host the official Copilot client uses.""" + _wrap_cli, main = wrap_modules + _clear_copilot_env(monkeypatch) + captured: dict[str, object] = {} + + def fake_launch_tool(**kwargs): # noqa: ANN003 + captured.update(kwargs) + + with ( + patch("headroom.cli.wrap.shutil.which", return_value="copilot"), + patch("headroom.cli.wrap.resolve_subscription_bearer_token", return_value="gho-sub"), + patch("headroom.cli.wrap.has_oauth_auth", return_value=True), + patch("headroom.copilot_auth._fetch_copilot_user_info", return_value=_ACCOUNT_USER_INFO), + patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool), + ): + result = runner.invoke( + main, + ["wrap", "copilot", "--subscription", "--no-rtk", "--", "--model", "gpt-5.4"], + ) + + assert result.exit_code == 0, result.output + env = captured["env"] + assert isinstance(env, dict) + assert captured["openai_api_url"] == DEFAULT_API_URL + assert env["OPENAI_TARGET_API_URL"] == DEFAULT_API_URL + assert env["COPILOT_PROVIDER_BEARER_TOKEN"] == "gho-sub" + + +def test_wrap_copilot_subscription_honors_api_url_override( + runner: CliRunner, + wrap_modules: tuple[types.ModuleType, click.Group], + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Enterprise / data-residency accounts that require a dedicated host pin it + via GITHUB_COPILOT_API_URL — the override must flow through --subscription.""" + _wrap_cli, main = wrap_modules + _clear_copilot_env(monkeypatch) + monkeypatch.setenv("GITHUB_COPILOT_API_URL", "https://api.enterprise.example.com") + captured: dict[str, object] = {} + + def fake_launch_tool(**kwargs): # noqa: ANN003 + captured.update(kwargs) + + with ( + patch("headroom.cli.wrap.shutil.which", return_value="copilot"), + patch("headroom.cli.wrap.resolve_subscription_bearer_token", return_value="gho-sub"), + patch("headroom.cli.wrap.has_oauth_auth", return_value=True), + patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool), + ): + result = runner.invoke( + main, + ["wrap", "copilot", "--subscription", "--no-rtk", "--", "--model", "gpt-5.4"], + ) + + assert result.exit_code == 0, result.output + assert captured["openai_api_url"] == "https://api.enterprise.example.com" + + +def test_resolve_copilot_api_url_ignores_user_info_and_never_calls_network( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Unit lock for #610: routing is override -> generic and must NOT depend on a + user-info lookup. Even with a token in hand and user-info advertising an + account host, the generic host is returned and no network call is made.""" + from headroom import copilot_auth + + monkeypatch.delenv("GITHUB_COPILOT_API_URL", raising=False) + with patch.object(copilot_auth, "_fetch_copilot_user_info") as fetch: + assert copilot_auth.resolve_copilot_api_url("gho-real") == copilot_auth.DEFAULT_API_URL + fetch.assert_not_called() + + monkeypatch.setenv("GITHUB_COPILOT_API_URL", "https://pin.example.com") + with patch.object(copilot_auth, "_fetch_copilot_user_info") as fetch: + assert copilot_auth.resolve_copilot_api_url("gho-real") == "https://pin.example.com" + fetch.assert_not_called() diff --git a/tests/test_copilot_subscription_smoke.py b/tests/test_copilot_subscription_smoke.py index e9759b5e8..74fc65e6b 100644 --- a/tests/test_copilot_subscription_smoke.py +++ b/tests/test_copilot_subscription_smoke.py @@ -67,7 +67,11 @@ def test_env_token_resolves_subscription_without_secret_store( ) assert copilot_auth.resolve_subscription_bearer_token() == "gho-env-universal" - assert copilot_auth.resolve_copilot_api_url("gho-env-universal") == BUSINESS_API + # Routing is override -> generic; the account host advertised by user-info is + # NOT used (it regressed newer models on the responses API, #610). With no + # GITHUB_COPILOT_API_URL pin set, the generic public host is returned. + monkeypatch.delenv("GITHUB_COPILOT_API_URL", raising=False) + assert copilot_auth.resolve_copilot_api_url("gho-env-universal") == copilot_auth.DEFAULT_API_URL def test_api_url_falls_back_to_default_when_user_info_unavailable( @@ -164,29 +168,36 @@ def test_proxy_injects_explicit_token_over_discovered_one( # --------------------------------------------------------------------------- -# 4. Full wrapper→proxy chain carries one consistent token, any account host. +# 4. Full wrapper→proxy chain carries one consistent token to a pinned host. # --------------------------------------------------------------------------- def test_end_to_end_subscription_chain(monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setattr(copilot_auth, "_provider", None) - # (a) wrapper side: resolve + validate the subscription token, then - # discover the account-specific API endpoint. + # (a) wrapper side: resolve + validate the subscription token. The API host + # comes from the GITHUB_COPILOT_API_URL pin — the supported way to target + # a dedicated enterprise / data-residency host. user-info is NOT used to + # route (#610), so it advertises a *different* host here to prove it is + # ignored when picking the upstream. _stub_all_secret_stores(monkeypatch) _clear_token_env(monkeypatch) monkeypatch.setenv("GITHUB_COPILOT_TOKEN", "gho-seat-token") + monkeypatch.setenv("GITHUB_COPILOT_API_URL", BUSINESS_API) monkeypatch.setattr( copilot_auth, "_fetch_copilot_user_info", - lambda token: {"endpoints": {"api": BUSINESS_API}} if token == "gho-seat-token" else None, + lambda token: ( + {"endpoints": {"api": "https://api.individual.githubcopilot.com"}} + if token == "gho-seat-token" + else None + ), ) resolved_token = copilot_auth.resolve_subscription_bearer_token() resolved_url = copilot_auth.resolve_copilot_api_url(resolved_token) assert resolved_token == "gho-seat-token" - assert resolved_url == BUSINESS_API + assert resolved_url == BUSINESS_API # the pin wins; the user-info host is ignored # (b) hand-off: the wrapper exports exactly these for the proxy. monkeypatch.setenv("GITHUB_COPILOT_API_TOKEN", resolved_token) - monkeypatch.setenv("GITHUB_COPILOT_API_URL", resolved_url) # (c) proxy side: build the upstream URL (Copilot has no /v1 prefix) and # inject the same token onto the outbound request. diff --git a/wiki/integration-guide.md b/wiki/integration-guide.md index 75f4cea32..de1e1b7b0 100644 --- a/wiki/integration-guide.md +++ b/wiki/integration-guide.md @@ -208,6 +208,8 @@ headroom wrap copilot --backend anyllm --anyllm-provider groq -- --model gpt-4o By default, `headroom wrap copilot` installs `rtk` and appends token-optimized shell guidance to `.github/copilot-instructions.md` so Copilot sessions reuse the same command-saving conventions as other wrapped agent CLIs. Use `--no-rtk` to skip that step. +For Copilot's **hosted** API (`--subscription` and the implicit OAuth path), Headroom routes to the generic host `https://api.githubcopilot.com`, which serves the full model set. **Enterprise / data-residency** tenants on a dedicated Copilot host pin it with `GITHUB_COPILOT_API_URL` (e.g. `export GITHUB_COPILOT_API_URL=https://api..githubcopilot.com`); the override flows through to the upstream request. See [`TESTING-copilot-subscription.md`](https://github.com/chopratejas/headroom/blob/main/TESTING-copilot-subscription.md). + ### With Cloud Providers ```bash