diff --git a/CHANGELOG.md b/CHANGELOG.md index 71b686432..9438e55f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Features +* **proxy:** add provider-only HTTP proxy routing via `--http-proxy` and + `HEADROOM_HTTP_PROXY`. Upstream LLM provider calls can now use an HTTP proxy + without setting process-wide `HTTP_PROXY`/`HTTPS_PROXY` variables that are + inherited by tool executions; proxied provider clients use HTTP/1.1 so HTTPS + provider APIs can tunnel through CONNECT. * **proxy:** add output shaping for OpenAI Responses traffic on `/v1/responses` HTTP requests and Codex WebSocket `response.create` frames, with stable output-savings holdout keys and counted WS token strata for the experiment. * **wrap:** `headroom wrap claude --1m` preserves the 1M context window. Behind a custom `ANTHROPIC_BASE_URL` (the proxy) Claude Code drops the `context-1m` beta header and caps the window at 200k for entitled subscription users; the opt-in flag sets `ANTHROPIC_MODEL=[1m]` on the launched process so the 1M window activates through Headroom. A model already selected via `ANTHROPIC_MODEL` is preserved (only the `[1m]` suffix is appended) ([#1158](https://github.com/chopratejas/headroom/issues/1158)). * **learn:** weight loops in `headroom learn`. A new loop detector (`headroom/learn/loops.py`) recognizes repeated tool-call patterns — including RTK re-fetch loops, where RTK's output truncation makes the agent re-run larger-limit variants of a *successful* command — collapses output-limit variants to one signature, measures the wasted tokens, surfaces loops as a highest-priority digest section, and weights loop guardrails above one-off rules by their measured waste. Previously loops had no special weight and a no-failure re-fetch loop was skipped entirely. Adds an RTK-loop eval (`benchmarks/rtk_loop_learn_eval.py`) that reproduces a loop, runs it through Learn, and asserts the generated guardrail ranks first and prevents re-triggering. diff --git a/docs/content/docs/configuration.mdx b/docs/content/docs/configuration.mdx index 2c53089fc..0fa0050dc 100644 --- a/docs/content/docs/configuration.mdx +++ b/docs/content/docs/configuration.mdx @@ -228,6 +228,7 @@ headroom proxy --learn --min-evidence 3 | `HEADROOM_MAX_CONNECTIONS` | Maximum upstream HTTP connections | `500` | | `HEADROOM_MAX_KEEPALIVE` | Maximum upstream keep-alive connections | `100` | | `HEADROOM_KEEPALIVE_EXPIRY` | Seconds an idle upstream keep-alive connection is kept open | `90` | +| `HEADROOM_HTTP_PROXY` | HTTP proxy URL for upstream provider requests only; HTTPS provider APIs use CONNECT | -- | | `HEADROOM_BUDGET` | Daily budget limit in USD | -- | | `HEADROOM_TELEMETRY` | Set to `on` to opt in to anonymous telemetry | `off` (opt-in) | | `HEADROOM_STATELESS` | Set to `true` to disable filesystem writes | `false` | @@ -252,6 +253,8 @@ headroom proxy --learn --min-evidence 3 | `HEADROOM_BETA_HEADER_STICKY` | Controls per-session `anthropic-beta` / `OpenAI-Beta` re-echo. `enabled` (default): the proxy unions beta tokens across turns within a session — if the client sends a token in turn N and omits it in turn N+1, the proxy re-injects it to preserve prefix-cache stability. `disabled`: the client's value is forwarded verbatim with no accumulation. Any other value raises at request time. See [Session Beta Header Tracking](/docs/configuration#session-beta-header-tracking). | `enabled` | | `HEADROOM_BETA_TRACKER_MAX_SESSIONS` | LRU capacity of the in-memory session beta tracker. Once full, the oldest session entry is evicted. | `1000` | +For provider-only proxying, prefer `HEADROOM_HTTP_PROXY` over process-wide variables such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY`. HTTPX reads those global variables, but Headroom also passes them through to tool executions. + ### Session Beta Header Tracking When running as a proxy, Headroom maintains a per-session union of `anthropic-beta` (and `OpenAI-Beta`) tokens via `SessionBetaTracker`. The session key is derived from the `x-headroom-session-id` header if present, otherwise from `md5(model + system_prompt[:500])[:16]` — stable across turns of the same conversation. diff --git a/docs/content/docs/proxy.mdx b/docs/content/docs/proxy.mdx index 4bd675c1a..2f619d5b9 100644 --- a/docs/content/docs/proxy.mdx +++ b/docs/content/docs/proxy.mdx @@ -34,6 +34,7 @@ Telemetry is **off by default** (opt-in). Opt in with `HEADROOM_TELEMETRY=on` or | `--limit-concurrency` | `1000` | Maximum concurrent connections before Uvicorn returns 503 | | `--max-connections` | `500` | Maximum upstream HTTP connections | | `--max-keepalive` | `100` | Maximum upstream keep-alive connections | +| `--http-proxy` | None | HTTP proxy URL for upstream provider requests only; HTTPS provider APIs use CONNECT | | `--mode` | `token` | Optimization mode: `token` prioritizes compression, `cache` preserves provider prefix-cache stability | | `--no-optimize` | `false` | Disable optimization (passthrough mode) | | `--no-cache` | `false` | Disable semantic caching | @@ -50,6 +51,14 @@ Telemetry is **off by default** (opt-in). Opt in with `HEADROOM_TELEMETRY=on` or | `--no-telemetry` | `false` | Force anonymous telemetry off (already the default) | | `--stateless` | `false` | Disable filesystem writes and keep runtime state in memory | +Use `--http-proxy` or `HEADROOM_HTTP_PROXY` when only provider API traffic should go through a proxy: + +```bash +headroom proxy --http-proxy http://proxy.internal:8080 +``` + +Avoid setting process-wide variables such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY` for this use case. HTTPX reads those variables too, but Headroom also inherits them into tool executions, so they can proxy unrelated tool traffic. + ### Context management | Option | Default | Description | diff --git a/headroom/cli/proxy.py b/headroom/cli/proxy.py index c537309ae..eab35a381 100644 --- a/headroom/cli/proxy.py +++ b/headroom/cli/proxy.py @@ -198,6 +198,15 @@ def dashboard(port: int, no_open: bool) -> None: "(SSLV3_ALERT_BAD_RECORD_MAC) when many concurrent streams are cancelled." ), ) +@click.option( + "--http-proxy", + default=None, + envvar="HEADROOM_HTTP_PROXY", + help=( + "HTTP proxy URL for upstream provider requests only " + "(HTTPS uses CONNECT; env: HEADROOM_HTTP_PROXY)." + ), +) @click.option( "--keepalive-expiry", "keepalive_expiry", @@ -845,6 +854,7 @@ def proxy( max_keepalive_connections: int, keepalive_expiry: float, http2: bool, + http_proxy: str | None, intercept_tool_results: bool, no_optimize: bool, no_cache: bool, @@ -1126,6 +1136,7 @@ def proxy( max_keepalive_connections=max_keepalive_connections, keepalive_expiry=keepalive_expiry, http2=http2, + http_proxy=http_proxy, log_file=None if is_stateless else log_file, log_full_messages=log_messages or os.environ.get("HEADROOM_LOG_MESSAGES", "").lower() in ("true", "1", "yes", "on"), diff --git a/headroom/proxy/models.py b/headroom/proxy/models.py index 34e9ec974..53088756b 100644 --- a/headroom/proxy/models.py +++ b/headroom/proxy/models.py @@ -287,6 +287,7 @@ class ProxyConfig: max_keepalive_connections: int = 100 keepalive_expiry: float = 90.0 http2: bool = True + http_proxy: str | None = None # Memory System memory_enabled: bool = False diff --git a/headroom/proxy/server.py b/headroom/proxy/server.py index 9bb228f58..a771360c9 100644 --- a/headroom/proxy/server.py +++ b/headroom/proxy/server.py @@ -597,6 +597,29 @@ def _apply_stateless_persistence(config: ProxyConfig) -> None: get_toin(TOINConfig(storage_path="")) +def _provider_httpx_client_options( + config: ProxyConfig, + verify: Any, +) -> tuple[bool, dict[str, Any]]: + client_kwargs: dict[str, Any] = { + "timeout": httpx.Timeout( + connect=config.connect_timeout_seconds, + read=config.request_timeout_seconds, + write=config.request_timeout_seconds, + pool=config.connect_timeout_seconds, + ), + "limits": httpx.Limits( + max_connections=config.max_connections, + max_keepalive_connections=config.max_keepalive_connections, + keepalive_expiry=config.keepalive_expiry, + ), + "verify": verify, + } + if config.http_proxy: + client_kwargs["proxy"] = config.http_proxy + return config.http2 and not config.http_proxy, client_kwargs + + class HeadroomProxy( StreamingMixin, AnthropicHandlerMixin, @@ -1372,27 +1395,12 @@ class HeadroomProxy( # is configured, else a strict-relaxed default context when # HEADROOM_TLS_STRICT=0, else httpx's default strict verification. _verify = build_httpx_verify() - _client_kwargs: dict[str, Any] = { - "timeout": httpx.Timeout( - connect=self.config.connect_timeout_seconds, - read=self.config.request_timeout_seconds, - write=self.config.request_timeout_seconds, - pool=self.config.connect_timeout_seconds, - ), - "limits": httpx.Limits( - max_connections=self.config.max_connections, - max_keepalive_connections=self.config.max_keepalive_connections, - keepalive_expiry=self.config.keepalive_expiry, - ), - "verify": _verify, - } - self.http_client = httpx.AsyncClient(http2=self.config.http2, **_client_kwargs) + _http2, _client_kwargs = _provider_httpx_client_options(self.config, _verify) + self.http_client = httpx.AsyncClient(http2=_http2, **_client_kwargs) # Reuse the primary client when HTTP/2 is already off; otherwise keep a # dedicated HTTP/1.1 client for ChatGPT passthrough. self.http_client_h1 = ( - self.http_client - if not self.config.http2 - else httpx.AsyncClient(http2=False, **_client_kwargs) + self.http_client if not _http2 else httpx.AsyncClient(http2=False, **_client_kwargs) ) logger.info("Headroom Proxy started") logger.info(f"Optimization: {'ENABLED' if self.config.optimize else 'DISABLED'}") @@ -1411,7 +1419,7 @@ class HeadroomProxy( logger.info( f"Connection Pool: max_connections={self.config.max_connections}, " f"max_keepalive={self.config.max_keepalive_connections}, " - f"http2={'ENABLED' if self.config.http2 else 'DISABLED'}" + f"http2={'ENABLED' if _http2 else 'DISABLED'}" ) # Unit 4 pre-upstream concurrency announcement. Report the resolved @@ -4189,6 +4197,7 @@ def _proxy_config_from_env() -> ProxyConfig: max_keepalive_connections=_get_env_int("HEADROOM_MAX_KEEPALIVE", 100), keepalive_expiry=_get_env_float("HEADROOM_KEEPALIVE_EXPIRY", 90.0), http2=_get_env_bool("HEADROOM_HTTP2", True), + http_proxy=os.environ.get("HEADROOM_HTTP_PROXY") or None, periodic_toin_stats_enabled=_get_env_bool("HEADROOM_PERIODIC_TOIN_STATS", True), proxy_token=os.environ.get("HEADROOM_PROXY_TOKEN") or None, offline=_get_env_bool("HEADROOM_OFFLINE", False), @@ -4246,7 +4255,7 @@ def run_server( # Format connection pool info pool_info = f"max={config.max_connections}, keepalive={config.max_keepalive_connections}" - http2_status = "ENABLED" if config.http2 else "DISABLED" + http2_status = "ENABLED" if (config.http2 and not config.http_proxy) else "DISABLED" backend_status = format_backend_status( backend=config.backend, @@ -4580,6 +4589,10 @@ if __name__ == "__main__": action="store_true", help="Disable HTTP/2 (enabled by default for better throughput)", ) + parser.add_argument( + "--http-proxy", + help=("HTTP proxy URL for upstream provider requests only (env: HEADROOM_HTTP_PROXY)"), + ) parser.add_argument( "--workers", type=int, @@ -4832,6 +4845,7 @@ if __name__ == "__main__": max_keepalive_connections=_get_env_int("HEADROOM_MAX_KEEPALIVE", args.max_keepalive), keepalive_expiry=_get_env_float("HEADROOM_KEEPALIVE_EXPIRY", args.keepalive_expiry), http2=not args.no_http2 and _get_env_bool("HEADROOM_HTTP2", True), + http_proxy=_get_env_str("HEADROOM_HTTP_PROXY", args.http_proxy or "") or None, read_maturation=_get_env_bool("HEADROOM_READ_MATURATION", False), read_maturation_quiesce_turns=_get_env_int("HEADROOM_READ_MATURATION_QUIESCE_TURNS", 5), read_maturation_max_hold_turns=_get_env_int("HEADROOM_READ_MATURATION_MAX_HOLD_TURNS", 25), diff --git a/tests/test_cli_proxy_improvements.py b/tests/test_cli_proxy_improvements.py index 927088710..c10263d26 100644 --- a/tests/test_cli_proxy_improvements.py +++ b/tests/test_cli_proxy_improvements.py @@ -89,6 +89,38 @@ class TestLearnNoLearnConflict: assert cfg.traffic_learning_enabled is False +class TestHttpProxyOption: + """--http-proxy should configure only the provider HTTPX clients.""" + + def test_http_proxy_cli_flag(self, runner: CliRunner, mock_run_server: dict) -> None: + result = runner.invoke( + main, + ["proxy", "--http-proxy", "http://proxy.local:8080"], + catch_exceptions=False, + ) + assert result.exit_code == 0, result.output + assert mock_run_server["config"].http_proxy == "http://proxy.local:8080" + + def test_http_proxy_env_var(self, runner: CliRunner, mock_run_server: dict) -> None: + result = runner.invoke( + main, + ["proxy"], + env={"HEADROOM_HTTP_PROXY": "http://proxy.local:8080"}, + catch_exceptions=False, + ) + assert result.exit_code == 0, result.output + assert mock_run_server["config"].http_proxy == "http://proxy.local:8080" + + def test_direct_server_env_http_proxy(self, monkeypatch: pytest.MonkeyPatch) -> None: + import headroom.proxy.server as server_mod + + monkeypatch.delenv(server_mod._MULTI_WORKER_CONFIG_ENV, raising=False) + monkeypatch.setenv("HEADROOM_HTTP_PROXY", "http://proxy.local:8080") + + config = server_mod._proxy_config_from_env() + assert config.http_proxy == "http://proxy.local:8080" + + class TestSubscriptionPollIntervalValidation: """--subscription-poll-interval should reject values outside 1-3600.""" diff --git a/tests/test_proxy_scalability.py b/tests/test_proxy_scalability.py index 864386f7e..1cd6a44a7 100644 --- a/tests/test_proxy_scalability.py +++ b/tests/test_proxy_scalability.py @@ -250,7 +250,12 @@ class TestWorkerConfiguration: from headroom.proxy.server import _MULTI_WORKER_CONFIG_ENV, run_server captured = {} - config = ProxyConfig(host="0.0.0.0", port=8787, max_connections=200) + config = ProxyConfig( + host="0.0.0.0", + port=8787, + max_connections=200, + http_proxy="http://proxy.local:8080", + ) def fake_run(app, **kwargs): captured["app"] = app @@ -270,6 +275,7 @@ class TestWorkerConfiguration: assert payload["host"] == "0.0.0.0" assert payload["port"] == 8787 assert payload["max_connections"] == 200 + assert payload["http_proxy"] == "http://proxy.local:8080" finally: # run_server sets this via raw os.environ. Pop it directly rather # than via monkeypatch.delenv: delenv records the current (JSON) @@ -312,3 +318,28 @@ class TestWorkerConfiguration: server_mod.run_server(ProxyConfig(), print_banner=False) assert "loop" not in captured["kwargs"] + + +class TestProviderHttpClientOptions: + """Provider HTTPX options should keep proxy settings scoped to provider clients.""" + + def test_default_http2_preserved_without_proxy(self): + from headroom.proxy.models import ProxyConfig + from headroom.proxy.server import _provider_httpx_client_options + + http2, kwargs = _provider_httpx_client_options(ProxyConfig(http2=True), verify=True) + + assert http2 is True + assert "proxy" not in kwargs + + def test_http_proxy_sets_proxy_and_forces_http1(self): + from headroom.proxy.models import ProxyConfig + from headroom.proxy.server import _provider_httpx_client_options + + http2, kwargs = _provider_httpx_client_options( + ProxyConfig(http2=True, http_proxy="http://proxy.local:8080"), + verify=True, + ) + + assert http2 is False + assert kwargs["proxy"] == "http://proxy.local:8080"