From 7ff842da170b5bceb5d67048473eeb8a18e09a51 Mon Sep 17 00:00:00 2001 From: Abhay Singh Date: Tue, 7 Jul 2026 21:57:54 +0530 Subject: [PATCH] fix(proxy/openai): thread savings-profile kwargs into chat completions (#1606) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Description OpenAI-compatible `/v1/chat/completions` requests didn't receive the same proxy savings/profile kwargs as the other compression paths. The live chat handler (`handle_openai_chat` in `headroom/proxy/handlers/openai.py`) called `openai_pipeline.apply()` with only `model_limit` / `context` / `frozen_message_count` / `biases` / `compression_policy` — it never passed `proxy_pipeline_kwargs(self.config)`. So when the proxy runs with `HEADROOM_SAVINGS_PROFILE=agent-90`, the effective config reports user/system-message compression and `target_ratio=0.10`, but the real chat path silently dropped all of it. OpenAI-compatible clients such as OpenCode kept protecting user messages and missed the configured profile. For contrast, `handlers/anthropic.py` passes `**proxy_pipeline_kwargs(self.config)` to every `apply()` call, and so does the dedicated OpenAI compress endpoint in this same module — only the two chat-completions `apply()` sites were missing it. Closes #1534 ## Fix Add `**proxy_pipeline_kwargs(self.config)` to both chat-path `apply()` calls (the token-mode branch and the non-token branch): ```python lambda: self.openai_pipeline.apply( messages=messages, model=model, model_limit=context_limit, context=extract_user_query(messages), frozen_message_count=openai_frozen_count, biases=_hook_biases, compression_policy=compression_policy, **proxy_pipeline_kwargs(self.config), # ← added ) ``` `proxy_pipeline_kwargs` is already imported in the module and is the exact helper the Anthropic handler and the OpenAI compress endpoint use, so the chat path now matches them. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/proxy/handlers/openai.py`: pass `**proxy_pipeline_kwargs(self.config)` on both `apply()` call sites in `handle_openai_chat` (token-mode and non-token branches). - `tests/test_proxy/test_openai_chat_savings_profile.py`: new regression test driving the chat handler with `savings_profile="agent-90"` and asserting the profile knobs reach `apply()`. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output The new test drives the real chat handler through the `create_app` + `TestClient` harness with a recording `apply()` stub. Before the fix it captures exactly the five kwargs the issue describes (no profile knobs); after the fix the profile knobs are present: ```text # before the fix (openai.py reverted, test kept) E AssertionError: assert None is True E + where None = {...}.get('compress_user_messages') # captured kwargs were: biases, compression_policy, messages, model, # model_limit, context, frozen_message_count — no profile knobs FAILED tests/test_proxy/test_openai_chat_savings_profile.py::test_chat_completions_threads_savings_profile_kwargs_into_apply # after the fix tests\test_proxy\test_openai_chat_savings_profile.py . ======================== 1 passed, 1 warning in 39.44s ======================== ``` No regression in the existing chat backend-path suite: ```text $ uv run pytest tests/test_proxy/test_openai_backend_path.py ======================== 5 passed, 1 warning in 15.78s ======================== $ uv run ruff check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_chat_savings_profile.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom built from this branch (`uv sync --extra dev`), proxy config `savings_profile="agent-90"`, `optimize=True`, `backend="anyllm"` with a mocked OpenAI upstream. - Exact command / steps: started the app with `create_app(config)`, replaced `proxy.openai_pipeline.apply` with a recording stub, and POSTed a real `/v1/chat/completions` request with a large user message so the compression decision fires. Inspected the kwargs the handler actually passed to `apply()`. - Observed result: before the fix the recorded `apply()` kwargs were `{biases, compression_policy, messages, model, model_limit, context, frozen_message_count}` — no profile knobs. After the fix the same call also carries `compress_user_messages=True`, `compress_system_messages=True`, `target_ratio=0.10`, `min_tokens_to_compress=120` (the agent-90 profile), matching the issue's "Expected". - Not tested: did not stand up a real OpenAI/OpenCode upstream end-to-end (no live key in this environment); the upstream is mocked and the assertion is on the kwargs the proxy threads into the compression pipeline, which is exactly what the bug was about. Did not run the full `mypy headroom` pass (two-line kwarg addition, no new types). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Two-line change plus comments; no new dependencies. Reuses the existing `proxy_pipeline_kwargs` helper, so behavior is consistent across Anthropic, the OpenAI compress endpoint, and now the OpenAI chat path. - @chopratejas flagging you for review — this aligns the OpenAI chat path with the savings-profile handling the other providers already had. Co-authored-by: JD Davis --- CHANGELOG.md | 1 + headroom/proxy/handlers/openai.py | 12 ++ .../test_openai_chat_savings_profile.py | 106 ++++++++++++++++++ 3 files changed, 119 insertions(+) create mode 100644 tests/test_proxy/test_openai_chat_savings_profile.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 8dcc624a8..d44291c26 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -108,6 +108,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Bug Fixes +* **proxy/openai:** thread the savings-profile kwargs into the live `/v1/chat/completions` compression path. The chat handler called `openai_pipeline.apply()` without `proxy_pipeline_kwargs(config)`, so `HEADROOM_SAVINGS_PROFILE=agent-90` (and the individual `compress_user_messages`/`target_ratio`/`min_tokens_to_compress`/... knobs) were silently dropped — OpenAI-compatible clients like OpenCode kept protecting user messages and missed the configured profile. Both the token-mode and non-token chat branches now pass the profile kwargs, matching `handlers/anthropic.py` and the dedicated OpenAI compress endpoint ([#1534](https://github.com/headroomlabs-ai/headroom/issues/1534)). * **proxy:** forward Codex Desktop `/v1/responses` posts byte-faithfully so they stop returning upstream `400 {"detail":"Bad Request"}`. `handle_openai_responses` decoded the inbound body to inspect it but always re-serialized a canonical body on the way out, and it never stripped the inbound `content-encoding` header — so a `content-encoding: zstd` Codex Desktop request was forwarded as already-decoded JSON still advertising `zstd`, and the upstream ChatGPT Codex endpoint rejected it. The handler now keeps the original decoded bytes and forwards them verbatim whenever nothing (compression or memory injection) mutated the request, and drops the stale `content-encoding` header, mirroring the byte-faithful passthrough the chat and Anthropic paths already use ([#1542](https://github.com/headroomlabs-ai/headroom/issues/1542)). * **wrap/codex:** `headroom unwrap codex` now removes the Headroom rtk instruction block from the Codex global `AGENTS.md`. `wrap codex` injects it there, but unwrap only restored `config.toml` and MCP state, so a plain `codex` launch kept following the "prefix shell commands with `rtk`" guidance and failed once the managed rtk binary was off PATH. Unwrap now strips the marker-fenced block (preserving the rest of the file), mirroring `unwrap copilot` ([#1421](https://github.com/headroomlabs-ai/headroom/issues/1421)). * **proxy/auth:** classify real Anthropic OAuth tokens correctly. `classify_auth_mode` matched OAuth on the `sk-ant-oat-` prefix, but real access tokens are `sk-ant-oat01-...` (a version number, no dash after `oat`), so every real subscription/OAuth token fell through to the `sk-` branch and was tagged `PAYG` — enabling aggressive lossy compression, auto `cache_control`, and `prompt_cache_key` injection on subscription-bound requests the classifier is meant to route to the passthrough-prefer path. The prefix is now the dash-less `sk-ant-oat` (still matches the legacy dashed shape). The existing parity tests only passed because they used a synthetic `sk-ant-oat-01-` fixture; a regression test now covers the real `sk-ant-oat01-` format. diff --git a/headroom/proxy/handlers/openai.py b/headroom/proxy/handlers/openai.py index 4f38810aa..24515188a 100644 --- a/headroom/proxy/handlers/openai.py +++ b/headroom/proxy/handlers/openai.py @@ -2211,6 +2211,14 @@ class OpenAIHandlerMixin: frozen_message_count=openai_frozen_count, biases=_hook_biases, compression_policy=compression_policy, + # Thread the savings-profile knobs (e.g. + # HEADROOM_SAVINGS_PROFILE=agent-90) onto the live + # chat-completions path, matching handlers/ + # anthropic.py and the dedicated OpenAI compress + # endpoint. Without this the profile's + # compress_user_messages/target_ratio/etc. were + # silently dropped here (#1534). + **proxy_pipeline_kwargs(self.config), ), timeout=COMPRESSION_TIMEOUT_SECONDS, ) @@ -2235,6 +2243,10 @@ class OpenAIHandlerMixin: frozen_message_count=openai_frozen_count, biases=_hook_biases, compression_policy=compression_policy, + # Same savings-profile threading as the token-mode + # branch above — the non-token chat path must honor + # the configured profile too (#1534). + **proxy_pipeline_kwargs(self.config), ), timeout=COMPRESSION_TIMEOUT_SECONDS, ) diff --git a/tests/test_proxy/test_openai_chat_savings_profile.py b/tests/test_proxy/test_openai_chat_savings_profile.py new file mode 100644 index 000000000..3cf8ddb2a --- /dev/null +++ b/tests/test_proxy/test_openai_chat_savings_profile.py @@ -0,0 +1,106 @@ +"""Regression test for #1534. + +The live OpenAI `/v1/chat/completions` compression path must thread the proxy +savings-profile kwargs (``proxy_pipeline_kwargs(config)``) into +``openai_pipeline.apply`` — the same way ``handlers/anthropic.py`` and the +dedicated OpenAI compress endpoint do. Before the fix the chat path only passed +``model_limit``/``context``/``frozen_message_count``/``biases``/ +``compression_policy``, so ``HEADROOM_SAVINGS_PROFILE=agent-90`` (and other +profile knobs) were silently dropped on the real chat path. +""" + +from __future__ import annotations + +from types import SimpleNamespace +from unittest.mock import AsyncMock, MagicMock, patch + +import pytest + +fastapi = pytest.importorskip("fastapi") +pytest.importorskip("httpx") + +from fastapi.testclient import TestClient # noqa: E402 + +from headroom.backends.base import BackendResponse # noqa: E402 +from headroom.proxy.server import ProxyConfig, create_app # noqa: E402 + + +def _make_mock_backend() -> MagicMock: + backend = MagicMock() + backend.name = "anyllm-openai" + backend.send_openai_message = AsyncMock( + return_value=BackendResponse( + body={ + "id": "chatcmpl-1", + "object": "chat.completion", + "model": "gpt-4o", + "choices": [ + { + "index": 0, + "message": {"role": "assistant", "content": "ok"}, + "finish_reason": "stop", + } + ], + "usage": {"prompt_tokens": 100, "completion_tokens": 2, "total_tokens": 102}, + }, + status_code=200, + headers={"content-type": "application/json"}, + ) + ) + return backend + + +def test_chat_completions_threads_savings_profile_kwargs_into_apply(): + """With HEADROOM_SAVINGS_PROFILE=agent-90, the chat path must pass the + profile knobs (compress_user_messages, target_ratio, ...) to apply().""" + config = ProxyConfig( + optimize=True, + cache_enabled=False, + rate_limit_enabled=False, + backend="anyllm", + anyllm_provider="openai", + savings_profile="agent-90", + ) + + captured: dict[str, object] = {} + + def recording_apply(**kwargs): + captured.update(kwargs) + sent = kwargs["messages"] + return SimpleNamespace( + messages=sent, + transforms_applied=[], + timing={}, + tokens_before=4000, + tokens_after=400, + waste_signals=None, + ) + + # A large user message so the compression decision actually fires. + big = "word " * 4000 + + mock_backend = _make_mock_backend() + with patch("headroom.proxy.server.AnyLLMBackend", return_value=mock_backend): + app = create_app(config) + with TestClient(app) as client: + proxy = client.app.state.proxy + proxy.openai_pipeline.apply = MagicMock(side_effect=recording_apply) + + resp = client.post( + "/v1/chat/completions", + json={ + "model": "gpt-4o", + "messages": [{"role": "user", "content": big}], + "stream": False, + }, + headers={"Authorization": "Bearer test-key"}, + ) + + assert resp.status_code == 200, resp.text + assert proxy.openai_pipeline.apply.call_count >= 1, "compression apply() never ran" + + # The agent-90 profile knobs must be present on the apply() call. + assert captured.get("compress_user_messages") is True + assert captured.get("target_ratio") == 0.10 + assert captured.get("min_tokens_to_compress") == 120 + assert captured.get("compress_system_messages") is True