From 5cece7bf587c38521f01951b96847892ba59ffe4 Mon Sep 17 00:00:00 2001 From: iotron <50877415+iotron@users.noreply.github.com> Date: Tue, 14 Jul 2026 09:12:34 +0530 Subject: [PATCH] fix(proxy): Strip Codex responses-lite marker from response.create frame body (#1820) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Description Codex CLI mirrors the `X-OpenAI-Internal-Codex-Responses-Lite` request header into the `response.create` WS frame body under `client_metadata.ws_request_header_x_openai_internal_codex_responses_lite`. The existing fix strips the header itself (both the HTTP fallback path and the WS handshake path) but never touched this frame-body copy, so Headroom still forwarded it to `wss://chatgpt.com/backend-api/codex/responses` unmodified. Upstream rejects `gpt-5.x` models whenever that field is truthy, so every Codex-through-Headroom turn on gpt-5.5/5.4 failed with a 400, even on builds that already contain the header-strip fix. Closes #1523 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/openai.py`: add `_strip_codex_lite_metadata()`, which removes `client_metadata.ws_request_header_x_openai_internal_codex_responses_lite` from a `response.create` frame body (checks both the flat-body shape and the `{"response": {...}}` envelope shape Codex uses depending on call path). Fail-safe no-op on non-JSON payloads or when the key is absent. - Wired the helper into both WS-forwarder send sites in the same file: the initial `first_msg_raw` send and the steady-state `_client_to_upstream` relay send. - `tests/test_openai_codex_ws_lifecycle.py`: added `test_ws_first_frame_strips_codex_lite_metadata_mirror`, which sends a `response.create` frame carrying the mirror key through the handler and asserts the frame actually forwarded to the fake upstream has the key removed while sibling `client_metadata` fields (e.g. `thread_id`) survive. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_ws_lifecycle.py All checks passed! $ mypy headroom/proxy/handlers/openai.py Success: no issues found (pre-existing unrelated notes in headroom/proxy/server.py only) $ pytest tests/test_openai_codex_ws_lifecycle.py tests/test_openai_codex_routing.py tests/test_codex_openai_contract_parity.py -q 52 passed, 23 warnings in 1.66s # Proof the new test actually catches the bug (checked out the parent commit's # openai.py, i.e. pre-fix, with the new test present): $ pytest tests/test_openai_codex_ws_lifecycle.py -q -k strips_codex_lite_metadata FAILED tests/test_openai_codex_ws_lifecycle.py::test_ws_first_frame_strips_codex_lite_metadata_mirror AssertionError: assert 'ws_request_header_x_openai_internal_codex_responses_lite' not in {'thread_id': 't_1', 'ws_request_header_x_openai_internal_codex_responses_lite': True} 1 failed, 26 deselected in 0.63s # Restored the fix -> same test passes (see "52 passed" run above). ``` ## Real Behavior Proof - Environment: macOS, Python 3.13, headroom-ai 0.30.0 (pip-installed copy patched identically to this diff), Codex CLI 0.142.5, ChatGPT Plus subscription auth, model `gpt-5.5`. - Exact command / steps: `headroom wrap codex` (proxy on 127.0.0.1:8787) → `codex exec "..."` with `model_provider = "headroom"` in `~/.codex/config.toml`. Also reproduced deterministically via a throwaway debug proxy instance with `HEADROOM_CODEX_WIRE_DEBUG=1` wire capture, isolated from the live account/session. - Observed result: pre-patch, deterministic `400 unsupported_value` / "This model is not supported when using X-OpenAI-Internal-Codex-Responses-Lite" on every single turn (reproduced repeatedly, across a header-only-stripped build). Post-patch (installed copy with this exact diff): clean full completions streamed to `response.completed` with zero error frames via wire capture, and — after restarting the live production proxy to load the patched module — the user confirmed `codex` working normally end-to-end through Headroom on `gpt-5.5` in real day-to-day use, not just the isolated repro. - Not tested: the HTTP (non-WS) fallback path for Codex — that path doesn't carry a `response.create` frame body in the same way, and the existing header-strip logic already covers it; no upstream 400s observed there in this investigation. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — backend WS proxy fix, no UI surface. ## Additional Notes - CHANGELOG.md / docs left untouched: this is a narrowly-scoped bugfix to existing (undocumented-at-user-level) WS forwarding internals; happy to add a CHANGELOG entry if maintainers want one. - Full root-cause writeup with wire-capture details is on the issue: https://github.com/headroomlabs-ai/headroom/issues/1523#issuecomment-4887989873 - Did not run the full repo-wide `pytest`/`mypy headroom` (whole package) — scoped to the modified file and the three most relevant existing test modules (`test_openai_codex_ws_lifecycle.py`, `test_openai_codex_routing.py`, `test_codex_openai_contract_parity.py`), all passing. Glad to run the full suite if a maintainer wants that in CI instead. Co-authored-by: JerrettDavis Co-authored-by: Tejas Chopra --- headroom/proxy/handlers/openai.py | 31 ++++++++++++++++-- tests/test_openai_codex_ws_lifecycle.py | 43 +++++++++++++++++++++++++ 2 files changed, 72 insertions(+), 2 deletions(-) diff --git a/headroom/proxy/handlers/openai.py b/headroom/proxy/handlers/openai.py index 2eb4d0d65..effb7b982 100644 --- a/headroom/proxy/handlers/openai.py +++ b/headroom/proxy/handlers/openai.py @@ -95,6 +95,33 @@ def _codex_ws_compression_timeout_seconds() -> float: _WS_ALLOWED_ORIGINS_ENV = "HEADROOM_WS_ORIGINS" _CORS_ALLOWED_ORIGINS_ENV = "HEADROOM_CORS_ORIGINS" _CODEX_RESPONSES_LITE_HEADER = "x-openai-internal-codex-responses-lite" +# Codex mirrors the responses-lite request header into the response.create +# frame body under client_metadata; upstream rejects gpt-5.x when it is +# truthy. Stripping the WS handshake header alone is insufficient +# (headroomlabs-ai/headroom#1523) — the frame-body mirror must be removed too. +_CODEX_LITE_METADATA_KEY = "ws_request_header_x_openai_internal_codex_responses_lite" + + +def _strip_codex_lite_metadata(raw_msg: str) -> str: + """Remove the Codex responses-lite marker mirrored into a response.create + frame's client_metadata. Fail-safe: returns raw_msg unchanged on any + parse issue or when the marker is absent.""" + try: + frame = json.loads(raw_msg) + except (json.JSONDecodeError, TypeError, ValueError): + return raw_msg + if not isinstance(frame, dict): + return raw_msg + changed = False + for container in (frame, frame.get("response")): + if isinstance(container, dict): + cm = container.get("client_metadata") + if isinstance(cm, dict) and _CODEX_LITE_METADATA_KEY in cm: + del cm[_CODEX_LITE_METADATA_KEY] + changed = True + return json.dumps(frame) if changed else raw_msg + + _OPENAI_CHAT_COMPLETIONS_PATH = "/chat/completions" _OPENAI_RESPONSES_PATH = "/responses" _OPENAI_ORIGINAL_PATH_HEADER = "x-headroom-original-path" @@ -5838,7 +5865,7 @@ class OpenAIHandlerMixin: if ws_connected: async with upstream: - await upstream.send(first_msg_raw) + await upstream.send(_strip_codex_lite_metadata(first_msg_raw)) # Unit 3: flag the upstream side flips on seeing # ``response.completed`` so the outer cause @@ -6211,7 +6238,7 @@ class OpenAIHandlerMixin: "transforms_applied": transforms_applied, }, ) - await upstream.send(msg) + await upstream.send(_strip_codex_lite_metadata(msg)) except asyncio.CancelledError: # Explicit cancel from the outer # orchestrator — re-raise so diff --git a/tests/test_openai_codex_ws_lifecycle.py b/tests/test_openai_codex_ws_lifecycle.py index a7ac5d915..5b1a5bc0a 100644 --- a/tests/test_openai_codex_ws_lifecycle.py +++ b/tests/test_openai_codex_ws_lifecycle.py @@ -1080,6 +1080,49 @@ async def test_ws_without_codex_lite_preserves_adjacent_headers_and_api_key_rout assert "ChatGPT-Account-ID" not in forwarded_headers +@pytest.mark.asyncio +async def test_ws_first_frame_strips_codex_lite_metadata_mirror(): + """Codex mirrors the lite header into response.create's client_metadata + (regression for #1523): stripping the handshake header alone is not + enough, upstream rejects gpt-5.x when the frame-body mirror survives. + """ + upstream_events = [ + json.dumps({"type": "response.created", "response": {"id": "r_1"}}), + json.dumps({"type": "response.completed", "response": {"id": "r_1"}}), + ] + upstream = _FakeUpstream(upstream_events) + fake_ws_mod = _make_fake_websockets_module(upstream) + + first_frame = json.dumps( + { + "type": "response.create", + "response": { + "model": "gpt-5.5", + "input": "hi", + "client_metadata": { + "thread_id": "t_1", + "ws_request_header_x_openai_internal_codex_responses_lite": True, + }, + }, + } + ) + client_ws = _FakeWebSocket( + frames=[first_frame], + headers=_codex_lite_headers(chatgpt=True), + ) + handler = _DummyOpenAIHandler() + + with patch.dict(sys.modules, {"websockets": fake_ws_mod}): + await handler.handle_openai_responses_ws(client_ws) + + assert len(upstream.sent) == 1 + sent_body = json.loads(upstream.sent[0]) + client_metadata = sent_body["response"]["client_metadata"] + assert "ws_request_header_x_openai_internal_codex_responses_lite" not in client_metadata + # Sibling metadata must survive the strip. + assert client_metadata["thread_id"] == "t_1" + + @pytest.mark.asyncio async def test_ws_connect_happens_before_accept(): """The upstream connect must complete before the client 101 is sent,