2 commits

Author	SHA1	Message	Date
Semianchuk Vitalii	63f74aa3e6	fix: replace computer_call_output with apply_patch_call_output in output_shaper (#2250) ... The _RESPONSES_TOOL_OUTPUT_TYPES frozenset in output_shaper.py had computer_call_output instead of apply_patch_call_output, making it inconsistent with the canonical definitions in handlers/openai.py and output_turn_policy.py. This caused apply_patch_call_output items to be misclassified, preventing effort routing optimization for those turns. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>	2026-07-15 21:33:25 +00:00
alex33d	71cbb6aaad	feat(proxy): extend output shaping to the OpenAI Responses path (Codex HTTP + WS) (#1943) ... ## Description Output shaping (`HEADROOM_OUTPUT_SHAPER`) so far only runs on the Anthropic `/v1/messages` path (`shape_request` is called only from `handlers/anthropic.py`). Codex traffic over `/v1/responses` — HTTP and WebSocket — is never shaped, so subscription Codex users get no output-token reduction. On a fleet where Codex is the majority of traffic, that's the largest unshaped output-token pool. This ports both output-shaping levers to the OpenAI Responses format with the same contracts as the Anthropic path, wired at the single funnel all three call paths already share. Closes # ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `output_shaper.py` — Responses-format counterparts of the existing levers: - `classify_responses_turn()`: structural turn classifier over the `input` item list. The trailing run of tool-output items (`function_call_output`, `custom_tool_call_output`, `local_shell_call_output`, `computer_call_output`) is a mechanical continuation; a trailing user message is a new ask. Error detection is structural JSON fields only (`exit_code`/`success`/`error`, incl. the common `{"output":…, "metadata":{…}}` nesting) — never prose — mirroring the Anthropic `is_error` handling so error turns keep full effort. - `apply_responses_verbosity_steering()`: appends the byte-stable steering block to the tail of the `instructions` string. Idempotent per level, replaced in place on level change — within a conversation every shaped turn sends identical `instructions` bytes, so the provider prefix cache stays hot after the first shaped turn (same contract as the Anthropic system-tail append). - `route_responses_effort()`: lowers an explicitly-present `reasoning.effort` on mechanical continuations only. Never injects `reasoning`, never raises an effort, leaves new asks and error continuations untouched. Responses gets its own rank table (`minimal` floor). - `shape_responses_request()`: the `shape_request` counterpart (same settings, labels, level-resolution contract). - `output_savings.py` — `conversation_key_from_responses_body()`: conversation-stable holdout key (model + first user input text) so whole conversations land in one A/B arm. - `handlers/openai.py` — `_shape_openai_responses_payload()` (module helper, never raises) called inside `_compress_openai_responses_payload_in_executor`'s closure — the single funnel for HTTP `/v1/responses`, the WS first frame, and WS subsequent frames. Runs before compression so the classifier sees the client's input as sent; serialization stays off the event loop. Shaper labels ride the existing transforms channel so `outcome.py record_from_labels` feeds the output-savings ledger unchanged. The `modified` flag is forced only when shaping actually mutated the payload — an unshaped control-arm request never breaks byte-faithful forwarding. - `tests/test_output_shaper_responses.py` — 40 tests covering the classifier (incl. error sniff + prose-never-inspected), steering (idempotency, level change, byte stability, non-string instructions), effort routing (never-inject/never-raise, error/new-ask untouched), conversation key stability, and the handler helper (disabled/treatment/full-holdout arms). Off by default; same env gates as the Anthropic path, all hot-reloadable via `/admin/runtime-env`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_output_shaper.py tests/test_output_shaper_responses.py \ tests/test_output_savings.py tests/test_verbosity_controller.py \ tests/test_verbosity_learn.py tests/test_codex_openai_contract_parity.py \ tests/test_codex_responses_waste_signals.py -q 154 passed $ ruff check headroom/proxy/output_shaper.py headroom/proxy/output_savings.py \ headroom/proxy/handlers/openai.py tests/test_output_shaper_responses.py All checks passed! $ mypy headroom/proxy/output_shaper.py headroom/proxy/output_savings.py --ignore-missing-imports exit 0 ``` ## Real Behavior Proof - Environment: macOS, Python 3.13, proxy run from this branch (`PYTHONPATH=. headroom proxy --port 8790`, `HEADROOM_OUTPUT_SHAPER=1 HEADROOM_VERBOSITY_LEVEL=2`), real OpenAI upstream (fake API key — shaping happens pre-upstream; upstream 401s prove the request went through the full pipeline). - Exact command / steps: POST three `/v1/responses` bodies — (a) trailing `function_call_output` with `exit_code:0` (mechanical), (b) plain user ask, (c) trailing `function_call_output` with `exit_code:1` (error). - Observed result: mechanical turn got `effort:high->low` + L2 steering; new ask and error turns kept full effort with L2 steering only — proxy request log `transforms_applied` below. ```text (a) ["output_shaper:stratum:gpt|mechanical_continuation|xs|tools", "output_shaper:verbosity:L2", "output_shaper:effort:high->low"] (b) ["output_shaper:stratum:gpt|new_user_ask|xs|notools", "output_shaper:verbosity:L2"] (c) ["output_shaper:stratum:gpt|error_continuation|xs|notools", "output_shaper:verbosity:L2"] ``` Mechanical turn gets `reasoning.effort` high→low; new ask and error continuation keep full effort; all three get the byte-stable L2 steering on the `instructions` tail. - Not tested: a live Codex WebSocket session end-to-end against the ChatGPT backend (the WS paths share the exact executor funnel exercised above); `test_codex_ws_compression_scheduler.py::test_concurrent_compression_has_no_semaphore_tail` fails in my env on a clean tree too (no compiled `headroom._core` in a source checkout) — unrelated. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>	2026-07-13 00:52:36 -04:00