mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
7ff842da17
## Description OpenAI-compatible `/v1/chat/completions` requests didn't receive the same proxy savings/profile kwargs as the other compression paths. The live chat handler (`handle_openai_chat` in `headroom/proxy/handlers/openai.py`) called `openai_pipeline.apply()` with only `model_limit` / `context` / `frozen_message_count` / `biases` / `compression_policy` — it never passed `proxy_pipeline_kwargs(self.config)`. So when the proxy runs with `HEADROOM_SAVINGS_PROFILE=agent-90`, the effective config reports user/system-message compression and `target_ratio=0.10`, but the real chat path silently dropped all of it. OpenAI-compatible clients such as OpenCode kept protecting user messages and missed the configured profile. For contrast, `handlers/anthropic.py` passes `**proxy_pipeline_kwargs(self.config)` to every `apply()` call, and so does the dedicated OpenAI compress endpoint in this same module — only the two chat-completions `apply()` sites were missing it. Closes #1534 ## Fix Add `**proxy_pipeline_kwargs(self.config)` to both chat-path `apply()` calls (the token-mode branch and the non-token branch): ```python lambda: self.openai_pipeline.apply( messages=messages, model=model, model_limit=context_limit, context=extract_user_query(messages), frozen_message_count=openai_frozen_count, biases=_hook_biases, compression_policy=compression_policy, **proxy_pipeline_kwargs(self.config), # ← added ) ``` `proxy_pipeline_kwargs` is already imported in the module and is the exact helper the Anthropic handler and the OpenAI compress endpoint use, so the chat path now matches them. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/proxy/handlers/openai.py`: pass `**proxy_pipeline_kwargs(self.config)` on both `apply()` call sites in `handle_openai_chat` (token-mode and non-token branches). - `tests/test_proxy/test_openai_chat_savings_profile.py`: new regression test driving the chat handler with `savings_profile="agent-90"` and asserting the profile knobs reach `apply()`. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output The new test drives the real chat handler through the `create_app` + `TestClient` harness with a recording `apply()` stub. Before the fix it captures exactly the five kwargs the issue describes (no profile knobs); after the fix the profile knobs are present: ```text # before the fix (openai.py reverted, test kept) E AssertionError: assert None is True E + where None = {...}.get('compress_user_messages') # captured kwargs were: biases, compression_policy, messages, model, # model_limit, context, frozen_message_count — no profile knobs FAILED tests/test_proxy/test_openai_chat_savings_profile.py::test_chat_completions_threads_savings_profile_kwargs_into_apply # after the fix tests\test_proxy\test_openai_chat_savings_profile.py . ======================== 1 passed, 1 warning in 39.44s ======================== ``` No regression in the existing chat backend-path suite: ```text $ uv run pytest tests/test_proxy/test_openai_backend_path.py ======================== 5 passed, 1 warning in 15.78s ======================== $ uv run ruff check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_chat_savings_profile.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom built from this branch (`uv sync --extra dev`), proxy config `savings_profile="agent-90"`, `optimize=True`, `backend="anyllm"` with a mocked OpenAI upstream. - Exact command / steps: started the app with `create_app(config)`, replaced `proxy.openai_pipeline.apply` with a recording stub, and POSTed a real `/v1/chat/completions` request with a large user message so the compression decision fires. Inspected the kwargs the handler actually passed to `apply()`. - Observed result: before the fix the recorded `apply()` kwargs were `{biases, compression_policy, messages, model, model_limit, context, frozen_message_count}` — no profile knobs. After the fix the same call also carries `compress_user_messages=True`, `compress_system_messages=True`, `target_ratio=0.10`, `min_tokens_to_compress=120` (the agent-90 profile), matching the issue's "Expected". - Not tested: did not stand up a real OpenAI/OpenCode upstream end-to-end (no live key in this environment); the upstream is mocked and the assertion is on the kwargs the proxy threads into the compression pipeline, which is exactly what the bug was about. Did not run the full `mypy headroom` pass (two-line kwarg addition, no new types). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Two-line change plus comments; no new dependencies. Reuses the existing `proxy_pipeline_kwargs` helper, so behavior is consistent across Anthropic, the OpenAI compress endpoint, and now the OpenAI chat path. - @chopratejas flagging you for review — this aligns the OpenAI chat path with the savings-profile handling the other providers already had. Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|---|---|---|
| .. | ||
| test_anthropic_buffered_timeout.py | ||
| test_anthropic_ccr_deferred_injection.py | ||
| test_anthropic_ccr_raise.py | ||
| test_anthropic_streaming_ccr_retrieve.py | ||
| test_background_compression.py | ||
| test_bedrock_passthrough.py | ||
| test_cc_switch_reconciler.py | ||
| test_ccr_frozen_prefix_coupling.py | ||
| test_compression_failure_action.py | ||
| test_compression_timeout_config.py | ||
| test_compute_turn_id.py | ||
| test_header_safe_transforms.py | ||
| test_mcp_stats_aggregation.py | ||
| test_openai_backend_path.py | ||
| test_openai_chat_savings_profile.py | ||
| test_openai_transport_path_prefix.py | ||
| test_openai_upstream_header.py | ||
| test_phase3_byte_identity.py | ||
| test_request_logger.py | ||
| test_transformations_feed.py | ||