fix(proxy/openai): thread savings-profile kwargs into chat completions (#1606)

## Description

OpenAI-compatible `/v1/chat/completions` requests didn't receive the
same proxy
savings/profile kwargs as the other compression paths. The live chat
handler
(`handle_openai_chat` in `headroom/proxy/handlers/openai.py`) called
`openai_pipeline.apply()` with only `model_limit` / `context` /
`frozen_message_count` / `biases` / `compression_policy` — it never
passed
`proxy_pipeline_kwargs(self.config)`.

So when the proxy runs with `HEADROOM_SAVINGS_PROFILE=agent-90`, the
effective
config reports user/system-message compression and `target_ratio=0.10`,
but the
real chat path silently dropped all of it. OpenAI-compatible clients
such as
OpenCode kept protecting user messages and missed the configured
profile.

For contrast, `handlers/anthropic.py` passes
`**proxy_pipeline_kwargs(self.config)`
to every `apply()` call, and so does the dedicated OpenAI compress
endpoint in
this same module — only the two chat-completions `apply()` sites were
missing it.

Closes #1534

## Fix

Add `**proxy_pipeline_kwargs(self.config)` to both chat-path `apply()`
calls (the
token-mode branch and the non-token branch):

```python
lambda: self.openai_pipeline.apply(
    messages=messages,
    model=model,
    model_limit=context_limit,
    context=extract_user_query(messages),
    frozen_message_count=openai_frozen_count,
    biases=_hook_biases,
    compression_policy=compression_policy,
    **proxy_pipeline_kwargs(self.config),   # ← added
)
```

`proxy_pipeline_kwargs` is already imported in the module and is the
exact
helper the Anthropic handler and the OpenAI compress endpoint use, so
the chat
path now matches them.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/proxy/handlers/openai.py`: pass
`**proxy_pipeline_kwargs(self.config)` on both `apply()` call sites in
`handle_openai_chat` (token-mode and non-token branches).
- `tests/test_proxy/test_openai_chat_savings_profile.py`: new regression
test driving the chat handler with `savings_profile="agent-90"` and
asserting the profile knobs reach `apply()`.
- `CHANGELOG.md`: Bug Fixes entry under Unreleased.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

The new test drives the real chat handler through the `create_app` +
`TestClient`
harness with a recording `apply()` stub. Before the fix it captures
exactly the
five kwargs the issue describes (no profile knobs); after the fix the
profile
knobs are present:

```text
# before the fix (openai.py reverted, test kept)
E   AssertionError: assert None is True
E    +  where None = {...}.get('compress_user_messages')
# captured kwargs were: biases, compression_policy, messages, model,
# model_limit, context, frozen_message_count  — no profile knobs
FAILED tests/test_proxy/test_openai_chat_savings_profile.py::test_chat_completions_threads_savings_profile_kwargs_into_apply

# after the fix
tests\test_proxy\test_openai_chat_savings_profile.py .
======================== 1 passed, 1 warning in 39.44s ========================
```

No regression in the existing chat backend-path suite:

```text
$ uv run pytest tests/test_proxy/test_openai_backend_path.py
======================== 5 passed, 1 warning in 15.78s ========================
$ uv run ruff check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_chat_savings_profile.py
All checks passed!
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12.11, headroom built from this
branch (`uv sync --extra dev`), proxy config
`savings_profile="agent-90"`, `optimize=True`, `backend="anyllm"` with a
mocked OpenAI upstream.
- Exact command / steps: started the app with `create_app(config)`,
replaced `proxy.openai_pipeline.apply` with a recording stub, and POSTed
a real `/v1/chat/completions` request with a large user message so the
compression decision fires. Inspected the kwargs the handler actually
passed to `apply()`.
- Observed result: before the fix the recorded `apply()` kwargs were
`{biases, compression_policy, messages, model, model_limit, context,
frozen_message_count}` — no profile knobs. After the fix the same call
also carries `compress_user_messages=True`,
`compress_system_messages=True`, `target_ratio=0.10`,
`min_tokens_to_compress=120` (the agent-90 profile), matching the
issue's "Expected".
- Not tested: did not stand up a real OpenAI/OpenCode upstream
end-to-end (no live key in this environment); the upstream is mocked and
the assertion is on the kwargs the proxy threads into the compression
pipeline, which is exactly what the bug was about. Did not run the full
`mypy headroom` pass (two-line kwarg addition, no new types).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

- Two-line change plus comments; no new dependencies. Reuses the
existing `proxy_pipeline_kwargs` helper, so behavior is consistent
across Anthropic, the OpenAI compress endpoint, and now the OpenAI chat
path.
- @chopratejas flagging you for review — this aligns the OpenAI chat
path with the savings-profile handling the other providers already had.

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Abhay Singh 2026-07-07 21:57:54 +05:30 committed by GitHub
parent 48201345be
commit 7ff842da17
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 119 additions and 0 deletions

View file

@ -0,0 +1,106 @@
"""Regression test for #1534.
The live OpenAI `/v1/chat/completions` compression path must thread the proxy
savings-profile kwargs (``proxy_pipeline_kwargs(config)``) into
``openai_pipeline.apply`` the same way ``handlers/anthropic.py`` and the
dedicated OpenAI compress endpoint do. Before the fix the chat path only passed
``model_limit``/``context``/``frozen_message_count``/``biases``/
``compression_policy``, so ``HEADROOM_SAVINGS_PROFILE=agent-90`` (and other
profile knobs) were silently dropped on the real chat path.
"""
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
fastapi = pytest.importorskip("fastapi")
pytest.importorskip("httpx")
from fastapi.testclient import TestClient # noqa: E402
from headroom.backends.base import BackendResponse # noqa: E402
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
def _make_mock_backend() -> MagicMock:
backend = MagicMock()
backend.name = "anyllm-openai"
backend.send_openai_message = AsyncMock(
return_value=BackendResponse(
body={
"id": "chatcmpl-1",
"object": "chat.completion",
"model": "gpt-4o",
"choices": [
{
"index": 0,
"message": {"role": "assistant", "content": "ok"},
"finish_reason": "stop",
}
],
"usage": {"prompt_tokens": 100, "completion_tokens": 2, "total_tokens": 102},
},
status_code=200,
headers={"content-type": "application/json"},
)
)
return backend
def test_chat_completions_threads_savings_profile_kwargs_into_apply():
"""With HEADROOM_SAVINGS_PROFILE=agent-90, the chat path must pass the
profile knobs (compress_user_messages, target_ratio, ...) to apply()."""
config = ProxyConfig(
optimize=True,
cache_enabled=False,
rate_limit_enabled=False,
backend="anyllm",
anyllm_provider="openai",
savings_profile="agent-90",
)
captured: dict[str, object] = {}
def recording_apply(**kwargs):
captured.update(kwargs)
sent = kwargs["messages"]
return SimpleNamespace(
messages=sent,
transforms_applied=[],
timing={},
tokens_before=4000,
tokens_after=400,
waste_signals=None,
)
# A large user message so the compression decision actually fires.
big = "word " * 4000
mock_backend = _make_mock_backend()
with patch("headroom.proxy.server.AnyLLMBackend", return_value=mock_backend):
app = create_app(config)
with TestClient(app) as client:
proxy = client.app.state.proxy
proxy.openai_pipeline.apply = MagicMock(side_effect=recording_apply)
resp = client.post(
"/v1/chat/completions",
json={
"model": "gpt-4o",
"messages": [{"role": "user", "content": big}],
"stream": False,
},
headers={"Authorization": "Bearer test-key"},
)
assert resp.status_code == 200, resp.text
assert proxy.openai_pipeline.apply.call_count >= 1, "compression apply() never ran"
# The agent-90 profile knobs must be present on the apply() call.
assert captured.get("compress_user_messages") is True
assert captured.get("target_ratio") == 0.10
assert captured.get("min_tokens_to_compress") == 120
assert captured.get("compress_system_messages") is True