mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
1 commit
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Jervis
|
09c66ac212
|
fix(proxy): batch small Codex Responses tool outputs (#2239)
## Description Batches small Codex/OpenAI Responses tool-output units through the existing ContentRouter instead of skipping each unit individually below the 512-byte floor. This fixes sessions where many small tool outputs are collectively worth compressing, but no single output clears the per-unit threshold. The change keeps larger units on the existing independent compression path, preserves CCR retrieval markers and protected tags across the batch envelope, rejects structurally invalid batch output, and leaves under-floor tails as size-floor passthroughs. Fixes #2234 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `headroom/transforms/compression_batches.py` for bounded compatible-unit batching, batch envelope parsing, tag/CCR marker preservation, and per-entry result splitting. - Updated the OpenAI Responses compression adapter to batch small tool-output text slots while keeping larger units on the existing cached per-unit path. - Switched the unit size floor to UTF-8 bytes so CJK and other multibyte text are measured consistently with the byte threshold. - Added regression coverage for batching, CJK byte floors, CCR marker preservation, malformed batch rejection, array output parts, and under-floor tails. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --with pytest --with fastapi --with httpx --with anyio --with uvicorn --with h2 pytest tests/test_compression_batches.py tests/test_compression_units.py tests/test_openai_responses_compression_units.py -q 47 passed, 1 warning $ uvx ruff==0.15.17 check headroom/proxy/handlers/openai.py headroom/transforms/compression_batches.py headroom/transforms/compression_units.py tests/test_compression_batches.py tests/test_compression_units.py tests/test_openai_responses_compression_units.py --output-format concise All checks passed! $ uvx ruff==0.15.17 format --check headroom/proxy/handlers/openai.py headroom/transforms/compression_batches.py headroom/transforms/compression_units.py tests/test_compression_batches.py tests/test_compression_units.py tests/test_openai_responses_compression_units.py 6 files already formatted $ uv run --with mypy mypy headroom/transforms/compression_batches.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13.3, local checkout of this PR branch. - Exact command / steps: ran the focused batching/unit/OpenAI Responses test suites above, including cases where four individually-small tool outputs collectively exceed the shared floor and where output arrays contain multiple text parts plus non-text parts. - Observed result: small outputs are sent through one router call and applied back to their original slots; under-floor tails remain unmodified; non-text parts are preserved; CCR markers are retained or the entire batch is rejected if moved/corrupted. - Not tested: a live Codex Responses proxy session against an upstream model; full-suite collection was not run locally. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |