## Description
Preserves Anthropic `tools` order when Headroom is forwarding a
passthrough/no-optimize request. This fixes a Claude Code style
`tool_result` continuation failure against stricter Anthropic-compatible
upstreams that treat the client's original tool ordering as part of the
conversation state.
Closes#1417
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Preserve client-provided Anthropic `tools` order when `optimize=False`
or the request is explicitly in Headroom passthrough/bypass mode.
- Keep deterministic tool sorting for optimized requests where Headroom
may rewrite the body for cache stability.
- Avoid sorting batch-request tools before the no-optimize passthrough
branch.
- Add regression coverage for the Anthropic HTTP path to prove
no-optimize forwarding keeps `Read`, then `Bash` tool order.
- Update existing cache-stability and byte-faithful forwarding tests so
no-optimize/passthrough expects preserved client order while optimized
mode still proves deterministic sorting.
## Testing
- [x] Focused unit tests pass (`pytest` on touched proxy test files)
- [x] Linting passes (`ruff check` and `ruff format --check` on touched
files)
- [x] Type checking passes (`mypy headroom`)
- [x] New regression tests added
- [x] Manual testing performed
### Test Output
```text
$ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with pytest --with pytest-asyncio --with anyio --with 'httpx[http2]' --with fastapi --with pydantic --with tiktoken --with click --with rich --with opentelemetry-api --with opentelemetry-sdk --with zstandard --with openai --with mcp --with uvicorn pytest tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py -q
============================= test session starts ==============================
platform darwin -- Python 3.13.11, pytest-9.1.1, pluggy-1.6.0
rootdir: /Users/vinaygupta/Desktop/git/headroom-fix-anthropic-tool-order
configfile: pyproject.toml
plugins: anyio-4.14.1, asyncio-1.4.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 87 items
tests/test_proxy_handler_helpers.py .......................... [ 29%]
tests/test_anthropic_stage_timings.py .... [ 34%]
tests/test_proxy_anthropic_cache_stability.py ......................... [ 63%]
tests/test_proxy_byte_faithful_forwarding.py ........................... [ 94%]
..... [100%]
=============================== warnings summary ===============================
.../fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
======================== 87 passed, 1 warning in 5.13s =========================
$ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff format --check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py
5 files already formatted
$ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py
All checks passed!
```
## Real Behavior Proof
- Environment: macOS, Python 3.13.11, local fake Anthropic-compatible
upstream, local Headroom proxy launched with `--no-optimize --no-cache
--no-rate-limit --stateless`.
- Exact command / steps: ran a local reproduction harness that starts a
fake `/v1/messages` upstream and Headroom proxy, then sends a Claude
Code style two-turn flow: first assistant `Bash` `tool_use`, then user
`tool_result`.
- Observed result: after this patch, both direct and proxied flows
returned `200` for `first_tool_use` and `second_tool_result`. The fake
upstream log showed the proxied `tools` array remained `["Read",
"Bash"]` on both turns.
```text
DIRECT
first_tool_use: 200
second_tool_result: 200
PROXIED
first_tool_use: 200
second_tool_result: 200
UPSTREAM REQUEST LOG
proxied first turn tools: ["Read", "Bash"]
proxied tool_result turn tools: ["Read", "Bash"]
```
- Not tested: full `pytest`, full-repo `ruff check .`, `mypy headroom`,
or a live third-party Anthropic-compatible provider.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A
## Additional Notes
- This PR intentionally does not add documentation because it fixes
passthrough behavior rather than introducing a new user-facing option.
- The code-comment checklist item is left unchecked because the change
is covered by a small helper docstring and regression tests; no extra
inline comments seemed necessary.
- `CHANGELOG.md` is left unchanged because this is a narrowly scoped bug
fix.
- Local pytest collection for these proxy tests required a local
`headroom._core` extension symlink, which was removed before committing.
## Description
Honor the proxy savings profile's `force_kompress` setting all the way
through the Anthropic proxy path.
`HEADROOM_SAVINGS_PROFILE=agent-90` already resolves to
`force_kompress=True`, but `ContentRouter` still paid for the full
auto-detection path before selecting Kompress. On long Claude Code /
tool-output conversations this can hang inside the detection/router path
before any `Transform content_router` line is emitted. This change makes
the forced-Kompress path skip unused strategy detection during
compression, while still preserving recent-code protection via the
lightweight regex detector.
This also passes `proxy_pipeline_kwargs(self.config)` through Anthropic
batch requests so batch traffic receives the same savings-profile knobs
as normal Anthropic messages.
Refs #946
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Skip `is_mixed_content()` / `_detect_content()` when runtime
`force_kompress` is set and route directly to
`CompressionStrategy.KOMPRESS`.
- Keep forced-Kompress recent-code protection, but use
`_regex_detect_content_type()` instead of the full router detection
chain.
- Read `_runtime_force_kompress` defensively in `ContentRouter.apply()`
so regular `ContentRouter()` instances keep the normal content-detection
path.
- Pass proxy savings-profile kwargs into Anthropic batch compression.
- Add regression tests for forced-Kompress routing, normal routing,
recent-code protection, and Anthropic batch profile propagation.
- Update `CHANGELOG.md`.
## Testing
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ ruff check headroom/transforms/content_router.py tests/test_transforms_content_router.py
All checks passed!
$ ruff format --check headroom/transforms/content_router.py tests/test_transforms_content_router.py
2 files already formatted
$ pytest tests/test_transforms_content_router.py::test_force_kompress_bypasses_content_detection \
tests/test_transforms_content_router.py::test_normal_compress_path_still_uses_content_detection \
tests/test_transforms_content_router.py::test_force_kompress_apply_uses_lightweight_detection \
tests/test_transforms_content_router.py::test_force_kompress_apply_lightweight_detection_protects_recent_code \
tests/test_proxy_anthropic_cache_stability.py::test_batch_optimization_passes_savings_profile_kwargs \
tests/test_bundled_tools_savings.py -q
============================= test session starts =============================
platform win32 -- Python 3.13.1, pytest-9.0.3, pluggy-1.6.0
rootdir: E:\work\code\third-party\headroom
configfile: pyproject.toml
plugins: anyio-4.12.1, langsmith-0.8.0, asyncio-1.3.0, cov-7.0.0, timeout-2.4.0
collected 11 items
tests\test_transforms_content_router.py .... [ 36%]
tests\test_proxy_anthropic_cache_stability.py . [ 45%]
tests\test_bundled_tools_savings.py ....ss [100%]
======================== 9 passed, 2 skipped in 9.77s =========================
```
Full-suite attempt status on Windows / Python 3.13 after installing
missing local test dependencies and bundled tools (`fastembed`,
`socksio`, `pytest-timeout`, `difft`, `scc`, cached HF models with
offline env vars):
```text
tests/test_adapter_hooks.py: 29 passed, 2 failed
- sqlite:///C:\... and jsonl:///C:\... URLs are parsed into invalid \C:\... paths on Windows.
tests/test_cache/test_client_integration.py: 16 failed
- Same Windows URL path parsing issue.
tests/test_cli/test_wrap_helpers.py: 29 passed, then KeyboardInterrupt during read/cleanup.
tests/test_memory tests/test_storage:
- Collection/run receives KeyboardInterrupt in this Windows environment.
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.13.1, `headroom-ai` v0.25.0,
Anthropic proxy on `127.0.0.1:8787`, Kompress ONNX backend,
`HEADROOM_SAVINGS_PROFILE=agent-90`,
`HEADROOM_COMPRESS_USER_MESSAGES=1`, `HEADROOM_MIN_TOKENS=120`.
- Exact command / steps: started the proxy with the local launcher, sent
a long `/v1/messages` request with a fake upstream token, and inspected
`/livez`, `/stats?include_config=true`, and
`~/.headroom/logs/proxy.log`.
- Observed result: request returned promptly with the expected upstream
auth failure after local compression, and logs showed the compression
ran before forwarding:
```text
/livez healthy
/v1/messages completed in ~3005ms with expected upstream 401
Transform content_router: 1900 -> 193 tokens (saved 1707) [1328.4ms]
Pipeline complete: 1907 -> 200 tokens (saved 1707, 89.5% reduction)
UPSTREAM_ERROR ... compressed=yes transforms=['router:kompress:0.06'] original_tokens=1886 optimized_tokens=119
PERF ... tok_before=1886 tok_after=119 tok_saved=1767 transforms=router:kompress:0.06
/stats tokens.saved = 1767
/stats compressions_by_strategy = {"kompress": 1}
```
- Not tested: full upstream CI matrix, full `uv run pytest`, full `ruff
check .`, `mypy headroom`, real Anthropic success response with a valid
upstream token, and Anthropic batch against the live upstream. The
Anthropic batch change is covered by a local handler regression test.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A
## Additional Notes
This PR is ready for human review. The patch is scoped to the
forced-Kompress profile path and does not change the default
auto-routing behavior when `force_kompress` is false.
The latest `PR Governance / template` check passes after the readiness
checkbox update. A later `PR Governance / label` run currently fails
while trying to execute `.github/scripts/pr-health-labels.py` from the
base checkout; that file is missing on the checked-out base ref, so this
appears to be a governance workflow issue rather than a
PR-template/content failure in this branch.
Two follow-ups surfaced when B6 and B7 were merged onto the megamerge
branch and the full suite ran:
1. tests/test_proxy_anthropic_cache_stability.py
PR-B7 added `injector.scan_for_markers(optimized_messages)` to the
Anthropic handler so the always-on tool-registration logic can see
detected hashes for the current request. The two pre-existing
`_FakeInjector` mocks (`test_ccr_system_instruction_injection_disabled_*`
and `test_ccr_tool_injection_disabled_*`) didn't implement that method.
Added a no-op `scan_for_markers` returning [] to both mocks — matches
the real injector's contract for the not-yet-compressed request shape
these tests exercise.
2. tests/test_memory_tool_mode.py::test_tool_mode_skip_emits_structured_log
The B6 caplog assertion passed in isolation but failed in the full
suite. Root cause: when an earlier test triggers proxy startup,
`_setup_file_logging` flips `headroom.propagate=False` and attaches a
RotatingFileHandler to the headroom logger. caplog captures via
propagation to root, so log records stop reaching it. The conftest
autouse fixture that resets `propagate=True` before every test gets
shadowed by fixture-ordering edge cases.
Principled fix: attach `caplog.handler` directly to
`headroom.proxy.memory_handler` for the duration of the test so the
capture is independent of propagation state. Restore the original
level + remove the handler in `finally` to keep the test hermetic.
Both B6 and B7 cherry-picks themselves are unmodified. This commit only
adjusts test harness code so the pre-existing mocks/capture stay
consistent with the new code paths.
Eliminates P0-2 universally. Every Python forwarder (server.py
`_retry_request`, handlers/streaming.py `_stream_response`,
handlers/openai.py `_ws_http_fallback`, handlers/batch.py `_batch_passthrough`
+ batch-create + Google batch passthrough, handlers/anthropic.py CCR
continuation + batch endpoint) now switches from `httpx ... json=body` to
`httpx ... content=raw_bytes`. The default httpx JSON encoder was
re-serializing every request with `, `/`: ` separators and `\\uXXXX` ASCII
escapes — collapsing Anthropic prompt-cache hit-rate.
Forwarder strategy:
- unmutated body → forward `await request.body()` verbatim;
- mutated body → re-serialize once via the new
`serialize_body_canonical(body) -> bytes` helper (compact separators,
`ensure_ascii=False`, dict insertion order preserved).
`HEADROOM_PROXY_PYTHON_FORWARDER_MODE` env var configures the mode:
- `byte_faithful` (default) — the new behavior;
- `legacy_json_kwarg` — explicit operator opt-in for emergency rollback.
Documented in `docs/content/docs/configuration.mdx`. NOT a fallback —
unknown values raise loudly per build constraint #4.
`BodyMutationTracker` accompanies each request through the handler so
transform sites mark the tracker (`memory_injection`,
`image_compression`, `compression_*`, `batch_compression`,
`ccr_continuation`, etc.). At forwarder dispatch we additionally compare
the final body dict against the parsed original bytes as a structural
safety net — any silent mutation we missed still triggers canonical
re-serialization.
A2 follow-up: `handlers/openai.py:534-540` (Chat Completions memory
injection) was prepending a system message; replaced with
`append_text_to_latest_user_chat_message`, the OpenAI Chat Completions
analog of `_append_context_to_latest_non_frozen_user_turn`. The cache
hot zone (system messages) is now sacrosanct on /v1/chat/completions
too. Honors `HEADROOM_MEMORY_INJECTION_MODE=disabled`.
Structured logging: every forwarder emits an `event=outbound_request`
log line with `forwarder`, `path`, `body_bytes`, `body_mutated`,
`mutation_reasons`, `source` (passthrough|canonical|legacy),
`request_id`. Never logs Authorization or full body.
`_read_request_json` factored to share `_read_request_body_bytes` with
new `read_request_json_with_bytes` so the anthropic handler can capture
both the parsed dict and the original (decompressed) bytes.
Tests:
- `tests/test_proxy_byte_faithful_forwarding.py` (28 tests):
SHA-256 byte-equality on /v1/messages and streaming, unicode
preservation, numeric precision, mutation-tracker invariants,
canonical-serializer properties, legacy-mode rollback, OpenAI
Chat memory routing.
- Existing test mocks updated to accept the new `**kwargs` on
`_retry_request` (no behavior change).
- `tests/test_proxy_handlers_batch.py` updated to read the captured
`content=` bytes (formerly `json=`).
- One A2 test corrected (`test_anthropic_tool_sort_and_context_append_helpers`)
to match the live-zone-tail semantics introduced by A2.
Constraints satisfied: configurable env var; no new regex / hardcodes;
no silent fallback (`legacy_json_kwarg` is operator opt-in);
performant (`prepare_outbound_body_bytes` is O(1) for passthrough);
elegant single-responsibility helpers; structured tracing logs.
The Anthropic token-mode handler walked past prefix_tracker.frozen_message_count
whenever an upcoming tool_result's content-hash matched comp_cache._stable_hashes
or should_defer_compression returned True. That conflated content equality with
positional cache membership.
Anthropic's prefix cache is POSITIONAL: bytes 0..K cached, anything past K is
fresh. _stable_hashes is content-keyed and grows unbounded. In long Claude Code
sessions where tool_result content rhymes across turns (repeated system prompts,
repeated file reads, repeated tool descriptions), the walker advanced
frozen_message_count to len(messages) on every turn and the pipeline produced
transforms_applied=[] on 73% of requests in user SvenMeyer's reported session
(headroom-stats-2026-05-01.json: 74 of 101 eligible requests "prefix_frozen") —
even after the prior fix in 44944fb. The 15 requests that did compress averaged
21%, proving compression itself works when reached.
Fix: delete the walker. The freeze boundary is now
frozen_message_count = min(
prefix_tracker.frozen_message_count, # positional ground truth
comp_cache.compute_frozen_count(messages), # local cache lower bound
)
compute_frozen_count's use of _stable_hashes can only LOWER the freeze via the
min clamp, never raise it past prefix_tracker's value. For any position in the
gap [compute_frozen_count, prefix_tracker.frozen_count], recompressing produces
byte-stable output (compression is deterministic on input content), so
Anthropic's prefix cache stays valid.
Cross-handler verification:
* OpenAI handler (proxy/handlers/openai.py:358-382) does not have this walker
— uses only compute_frozen_count. Codex routes through OpenAI handler. Both
unaffected.
* Streaming and non-streaming both invoke anthropic_pipeline.apply() before the
upstream call. One fix covers both paths.
* Cache mode (is_cache_mode) takes the _extract_cache_stable_delta path and is
independent of the walker. Unaffected.
Tests: six new regression tests lock down the post-fix invariants — clamp to
min(prefix_tracker, compute_frozen_count); fresh tool_result whose hash matches
old _stable_hashes entry is not frozen; frozen prefix byte-stable across the
pipeline; 10-turn session produces non-empty compression suffix every turn;
streaming and non-streaming compute identical frozen_message_count; OpenAI
handler never calls the walker functions. Plus scripts/smoke_issue_327.py
(gated by RUN_LIVE_API=1) drives a 10-turn conversation against
api.anthropic.com in both shapes (string + list-of-blocks) and both modes
(streaming + non-streaming).
ci-precheck clean. 191 tests pass.
Follow-ups (separate PRs):
* Fix _cache perpetually empty (anthropic.py result.messages != working_messages
comparison rarely fires in token mode).
* Cap _stable_hashes with bounded LRU + 1h TTL — hygiene only after the freeze
gate is removed.
* List-shape tool_result content gates at content_router.py:1975 and
intelligent_context.py:657 (cluster A from the audit).
The Anthropic handler's CCR injector path applied a frozen_message_count
guard to system instruction injection but not to tool injection. When
Kompress fired for the first time in a session, the tools array was
mutated unconditionally, invalidating Anthropic's prefix cache and
dropping cache_read_input_tokens to zero on calls where ~48K tokens
were previously being cached.
Mirror the existing inject_system_instructions guard for inject_tool:
when frozen_message_count > 0, defer tool injection so the warm prefix
stays intact.
Adds test_ccr_tool_injection_disabled_when_prefix_frozen as a direct
companion to the existing test_ccr_system_instruction_injection_
disabled_when_prefix_frozen.
Fixes#294
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sync the Anthropic cache stability test double with the prefix tracker contract used by the handler.
Format the benchmark scripts that were failing ruff format --check in CI.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
