mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
1 commit
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tejas Chopra
|
e0d2cd0c5a
|
fix(cache): preserve cache_control ttl when re-anchoring a breakpoint (#2651)
## Description
`normalize_message_cache_control` deliberately reuses the client's
marker verbatim so an explicit `cache_control.ttl` (e.g. `"1h"`)
survives breakpoint consolidation instead of silently downgrading to the
5-minute default (#2375).
Two other sites also strip a breakpoint and re-place it, and both
hardcoded a bare `{"type": "ephemeral"}` — undoing that guarantee.
A downgrade is invisible: the request still succeeds, and the cost shows
up later as a full prefix re-write on every idle gap past 5 minutes.
Measured over 10,409 local Claude Code API requests, cache writes are
**6.1% of raw input tokens but 44.8% of the price-weighted input bill**
(5m write 1.25x vs read 0.1x), and **89% of those write tokens are
re-writes of content cached one request earlier**. Honoring a 1h TTL
when the client asks for it is the cheapest thing we can do about that.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/transforms/read_maturation.py` — `relocate_cache_breakpoint`
now carries the stripped marker forward when re-anchoring before the
held-Read region. This is the one that mattered most: it runs **after**
`normalize_message_cache_control` in the Anthropic handler
(`anthropic.py:1747` vs `:1642`), so it had the final say — a 1h client
with read maturation enabled was being downgraded to 5m.
- `headroom/proxy/helpers.py` — `inject_tool_search_deferral` keeps the
dropped marker when moving the tools-array breakpoint off a now-deferred
tool onto the last resident real tool.
- Both fall back to a bare ephemeral only when the client sent no ttl,
and neither invents a breakpoint where none existed.
- `headroom/transforms/compression_policy.py` — comment only. Notes that
`CACHE_WRITE_MULTIPLIER` is hardcoded to the 5m tier (1.25x), so a
client already on 1h caching (2.0x) has its mutations gated with a ~40%
under-stated write penalty. Harmless while the net-cost gate stays
default-off (`HEADROOM_NET_COST_POLICY`); names the plumbing needed if
it is ever enabled.
Both changed code paths sit behind off-by-default flags
(`HEADROOM_READ_MATURATION`, `HEADROOM_TOOL_SEARCH`), so this is a
latent-bug fix with **no default behavior change**.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_cache_ttl_preserved.py tests/test_read_maturation.py \
tests/test_read_maturation_handler_nobust.py tests/test_cache_control_move_bust.py -q
tests/test_cache_ttl_preserved.py ..... [ 12%]
tests/test_read_maturation.py ...................... [ 67%]
tests/test_read_maturation_handler_nobust.py ... [ 75%]
tests/test_cache_control_move_bust.py .......... [100%]
============================= 40 passed in 15.52s ==============================
$ ruff check headroom/ tests/ --exclude headroom/dashboard/templates
All checks passed!
$ mypy headroom
Success: no issues found in 509 source files
```
Broader regression sweep over every cache/breakpoint-adjacent suite:
```text
$ python -m pytest tests/ -q -k "read_maturation or tool_search or cache_control or prefix_tracker or ttl_preserved"
204 passed, 10145 deselected in 59.95s
```
## Real Behavior Proof
- **Environment:** macOS 25.4.0 (arm64), Python 3.12.6, pytest 9.0.2,
branched from `main` at
|