## Description
`read_lifecycle` emits STALE/SUPERSEDED Read markers containing
`Retrieve original: hash=...`, but `headroom_retrieve(hash)` 404s on
every such marker — the original content is never actually stored.
Affects the default config (`read_lifecycle=on`, `compress_stale=on`)
and the common Claude Code flow: read a file, edit it, then want the
prior content back.
**Root cause:** `ContentRouter.transform` instantiated
`ReadLifecycleManager` with
`compression_store=kwargs.get("compression_store")`, but no caller ever
sets that kwarg. `self.store` was always `None`, so `read_lifecycle.py`
emitted the marker with a SHA-256 hash but skipped the
`store.store(...)` call. Every other compressor (SmartCrusher, Kompress,
search/log/diff/code) resolves its store directly via
`get_compression_store()`.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/transforms/content_router.py`: inject a CCR store into
`ReadLifecycleManager` via an explicit `is None` check + guarded
`get_compression_store()` import (matches `smart_crusher.py`'s pattern).
Falls back to marker-only when the module is absent in stripped builds.
- `headroom/transforms/read_lifecycle.py`: wrap `store.store(...)` in
`try/except` with a precomputed fallback hash so a transient backend
failure can't break `compress()` (mirrors `read_maturation.py`). Pass
`explicit_hash=ccr_hash` to avoid double SHA-256 and keep marker/store
key in lockstep.
- `tests/test_transforms/test_read_lifecycle.py`: regression test
(`TestContentRouterIntegration`) that drives `headroom.compress()` and
asserts the STALE marker's hash resolves in the global CCR store.
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`) — not run locally
- [ ] Type checking passes (`mypy headroom`) — not run locally
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ HEADROOM_CCR_BACKEND=memory .venv/bin/python -m pytest tests/test_transforms/test_read_lifecycle.py -v
============================== 23 passed in 0.43s ==============================
```
## Real Behavior Proof
- Environment: Python 3.13, headroom-ai dev install (`uv sync --extra
dev`), `HEADROOM_CCR_BACKEND=memory`, Linux x86_64.
- Exact command / steps: Run `headroom.compress()` on a synthetic STALE
conversation (Read then Edit of the same file):
```python
from headroom import compress
result = compress([
{"role": "assistant", "content": [{"type": "tool_use", "id": "t1",
"name": "Read",
"input": {"file_path": "/tmp/foo.txt"}}]},
{"role": "user", "content": [{"type": "tool_result", "tool_use_id":
"t1",
"content": "source line\n" * 500}]},
{"role": "assistant", "content": [{"type": "tool_use", "id": "t2",
"name": "Edit",
"input": {"file_path": "/tmp/foo.txt"}}]},
{"role": "user", "content": [{"type": "tool_result", "tool_use_id":
"t2",
"content": "edited"}]},
], model="claude-sonnet-4-5-20250929")
```
then `get_compression_store().retrieve(<hash-from-marker>)`.
- Observed result: post-fix `retrieve(hash)` returns HIT (`tool=Read`,
`strategy=read_lifecycle:stale`); pre-fix it returned MISS (the bug).
Full log:
```text
transforms_applied: ['read_lifecycle:stale:/tmp/foo.txt',
'router:excluded:tool', 'router:excluded:tool']
hashes from markers: ['3fbd603ecf1bcf50a86650d2']
store backend: InMemoryBackend
retrieve(3fbd603ecf1bcf50a86650d2) -> HIT tool=Read
strategy=read_lifecycle:stale
```
- Not tested: SQLite backend persistence across processes; Rust `_core`
extension code path; OpenAI / Gemini providers; Claude Code live (proxy
+ MCP server end-to-end).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation — N/A
(internal fix, no public API change)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable — leaving to
maintainers' convention
## Additional Notes
- No existing issue. #389 describes the same symptom class with a
different root cause (SmartCrusher row-drop CCR bridge); it explicitly
lists `read_lifecycle.py` as a producer that populates the store — this
PR makes that claim true.
- Commits: `dde42478` (initial fix) → `55a0dfde` (Copilot round 1: `is
None` + import guard + best-effort `store.store()`) → `2e8c41a2`
(Copilot round 2: regression test + `explicit_hash`).
## What
Enrich the `transforms_applied` tags emitted by `ReadLifecycleManager`
and `SmartCrusher` so each tag carries the specific target it acted on,
instead of being an opaque counter:
- `read_lifecycle:<state>` -> `read_lifecycle:<state>:<file_path>`
- `smart_crush:<n>` -> `smart_crush:<n>:<tool1,tool2,...>` (tool names
resolved from the assistant's `tool_calls` / `tool_use` metadata; falls
back to `smart_crush:<n>` when no name resolves)
Downstream UIs can then show *what* a compression acted on (which file
was a stale read, which tools had their output crushed), not just that
it happened.
## Note on the rebase
The original revision targeted `ToolCrusher` / `tool_crush:<n>`. That
transform has since been retired and replaced by the Rust-backed
`SmartCrusher` (which emits `smart_crush:<n>`), so the tool-name
attribution moved to `smart_crusher.py`. The `read_lifecycle` half is
unchanged.
## Response-header compatibility
`x-headroom-transforms` is built as `",".join(transforms_applied)`. A
tag containing a comma (tool-name lists; file paths) would make that
header ambiguous to split back into tags. To keep the header backward
compatible, `header_safe_transforms` (`headroom/proxy/cost.py`)
collapses the enriched tags back to their legacy counter shape **for the
header only** -- the full enriched detail still flows through the
structured `transforms_applied` list (dashboards, request logs, activity
feed). Applied at all three header sites (openai / anthropic / gemini
handlers).
Paths containing `:` survive in `transforms_applied` because consumers
bound their split to 3 parts.
## Tests
- `tests/test_transforms/test_read_lifecycle.py` -- OpenAI + Anthropic
tag shape, colon-in-path preservation
- `tests/test_transforms/test_smart_crusher_attribution.py` -- OpenAI +
Anthropic tool-name resolution, dedup, no-name fallback, id/name-missing
skips
- `tests/test_proxy/test_header_safe_transforms.py` -- header
normalization keeps the joined header unambiguous (incl. comma-in-path)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Dashboard was showing wildly incorrect metrics (99.5% savings, 3ms overhead)
due to using Anthropic API's non-cached input_tokens instead of optimized_tokens,
and dividing overhead by total request count instead of optimized-only count.
Key fixes:
- Use optimized_tokens (what we sent) for dashboard aggregation, not API's
input_tokens which excludes cached portion
- Track overhead_count separately from latency_count for correct averages
- Add TTFB (time to first byte) measurement, replace full stream latency in UI
- Eager-load LLMLingua model at proxy startup (eliminates 5.9s first-request delay)
- Simplify CostTracker to token-based accounting with counterfactual cost display
- Add two-tier compression cache to ContentRouter (skip set + result cache)
- Fix compression pinning to detect both CCR and ReadLifecycle markers
- Clamp tokens_saved to max(0, ...) across all provider paths
- Add per-transform timing instrumentation to pipeline
- Guard against over-aggressive code compression (<5% ratio)
- Fix ReadLifecycle partial read supersede logic (_read_covers range check)
- Disable CacheAligner and compress_superseded by default
- Fix all pre-existing mypy errors (CompressionCache return types)
- Fix test mocks to accept **kwargs for cache token parameters
Enable ReadLifecycle by default so stale/superseded Read outputs are
automatically replaced with compact CCR markers — these are provably safe
to compress (file was edited or re-read).
Replace static compression thresholds with adaptive parameters that scale
with conversation length and context pressure:
- protect_recent_reads_fraction: protects the most-recent 50% of messages
from Read exclusion. Old Reads beyond this window become compressible,
preventing the "28 excluded Read/Glob, 0 tokens saved" problem.
- min_ratio_relaxed / min_ratio_aggressive: compression acceptance
threshold interpolates linearly with context pressure (tokens / model
limit). Low pressure → 0.85 (picky), high pressure → 0.65 (accept
anything helpful). Eliminates the fixed 0.9 gate that was rejecting
20+ messages per request.
Also adds --no-read-lifecycle CLI flag, and fixes a missing
pytest.importorskip guard for sentence-transformers in memory tests.
Detects Read tool outputs that became stale (file was later edited) or
superseded (file was later re-Read) and replaces them with compact markers
+ CCR hashes. Fresh Reads are never touched.
Adds ReadLifecycleConfig to config.py and integrates ReadLifecycleManager
as a pre-processing pass in ContentRouter. Opt-in via config flag to
preserve backward-compatible behavior.
