fix(content-router): protect_tool_results must not be weakened by profile-derived read_protection_window (#2105)

## Description

`ContentRouter.apply()` computes `read_protection_window` from
`protect_recent_reads_fraction`, where `0.0` (the sentinel
`--protect-tool-results` sets, per #1374's documented contract) means
"protect all excluded-tool output regardless of conversation depth." The
method then unconditionally overwrote that window with a per-request
`read_protection_window` kwarg whenever one was present.
`proxy_pipeline_kwargs()` supplies that kwarg on every request from the
active `AgentSavingsProfile.protect_recent` (the default `coding`
profile sets `protect_recent=2`), so in practice only the last 2
messages ever kept read-protection regardless of
`--protect-tool-results` — older excluded-tool output (`Read`, `Glob`,
`Grep`, `Write`, `Edit` results) silently fell through to lossy Kompress
compression.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/transforms/content_router.py`: the runtime
`read_protection_window` kwarg may now only *narrow* the window when
`self.config.protect_recent_reads_fraction > 0`. It can no longer
override the `0.0` ("protect everything") sentinel that
`--protect-tool-results` sets.
- `tests/test_content_router_exclude_tools.py`: regression coverage that
`--protect-tool-results`-equivalent config
(`protect_recent_reads_fraction=0.0`) stays fully protected even when a
savings-profile kwarg would otherwise shrink the window.
- `tests/test_transforms/test_content_router.py`: unit coverage of the
precedence logic itself (kwarg narrows when fraction > 0, kwarg is
ignored when fraction == 0.0).
- `CHANGELOG.md`: added an `### Bug Fixes` entry under `Unreleased`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_content_router_exclude_tools.py tests/test_transforms/test_content_router.py -q
============================= test session starts ==============================
platform darwin -- Python 3.13.13, pytest-9.0.3, pluggy-1.6.0
collected 64 items

tests/test_content_router_exclude_tools.py ......                        [  9%]
tests/test_transforms/test_content_router.py ........................... [ 51%]
...............................                                          [100%]

============================== 64 passed in 2.77s ==============================

$ uv run ruff check headroom/transforms/content_router.py tests/test_content_router_exclude_tools.py tests/test_transforms/test_content_router.py
All checks passed!

$ uv run mypy headroom/transforms/content_router.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- **Environment:** personal fork deployed as a real proxy (macOS launchd
service, `headroom install apply`) with `--backend bedrock --mode token
--code-aware --protect-tool-results Bash`,
`HEADROOM_SAVINGS_PROFILE=coding` (library default `protect_recent=2`),
fronting a live Claude Code session.
- **Exact command / steps:** in a long-running Claude Code session
against this deployment, `Read` a source file, continue the conversation
past 2 more assistant turns (so the file's `Read` result ages past the
profile's `protect_recent=2` window), then have the agent re-read or
reference the same file.
- **Observed result:** before the fix, the aged `Read` output for a
plain (non-code) file came back as `[N items compressed to M. Retrieve
more: hash=...]` despite `--protect-tool-results` being set and `Read`
sitting in `DEFAULT_EXCLUDE_TOOLS` — confirmed by direct proxy log
inspection (`content_router.py`'s override silently winning over the
`0.0` sentinel) and by byte-diffing the installed pipx package against
this same fork's git source to rule out a stale build. After applying
the fix, the same sequence leaves the aged `Read` output intact (no
compression marker) — verified via `pytest` regression tests plus a
fresh live-session check post-deploy.
- **Not tested:** this deployment has since switched to `--mode cache`
(upstream's tested/benchmarked default for the `coding` profile as of
`68676daa`), where the whole `read_protection_window` mechanism this bug
lives in is structurally unreachable for anything inside the frozen
prefix — so the precedence fix in this PR is primarily relevant to
`token`-mode deployments (or any deployment where cache mode's
frozen-prefix boundary hasn't yet advanced past the affected message).
It has not been independently re-verified live under `--mode token`
after the most recent rebase onto `main` (only the automated test suite
was rerun post-rebase).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A — backend logic change, no UI surface.

## Additional Notes

- "I have made corresponding changes to the documentation" is unchecked:
no file in `docs/`, `README.md`, or `CONTRIBUTING.md` documents
`read_protection_window`, `protect_recent_reads_fraction`, or
`--protect-tool-results` precedence at all, so there was no existing
section to update, and no new section was added either. This is arguably
a pre-existing documentation gap this PR doesn't close.
- No linked issue number: this was found via independent investigation
of a personal deployment, not filed as a `headroomlabs-ai/headroom`
issue first.

Co-authored-by: Ingmar Krusch <ingmar.krusch@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>

CHANGELOG.md

@@ -102,6 +102,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Bug Fixes
 
+* **transforms/content-router:** stop a profile-derived `read_protection_window` kwarg from weakening an explicit `--protect-tool-results` guarantee. `ContentRouter.apply()` computes `read_protection_window` from `protect_recent_reads_fraction`, where `0.0` (the sentinel `--protect-tool-results` sets) means "protect all excluded-tool output regardless of conversation depth" per #1374's documented contract — but the method then unconditionally overwrote that window with a `read_protection_window` kwarg whenever one was present. `proxy_pipeline_kwargs()` supplies that kwarg on every request from the active `AgentSavingsProfile.protect_recent` (the default `coding` profile sets `protect_recent=2`), so in practice only the last 2 messages ever kept read-protection and older excluded-tool output silently fell through to lossy compression. The runtime kwarg may now only narrow the window when `protect_recent_reads_fraction > 0`; it can no longer shrink the "protect everything" guarantee set by `--protect-tool-results`.
 * **memory:** include the Ollama server URL in the embedder cache key so a second backend can't get an embedder bound to the wrong server. `_create_embedder` cached by `(backend, model)` only, but the Ollama embedder is constructed with `base_url=config.ollama_base_url`. Two configs in the same process that shared a backend and model but pointed at different Ollama servers (e.g. a per-project storage router) collided on one cache slot, so the second silently reused the first's embedder and embedded against the wrong host. The cache key now also includes `ollama_base_url`.
 * **tokenizers:** use `o200k_base` for the gpt-4.1 / gpt-4.5 / o4 families in `get_encoding_for_model`. `gpt-4.1*` and `gpt-4.5*` matched the broad `gpt-4` prefix and were encoded with `cl100k_base`, and `o4*` matched no prefix and fell through to the `cl100k_base` default — all three use `o200k_base`, so their token counts were computed with the wrong vocabulary. Added explicit `gpt-4.1`/`gpt-4.5` prefixes ahead of `gpt-4` and an `o4` prefix; `gpt-4` and `gpt-3.5` snapshots still resolve to `cl100k_base`.
 * **cache/ccr:** stop counting a successful eviction as a retrieval in the compression feedback learner, which inverted the learning signal. When an entry is evicted without ever being retrieved, `CompressionStore` emits a synthetic `retrieval_type="eviction_success"` event to mark that the compression was sufficient (the LLM never needed the original). `CompressionFeedback.record_retrieval` had no branch for it, so — because the type is not `"full"` — it was counted as a *search retrieval*, inflating the tool's `retrieval_rate`/`search_rate`. `get_compression_hints` reads a high retrieval rate as "compressing too aggressively" and backs off, so a compression that actually worked pushed the learner toward *less* compression (a standalone repro scores one successful eviction as a 100% retrieval rate). The event is now recognized and left out of the retrieval counters; the compression is still counted by `record_compression` at store time, so a never-retrieved entry correctly yields a low retrieval rate. Genuine retrievals are unaffected.

headroom/transforms/content_router.py

@@ -3297,7 +3297,16 @@ class ContentRouter(Transform):
         else:
             read_protection_window = num_messages  # 0.0 = protect all (old behavior)
         runtime_read_protection_window = kwargs.get("read_protection_window")
-        if runtime_read_protection_window is not None:
+        if (
+            runtime_read_protection_window is not None
+            and self.config.protect_recent_reads_fraction > 0
+        ):
+            # A profile-derived window may only narrow protection when the
+            # deployment hasn't explicitly opted into "protect everything"
+            # (protect_recent_reads_fraction == 0.0, set by --protect-tool-results).
+            # See #1374's documented contract: protected tool output must never
+            # lossy-compress "regardless of conversation depth" -- a per-request
+            # savings-profile kwarg must not silently weaken that.
             read_protection_window = max(0, int(runtime_read_protection_window))
 
         # Adaptive compression ratio: scale with context pressure

tests/test_content_router_exclude_tools.py

@@ -141,6 +141,79 @@ def test_bash_tool_result_passthrough_when_protected() -> None:
     assert "router:excluded:tool" in result.transforms_applied
 
 
+# ---------------------------------------------------------------------------
+# Test 4: protect_tool_results sentinel survives a profile-derived
+# read_protection_window kwarg, even when the protected output is old
+# ---------------------------------------------------------------------------
+
+
+def test_protect_tool_results_survives_runtime_read_protection_window_kwarg() -> None:
+    """A profile-derived `read_protection_window` kwarg (e.g. from
+    AgentSavingsProfile.protect_recent=2, threaded in via
+    proxy_pipeline_kwargs()) must not shrink protection below what
+    protect_recent_reads_fraction == 0.0 (the --protect-tool-results
+    sentinel) already guarantees for the whole conversation.
+
+    Regression test for the precedence bug: content_router.py used to apply
+    the runtime kwarg unconditionally, so a Bash tool_result more than
+    `read_protection_window` messages old fell through to lossy compression
+    even though --protect-tool-results promised it would never compress
+    "regardless of conversation depth" (see PR #1374)."""
+    pytest.importorskip("tiktoken")  # needed for OpenAI tokenizer
+
+    from headroom.providers import OpenAIProvider
+    from headroom.tokenizer import Tokenizer
+
+    provider = OpenAIProvider()
+    token_counter = provider.get_token_counter("gpt-4o")
+    tokenizer = Tokenizer(token_counter, "gpt-4o")
+
+    proxy = _build(protect_tool_results=frozenset({"Bash", "bash"}), mode="token")
+    router = _router(proxy)
+
+    bash_output = "\n".join(
+        f"line {i}: some output from a bash command that is long enough to compress"
+        for i in range(80)
+    )
+    messages: list[dict[str, object]] = [
+        {
+            "role": "assistant",
+            "content": None,
+            "tool_calls": [
+                {
+                    "id": "call_bash_1",
+                    "type": "function",
+                    "function": {"name": "Bash", "arguments": "{}"},
+                }
+            ],
+        },
+        {
+            "role": "tool",
+            "tool_call_id": "call_bash_1",
+            "content": bash_output,
+        },
+    ]
+    # Pad with enough intervening turns that the Bash tool_result above
+    # falls outside a read_protection_window=2 (it's ~9-10 messages from
+    # the end once padding is added).
+    for i in range(8):
+        messages.append({"role": "user", "content": f"follow-up turn {i}"})
+        messages.append({"role": "assistant", "content": f"reply {i}"})
+
+    # Simulate the profile-derived kwarg the proxy threads into every
+    # request via proxy_pipeline_kwargs() (AgentSavingsProfile("coding")
+    # sets protect_recent=2).
+    result = router.apply(messages, tokenizer, read_protection_window=2)
+
+    tool_msg = next(m for m in result.messages if m.get("tool_call_id") == "call_bash_1")
+    assert tool_msg["content"] == bash_output, (
+        "Bash tool_result must stay verbatim: protect_recent_reads_fraction == 0.0 "
+        "(set by --protect-tool-results) must not be weakened by a profile-derived "
+        "read_protection_window kwarg"
+    )
+    assert "router:excluded:tool" in result.transforms_applied
+
+
 # ---------------------------------------------------------------------------
 # Baseline: Bash NOT in DEFAULT_EXCLUDE_TOOLS (unchanged by this PR)
 # ---------------------------------------------------------------------------

tests/test_transforms/test_content_router.py

@@ -1049,6 +1049,60 @@ class TestExcludeTools:
 
         assert "router:excluded:tool" not in result.transforms_applied
 
+    def test_protect_recent_reads_fraction_zero_overrides_runtime_window(self, tokenizer):
+        """protect_recent_reads_fraction == 0.0 (the --protect-tool-results
+        sentinel) means "protect all excluded-tool output forever". A
+        profile-derived read_protection_window kwarg must not be allowed to
+        shrink that back down -- regression test for the precedence bug
+        where the runtime kwarg unconditionally overrode this config-level
+        guarantee."""
+        config = ContentRouterConfig(
+            min_section_tokens=10,
+            min_chars_for_block_compression=10,
+            exclude_tools={"Glob"},
+            protect_recent_reads_fraction=0.0,
+        )
+        router = ContentRouter(config)
+
+        # Plain unstructured text (not grep/log/json-shaped) so
+        # _lossless_compact_excluded returns None and the router takes the
+        # bare "protect as before" branch, matching the tag this test
+        # asserts on.
+        old_tool_content = "\n".join(
+            f"line {i}: some output from a glob command that is long enough to compress"
+            for i in range(80)
+        )
+        messages = [
+            {
+                "role": "assistant",
+                "content": [
+                    {
+                        "type": "tool_use",
+                        "id": "toolu_glob_old",
+                        "name": "Glob",
+                        "input": {"pattern": "*.py"},
+                    }
+                ],
+            },
+            {
+                "role": "user",
+                "content": [
+                    {
+                        "type": "tool_result",
+                        "tool_use_id": "toolu_glob_old",
+                        "content": old_tool_content,
+                    }
+                ],
+            },
+            {"role": "assistant", "content": "ack"},
+            {"role": "user", "content": "continue"},
+            {"role": "assistant", "content": "ack"},
+        ]
+
+        result = router.apply(messages, tokenizer, read_protection_window=2)
+
+        assert "router:excluded:tool" in result.transforms_applied
+
     def test_mixed_excluded_and_non_excluded_tools(self, tokenizer):
         """Multiple tools in same conversation - only excluded ones pass through."""
         config = ContentRouterConfig(