Fixes#816
## What
The three `_persist_to_python_ccr` shims (`search_compressor.py`,
`diff_compressor.py`, `log_compressor.py`) called `store.store(original,
compressed)` with the default key — `SHA-256(original)[:24]` since PR
#395 — while the Rust side embeds `MD5(original)[:24]` in the emitted
`Retrieve more: hash=...` marker. Marker key and storage key never
matched, so **every retrieval of a Rust search/diff/log marker returned
"Entry not found or expired"** (inside any TTL — the symptom class
reported in #714).
Fix is exactly what #816 proposed: pass the marker's key via
`explicit_hash=cache_key` at all three call sites, the same contract
SmartCrusher has used since PR #395. No store changes needed — `store()`
already validates and honors `explicit_hash`. Also corrected the stale
comment in `search_compressor.py` that still claimed "both use
MD5(original)[:24]".
## Tests
`tests/test_ccr_rust_marker_hash_bridge.py` (companion to
`test_ccr_row_drop_store_bridge.py`, which pinned the same bug class for
SmartCrusher in #389): for each shim, the store entry must be
retrievable under the Rust marker key AND absent under the SHA-256
default key.
Verified red→green: all 3 tests fail on main with the exact issue
symptom ("store has no entry under the Rust marker key ...; the marker
dangles") and pass with the fix. `test_ccr_row_drop_store_bridge.py`
still green. `ruff check` + `ruff format --check` clean.
---------
Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
Ports `headroom.transforms.search_compressor` to Rust as the first
consumer of the `signals::LineImportanceDetector` trait shipped in
Phase 3e.1.
The Python regex registry (`_GREP_PATTERN`/`_RG_CONTEXT_PATTERN`)
silently misparsed two real-world inputs. The hand-rolled Rust parser
fixes both:
* **Windows paths.** `^([^:]+):(\d+):(.*)$` captured only the drive
letter from `C:\Users\foo\bar.py:42:line`, then the `\d+` group
failed on `\`. Result: every Windows-formatted line was silently
dropped from `file_matches`. The Rust parser detects the drive
prefix and starts the line-number scan after the drive colon.
* **Filenames with `-`.** `_RG_CONTEXT_PATTERN`'s `[^:-]+` excluded
dashes from the path, so legitimate names like
`pre-commit-config.yaml-42-line` parsed wrong. The Rust parser
anchors on the *line-number marker* (`<sep>\d+<sep>`), so paths
can contain dashes freely.
Two further hardening changes:
* CCR storage failures are loud (Python silently swallowed them).
* Per-file dedup is `O(n log n)` via `BTreeSet<(line_no, content_hash)>`
(Python used linear `match not in file_selected`, worst-case
quadratic for big files).
The Rust `SearchCompressor` owns a `Box<dyn LineImportanceDetector>`
defaulting to `KeywordDetector`. Priority scoring routes through the
trait instead of a hardcoded regex list, so a future BGE classifier
head (per the trait extension docs) can take over without touching
the compressor.
Sidecar `SearchCompressorStats` captures lines unparsed, files
dropped by `max_files`, matches dropped by per-file vs global caps,
and the CCR skip reason -- diagnostics Python never emitted.
`headroom.transforms.search_compressor` is now a thin shim that
delegates `compress()` to Rust end-to-end (so the parser bug fixes
land in production), and keeps the legacy `_parse_search_results`
helper routed through the same Rust parser. The other internal
helpers (`_score_matches`, `_select_matches`, `_format_output`)
stay Python -- they're heavily covered by existing direct-call tests
and Rust scoring is byte-equivalent.
The 4 public dataclasses are unchanged. Tests that monkeypatched the
old internal `_store_in_ccr` helper are updated to exercise the new
`_persist_to_python_ccr` boundary instead.
* 16 Rust unit tests (parser, scoring, selection, CCR round-trip) +
3 explicit `fixed_in_3e2` markers for the bug-fix lines
* 53 Python tests (existing suite intact; 2 updated for new shape)
* `make ci-precheck` clean
Stacks on PR #317 (signals trait module).
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but
74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings,
violating that contract. Every macOS/Linux clone reports these files as
"modified" on fresh checkout because git's diff engine sees the stored
bytes don't match the attribute contract, even though the working tree
and index match byte-for-byte.
Running `git add --renormalize .` rewrites each affected blob so the
stored form matches the attribute declaration. No semantic changes —
every affected file's diff is "N insertions, N deletions" with inserts
and deletes being the same lines modulo line endings.
Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub
blame skip this mechanical commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
