Closes#858.
## What
Adds an opt-in **Markdown-KV** renderer to the lossless-first compaction
stage, plus the plumbing to pick a compaction formatter by name. Default
behavior is unchanged (`csv-schema`).
Format-comprehension benchmarks show models retrieve values from
Markdown-KV substantially more reliably than from CSV (~60.7% vs ~44.3%)
— token-cheapest is not the same as most comprehensible. This makes the
trade-off selectable per workload.
## How
- **`MarkdownKvFormatter`** (`compaction/formatter.rs`): keeps the
`[N]{cols}` declaration line, renders each row as a Markdown list item
with `key: value` lines.
- Missing cells omitted entirely (the KV advantage over positional CSV).
- Strings ambiguous on a line (newlines, leading/trailing whitespace,
empty) render JSON-quoted; everything else raw — commas and quotes need
no escaping.
- Nested cells inline compact JSON; opaque cells keep the fixed
`<<ccr:HASH,KIND,SIZE>>` marker contract shared by all formatters.
- **`CompactionStage::from_format_name`** maps `"csv-schema" | "json" |
"markdown-kv"` to presets.
- **Core**: `SmartCrusher::with_compaction_format(config, name)` —
standard OSS composition with the named formatter.
- **PyO3 bridge**: `SmartCrusher.with_compaction_format(config,
format_name)` staticmethod; `ValueError` on unknown names (loud, no
silent fallback).
- **Python**: `SmartCrusher(compaction_format=...)` kwarg, falling back
to the `HEADROOM_COMPACTION_FORMAT` env var, default `"csv-schema"`.
## Safety
- **Default-off**: the default constructor path still calls the Rust
`new()` constructor, so byte-parity coverage stays on the exact
production codepath. A test asserts default output is byte-identical to
an explicit `csv-schema` opt-in.
- The existing `lossless_min_savings_ratio` gate (0.30) still applies.
Markdown-KV repeats field names per row, so it clears the gate less
often than CSV and falls through to the lossy path — we never inline a
"lossless" rendering that isn't actually smaller.
- CCR marker format unchanged across formatters; downstream retrieval
pattern-matching keeps working.
- No user/assistant content dropped — the formatter is a pure rendering
of the same Compaction IR.
## Tests
- Rust: 10 new unit tests in `compaction/formatter.rs` (table/buckets
rendering, missing-cell omission, string quoting, CCR markers, drop
summary, byte-size sanity vs raw JSON). `cargo test -p headroom-core`:
894 passed. Clippy + fmt clean.
- Python: `tests/test_compaction_markdown_kv.py` (10 tests) — bridge
rendering end-to-end, name→preset parity with the default constructor,
kwarg/env knob precedence, loud failure on unknown names,
default-output-unchanged guarantee. Existing smart_crusher suite: 38
passed.
---------
Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
