Fixes#906.
## What
Part of #904 (net-cost policy completion tracking). Follows up #856 /
#857 with the corrected gain term raised in [this #856
comment](https://github.com/chopratejas/headroom/issues/856#issuecomment-4679706939)
— prerequisite for P2 (pipeline consumption), which would otherwise wire
in a formula that is always-pro-mutation by exactly `P_alive·(w−r)·ΔT`.
## Why the corrected form is right
With a live cache, the ΔT tokens a mutation removes are **already
cache-written** — keeping them costs only reads (`ΔT·r·R`), so a
mutation cannot avoid a fresh write of them. Blending alive (`ΔT·r·R −
(w−r)·S`) and dead (`ΔT·(w + r·(R−1))`, no suffix penalty) cases over
`P_alive`:
```
gain = ΔT·(w + r·(R−1)) − P_alive·(w−r)·(S + ΔT)
```
Three independent confirmations:
1. **Direct cost check** (w=1.25, r=0.1, warm, ΔT=50K, S=10K, R=2):
keeping costs 60K·0.1·2 = 12,000 in reads; mutating costs 10K·1.25
(suffix rewrite, the first of the R touches) + 10K·0.1 (remaining read)
= 13,500 — mutation loses 1,500, matching the corrected gain of −1,500.
The old form said +56,000.
2. **The issue's own anchors**: corrected break-even is exactly `R =
11.5·S/ΔT` → 2K/50K = 287.5 (~290, as the issue says) and 50K/10K = 2.3
— the spec text's anchor numbers can only be derived from the corrected
penalty. The implemented form gave 276 and *negative*.
3. **Internal consistency**: `break_even_reads` already shipped with the
~11.5·S/ΔT shape; this PR reconciles `net_mutation_gain` with it (and
drops break_even's stray −1 term).
## Behavior changes (formula is still dead code — nothing consumes it
yet)
- 50K-shave/10K-suffix/R=3 golden: +61,000 → **+3,500** (tight win,
consistent with 2.3-read break-even).
- 2K-shave/50K-suffix/R=10 golden: −53,200 → **−55,500**.
- S=0 boundary: an edit of already-cached content with no suffix is
profitable whenever ≥1 read remains (`gain = ΔT·r·R`), and exactly 0 at
R=0 warm. Not-yet-cached (live-zone) content should bypass the formula —
now documented on both implementations.
Rust + Python goldens updated in lockstep: 13 Rust + 19 Python tests
green.
## Next (separate PRs)
- **P2**: flag-gated consumption (`HEADROOM_NET_COST_POLICY=1`) with
decision telemetry.
- **P3**: batch deep edits (reclaim threshold), idle-timer compaction
near TTL lapse.
Co-authored-by: integration-check <integration@local>
Closes#856
**P1 of the #856 phased plan** — pure functions, zero behavior change.
(Closing keyword links the issue; if P2 hasn't started when this merges,
reopen#856 or it remains the design record for the P2/P3 follow-up
PRs.)
## What
Adds the break-even decision rule for deep (pre-cache-marker) edits to
`CompressionPolicy`:
```
gain = ΔT · (w + r·(R−1)) − P_alive · (w − r) · S
```
- `net_mutation_gain()`, `should_mutate_deep()` (gain > 0),
`break_even_reads()` (R = ((w−r)/r)·(S/ΔT−1) ≈ 11.5·S/ΔT) on the Rust
struct (source of truth) and the Python hand-mirror, following the
existing F2.1/F2.2 parity pattern.
- `CACHE_WRITE_MULTIPLIER = 1.25` / `CACHE_READ_MULTIPLIER = 0.1` public
constants (Anthropic 5-minute tier).
- Inputs clamped (`expected_reads ≥ 0`, `p_alive ∈ [0,1]`); methods take
`&self`/`self` so a follow-up can add per-mode margins.
- The formula derives the existing Subscription live-zone policy as its
S=0 special case rather than contradicting it.
**No callers yet.** P2 (consuming this in `TransformPipeline` behind
`HEADROOM_NET_COST_POLICY`, replacing the binary `live_zone_only` gate,
with decision telemetry) is specified in #856 and awaits maintainer
direction — this PR just lands the audited arithmetic both dispatchers
will share.
## Tests
Golden-value parity: 6 new Rust unit tests and 7 new Python tests assert
the **identical scenario numbers** (loss −53 200 for a 2K shave under a
50K warm suffix at R=10; win +61 000 for a 50K shave under a 10K suffix
at R=3; S=0 always profitable; P_alive=0 always profitable — the
idle-timer window; clamping; break-even 276 reads for the 2K/50K
anchor). A drift on either side trips the pair loudly, same contract as
the existing field-map parity test.
- `cargo test -p headroom-core --lib compression_policy`: 12 passed (6
existing + 6 new)
- `pytest tests/test_compression_policy.py`: 17 passed (10 existing + 7
new)
- `cargo fmt --check`, `cargo clippy -p headroom-core` clean; `ruff
check` + `ruff format --check` clean
## Real behavior proof
Not applicable in the runtime sense — this PR intentionally adds **no
runtime behavior** (pure functions, no call sites). The arithmetic is
validated against the research anchors above in both languages' test
suites; live decision telemetry arrives with P2 where the formula first
gates real traffic.
## Out of scope
P2 (flag-gated pipeline consumption + telemetry), P3 (deep-edit
batching, idle-timer compaction near TTL lapse), retiring the deprecated
`volatile_token_threshold`/`max_lossy_ratio` fields — all tracked in
#856.
---------
Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
Adds three per-mode tuning fields to the F2.1 CompressionPolicy struct
on both sides of the parity bridge:
- volatile_token_threshold (u32 / int) — per-mode threshold below which
content is treated as cache-stable. PAYG=128 (relaxed), Subscription=32
(strict). Plumbed but unconsumed in F2.2 — the volatile detector in
cache_aligner.py is shape-based; wiring it is a follow-up.
- max_lossy_ratio (f32 / float, [0.0, 1.0]) — per-mode upper bound on
lossy compression aggressiveness. PAYG=0.45, Subscription=0.25.
Plumbed but unconsumed in F2.2 — distinct from the caller-driven
target_ratio kwarg in ContentRouter.
- toin_read_only (bool) — TOIN learning gate. True = serve cached
patterns but never write new observations from this request.
PAYG/OAuth=false (network effect feeds on aggressive traffic),
Subscription=true (consistency over learning).
OAuth stays identical to PAYG across all five fields; the canary parity
test (oauth_matches_payg_today) covers the full struct so a future
divergence on any field trips the assertion just as loudly as a flag flip.
Per-mode defaults are CONSERVATIVE pending F2.1 bake telemetry; F2.2-
followup will tune. Per the realignment build constraints, the
configuration IS the per-mode default — no separate env var per field.
What's NOT in this commit:
- TOIN gate wiring (next commit, c2/3)
- policy_selected log extension (next commit, c2/3)
Tests:
- Rust: 6 unit tests in compression_policy::tests (3 new). OAuth=PAYG
canary now compares ALL fields.
- Python: 10 tests in tests/test_compression_policy.py. Hard-coded
expected_fields set in TestRustParityFieldMap extended.
Refs: F2.1 (#400)
Phase F2.1, commit 4 of 5 (consolidated from the 6-commit plan after
finding the Rust dispatcher gate is a no-op in F2.1 — Subscription
still gets live-zone compression by design, so the only behaviour
change is on the Python detector side).
What lands:
- New `headroom/transforms/compression_policy.py` — hand-mirror of
`headroom_core::compression_policy::CompressionPolicy`. Two fields,
`policy_for_mode(AuthMode)` constructor, `policy_default_payg()`
helper for the enforcement-flag-off path. Source of truth is the
Rust crate; a parity test guards against silent drift.
- `CacheAligner.should_apply` now reads `kwargs["compression_policy"]`
and returns `False` when `policy.cache_aligner_enabled is False`.
This is THE behaviour change for F2.1: subscription requests stop
triggering volatility warnings and the per-pipeline-instance
`_previous_prefix_hash` is no longer updated by them.
Hidden state caveat: the hash field is per-pipeline-instance, not
per-request. Clearing it on skip would race with concurrent PAYG
requests on the same pipeline, so we don't. The behaviour is "skip
silently" — exactly what cache-stability-sensitive callers want.
- 6 new tests in `tests/test_compression_policy.py`:
- per-mode field assertions (mirror Rust unit tests)
- `oauth_matches_payg_today` canary for F2.2 divergence
- immutability check (`@dataclass(frozen=True)`)
- field-set parity guard against Rust + Python drift
- 2 new tests in `tests/test_cache_aligner_detector_only.py`:
- subscription policy short-circuits should_apply
- PAYG policy does NOT short-circuit (sanity)
Note: this commit does NOT yet plumb the policy from the proxy
handlers into `pipeline.apply(...)` kwargs. That happens in c5/5
alongside the flag default flip. Until c5/5 lands, the gate is
reachable but unfired — `kwargs["compression_policy"]` is absent
from every production call site, so `policy.get(...)` returns None
and current behaviour is preserved bit-for-bit.
Verified: 28/28 affected Python tests pass.
