chopratejas 01a423a316 fix(rust): reformat/offload pipeline + log templates + diff noise (Phase 3g rework) ... Replaces PR1's lossless/lossy split with ReformatTransform (pack denser, no info lost) and OffloadTransform (drop bytes, CCR-stash original via required cache_key). With CCR every transform is information-preserving end-to-end, so the lossless/lossy distinction misnamed the architecture. OffloadTransform carries a cheap, structural estimate_bloat() method scoped to its domain — generic byte-redundancy heuristics miss domain semantics. The orchestrator runs reformat phase + per-offload bloat estimation in parallel via rayon::join + par_iter, then runs offload iff bloat clears threshold OR reformat underwhelmed. Transforms shipped: REFORMATS (lossless): - JsonMinifier: serde_json round-trip whitespace stripping. - LogTemplate: Drain-inspired order-preserving template miner. Collapses consecutive runs of same-template lines into [Template Tn: ...] (Nx) + variant table. Win comes from emitting the constant-token prefix once instead of N times. Lossless: every original line reconstructible from template + variants. OFFLOADS (drop bytes, stash original via CCR): - LogOffload: wraps existing LogCompressor; bloat = repetition x uniqueness_weight + dilution x priority_dilution_weight. - DiffOffload: wraps existing DiffCompressor; bloat = context-to- change ratio. Bug-fix-on-port — persists original under the cache_key the parity-bound DiffCompressor mints (closes a leak). - DiffNoise: drops lockfile hunks (Cargo.lock, package-lock.json, yarn.lock, etc., suffix list configurable in TOML) and whitespace-only hunks. Stashes original via CCR for retrieval. Search offload exists but is not in default re-exports — modern agents (Claude Code, Codex) use scoped rg/grep, the marginal value didn't justify default registration. Reach via the explicit module path if opting in. JSON Offload is intentionally absent from this PR — already lives at SmartCrusher; Phase 3g PR3 wraps it in the OffloadTransform contract. Thresholds and weights live in config/pipeline.toml, embedded via include_str!; PipelineConfig::from_toml_str loads runtime overrides. 98 new pipeline tests; full headroom-core suite (714) and workspace tests green; cargo fmt clean. No regex, per project convention.		2026-04-30 11:10:24 -07:00
..
marketplace.json	fix(rust): reformat/offload pipeline + log templates + diff noise (Phase 3g rework)	2026-04-30 11:10:24 -07:00