headroom/benchmarks
chopratejas e190544c77 fix: B2 — live-zone block dispatcher skeleton
Phase B step 2 of the live-zone-only realignment. Replaces PR-A1's
unconditional "passthrough" stub with a real dispatcher that
inspects the Anthropic /v1/messages body, identifies the live zone
(latest user message at index >= frozen_message_count), and routes
each block to a per-type compressor. PR-B2 wires every per-type
compressor to a no-op, so the dispatcher returns
LiveZoneOutcome::NoChange on every call — bytes-in == bytes-out.
PR-B3+ replaces the no-ops with SmartCrusher, Log, Search, Diff,
and Code compressors.

Adds:
- crates/headroom-core/src/transforms/live_zone.rs — public API:
  - `compress_live_zone(body, frozen_message_count, AuthMode)`
  - `LiveZoneOutcome::{NoChange, Modified}`
  - `CompressionManifest` with per-block outcomes (message_index,
    block_index, block_type, BlockAction).
  - `BlockAction::{NoOpSkeleton, Excluded { reason }}`. The
    HOT_ZONE_BLOCK_TYPES list (`tool_use`, `thinking`,
    `redacted_thinking`, `compaction`) excludes blocks even when
    they appear in the latest user message.
  - `AuthMode::{Payg, OAuth, Subscription}` — accepted but unused
    in B2; PR-F2 wires the auth-mode gate.
  - 12 unit tests pin: empty messages, no messages field, invalid
    JSON, latest user message selection, frozen_count respect,
    hot-zone block exclusion, string-shaped content, no user msg
    in live zone, AuthMode no-op, NoChange contract, manifest
    counters, frozen-count clamping.

- crates/headroom-proxy/src/compression/live_zone_anthropic.rs —
  new entry point. `compress_anthropic_request` parses the body,
  resolves frozen_count via `resolve_frozen_count` (PR-A4 helper),
  dispatches via `compress_live_zone`, and returns
  `Outcome::NoCompression` on PR-B2 success / `Outcome::Passthrough
  { reason: NotJson | NoMessages | ModeOff }` on body-shape /
  policy issues. Six unit tests pin: mode_off short-circuit, no
  messages field, invalid JSON, valid body NoCompression,
  empty body, cache_control disabled.

Modifies:
- compression/mod.rs — re-exports `compress_anthropic_request` from
  `live_zone_anthropic` instead of `anthropic`. The old anthropic
  module is reduced to the `resolve_frozen_count` helper only
  (not deleted, because its CacheControlAutoFrozen-policy gate is
  reused).
- proxy.rs — passes `state.config.cache_control_auto_frozen` into
  the dispatcher. Drops the obsolete "live_zone reserved for
  Phase B" warning that PR-A1 emitted on every request.
- compression/anthropic.rs — pruned to the resolve_frozen_count
  helper plus its tests. The PR-A1 passthrough stub
  `compress_anthropic_request` is gone (live_zone_anthropic owns
  the name now).
- config.rs — `compression_mode` doc updated to reflect the wired
  dispatcher (no longer "reserved for Phase B").
- tests/integration_compression.rs — `compression_decision_logged`
  pins the new log contract (`decision="no_change"`,
  `reason="no_op_skeleton_pr_b2"`, plus manifest fields
  `frozen_message_count`, `messages_total`, `live_zone_blocks`).
  Asserts the obsolete Phase A warning is NOT emitted.
- proxy.rs no longer imports CompressionMode (only used inside the
  retired warning).

Benchmark cleanup (B1 leftovers that surfaced now):
- benchmarks/proxy_mode_benchmark.py + claude_session_mode_benchmark.py:
  drop `intelligent_context=False` arg from ProxyConfig (the field
  was retired in B1; tests/test_proxy_mode_benchmark.py and
  tests/test_claude_session_mode_benchmark.py imported these
  factories and started failing).
- benchmarks/bench_transforms.py: delete TestRollingWindowBenchmarks
  class; rewire TestTransformPipelineBenchmarks fixture without
  RollingWindow.
- benchmarks/conftest.py: drop rolling_window_config fixture.
- benchmarks/run_benchmarks.py: drop the `window` suite + table
  rows referencing RollingWindow.

Cache-safety invariant:
- PR-B2 dispatcher never mutates body bytes (no-op skeleton). The
  proxy forwards the original buffered bytes byte-equal. Phase A's
  SHA-256 fixtures pin this.
- `passthrough_mode_live_zone_currently_passthrough_byte_equal_sha256`
  retitled comment to reflect the dispatcher being live but
  no-op.

Acceptance:
- cargo build --workspace + clippy + fmt: green.
- cargo test --workspace --exclude headroom-py: all green
  (777 + 12 new live_zone + 6 new live_zone_anthropic tests).
- pytest: 4678 passed, 240 skipped, 0 failed.
- Anthropic decision log includes manifest fields per the
  observability contract documented in
  REALIGNMENT/02-architecture.md.

Per-PR-B2 plan: REALIGNMENT/04-phase-B-live-zone.md.
2026-05-02 12:45:43 -07:00
..
scenarios Fix all ruff lint and format errors for CI 2026-01-10 15:33:44 -08:00
__init__.py Fix all ruff lint and format errors for CI 2026-01-10 15:33:44 -08:00
adversarial_ccr_tests.py Fix all ruff lint and format errors for CI 2026-01-10 15:33:44 -08:00
agent_cost_benchmark.py Fix all ruff lint and format errors for CI 2026-01-10 15:33:44 -08:00
bench_latency.py chore(transforms): retire dead text_compressor module (Phase 3e.3) 2026-04-29 21:14:36 -07:00
bench_relevance.py Fix all ruff lint and format errors for CI 2026-01-10 15:33:44 -08:00
bench_transforms.py fix: B2 — live-zone block dispatcher skeleton 2026-05-02 12:45:43 -07:00
cache_bust_trace_report.py test: align anthropic cache stability fixtures 2026-04-06 23:26:51 -05:00
cache_validation_bundle.py test: align anthropic cache stability fixtures 2026-04-06 23:26:51 -05:00
ccr_regression_benchmark.py Fix all ruff lint and format errors for CI 2026-01-10 15:33:44 -08:00
claude_session_branch_compare.py test: align anthropic cache stability fixtures 2026-04-06 23:26:51 -05:00
claude_session_mode_benchmark.py fix: B2 — live-zone block dispatcher skeleton 2026-05-02 12:45:43 -07:00
comprehensive_eval.py Add quality retention eval and fix linting for Python 3.12 2026-01-19 21:52:18 -08:00
compression_benchmark.py chore: add nosec B324 annotations to non-cryptographic MD5 usages and update temporary database path to use system temp directory 2026-04-07 13:07:26 +06:00
conftest.py fix: B2 — live-zone block dispatcher skeleton 2026-05-02 12:45:43 -07:00
dynamic_detector_benchmark.py Fix all ruff lint and format errors for CI 2026-01-10 15:33:44 -08:00
headroom_adversarial_benchmark.py chore: add nosec B324 annotations to non-cryptographic MD5 usages and update temporary database path to use system temp directory 2026-04-07 13:07:26 +06:00
headroom_worst_case_benchmark.py chore: add nosec B324 annotations to non-cryptographic MD5 usages and update temporary database path to use system temp directory 2026-04-07 13:07:26 +06:00
prefix_cache_benchmark.py Format prefix_cache_benchmark.py for ruff 0.15 compatibility 2026-03-12 22:03:38 -07:00
proxy_mode_benchmark.py fix: B2 — live-zone block dispatcher skeleton 2026-05-02 12:45:43 -07:00
real_world_agent_benchmark.py chore: add nosec B324 annotations to non-cryptographic MD5 usages and update temporary database path to use system temp directory 2026-04-07 13:07:26 +06:00
run_benchmarks.py fix: B2 — live-zone block dispatcher skeleton 2026-05-02 12:45:43 -07:00
synthetic_long_cache_suite_report.py test: align anthropic cache stability fixtures 2026-04-06 23:26:51 -05:00
synthetic_token_cache_bust_report.py test: align anthropic cache stability fixtures 2026-04-06 23:26:51 -05:00