perf(proxy): bound upstream calls and hot-path costs (#2852)

Seven commits from one week of load testing: one hang, two request-path
correctness fixes, and four hot-path costs that only show up in
production.

## Reliability

**Bound every upstream call.** The litellm backend had no timeout at
all, so a
request the upstream never answered blocked its caller forever. Observed
under
load on 2026-08-07: four agent workers on ESTABLISHED connections for
36+
minutes while `/readyz` answered in 0.11s. No error, no retry, no log
line —
indistinguishable from slow work, which is the worst shape a failure can
take.

A float rather than an `httpx.Timeout`, deliberately: litellm expands a
float
across all four httpx phases, so on a streaming call it becomes the
maximum gap
*between chunks*, not a cap on total generation. A long answer streaming
steadily is never cut off; a stalled one dies. Default 600s via
`HEADROOM_UPSTREAM_TIMEOUT`; 0, negative, and junk fall back to the
default
rather than meaning "no timeout".

**Keep the consistency re-count off the event loop.** It ran
`tokenizer.count_messages` twice directly on the loop. Since Claude
counting
moved to a real BPE that is CPU-bound work stalling every other
in-flight
request — ~1s on a 2.3 MB body, with `/healthz` gaps tracking body size.
Offloaded via `asyncio.to_thread` on the same tokenizer instance, so
reported
values are unchanged. (#2810)

**Survive a re-parse MemoryError.** `MemoryError` is not a `ValueError`,
so on
1M-context payloads the byte-faithful forwarder's verification re-parse
escaped
the handler and aborted an otherwise-fine request — 14 aborts across 8
days of
reporter logs. (#2768)

## Performance

All four are measured, not guessed. Each degrades with something a short
benchmark does not vary: uptime, content shape, or process age.

| fix | before | after |
|---|---|---|
| Cost-record walk per request (at 100k records) | 13.6 ms | bounded by
model count |
| JSON-block scan, JS-style object logs (1200 lines) | 4643 ms | 183 ms
|
| JSON-block scan, truncated JSONL | 3737 ms | 116 ms |
| Lazy imports inside user requests | multi-second | paid at startup |
| `count_text` (80% of local CPU) | — | memoised |

Two worth calling out:

- **The cost walk degrades with proxy *uptime*, not load.** A freshly
started
proxy pays ~0.01 ms; a month-old one pays 4–13 ms on every request, on
the
event loop, holding the metrics lock. Deliberately not a TTL cache over
`stats()`: those values feed `check_budget()` when `--budget` is set,
and a
stale reading under-enforces the budget. The fix is to stop computing
what
  the caller discards.
- **The JSON-block memo is built only *after* a scan fails to balance.**
That
ordering is load-bearing, not an optimisation — caching from the start
made
pretty-printed JSON ~2x slower, since content that balances on the first
scan
  has nothing to reuse and just pays the per-line dict traffic. Still a
  constant-factor fix, not an asymptotic one.

## Tests

+1202 lines, 20 files. Each fix is pinned by a test that fails on the
unmodified code: the re-count test asserts no `count_messages` pass runs
with a
live event loop in its thread; the re-parse test drives a `MemoryError`
through
the real request path and expects a 200; `totals()` equality with
`stats()` is
asserted across model counts, request volumes, and both pricing
branches. The
timeout test is structural rather than a mock — the failure mode is a
dispatch
path someone adds later without a guard, which mocking the existing four
cannot
catch.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

docs/context-mode-integration-analysis.md

@@ -0,0 +1,378 @@
+# context-mode → Headroom: enterprise plugin & variant analysis
+
+Analysis date: 2026-07-29. Sources: `/Users/tcms/demo/context-mode` @ v1.0.169, `/Users/tcms/demo/headroom` @ main.
+
+---
+
+## 1. Bottom line
+
+context-mode and Headroom attack the same cost problem at **two different layers**, and they do not
+overlap where it matters:
+
+| | context-mode | Headroom |
+|---|---|---|
+| Interception point | agent **tool-call boundary** (host hooks + MCP) | model **API boundary** (proxy / SDK / MCP) |
+| Position relative to context | **pre-context** — data never enters | **in-context** — data already entered, gets squeezed |
+| Mechanism | admission control: block, redirect, sandbox, externalize | compression: crush, cache, retrieve |
+| Touches the wire request | never | always |
+| Loss | lossless (full content in FTS5, queryable) | lossy squeeze + hash rehydrate |
+
+Headroom's own realignment doc identifies its correct compression target as the **live zone**:
+"latest user message content + latest `tool_result` + latest `function_call_output` + latest
+`local_shell_call_output`" (`REALIGNMENT/00-overview.md`, Phase B).
+
+**That is precisely the payload context-mode intercepts one layer earlier.** Headroom Phase B is
+building a Rust engine to compress the latest tool result *after* it hits the wire. context-mode
+stops that tool result from being produced at all. These are complements, not competitors — and the
+upstream position is strictly cheaper: nothing to compress, nothing to cache-invalidate, no
+token-validation fallback needed.
+
+Three strategic unlocks, in order of value:
+
+1. **Cache safety.** Headroom's #1 identified bug class is prompt-cache busting from request
+   mutation (5 top-tier cache-killer bugs, `REALIGNMENT/00-overview.md`). context-mode has
+   *structurally zero* cache-bust risk because it never touches the request body.
+2. **Subscription safety.** The realignment flags "fingerprint-class subscription-revocation
+   risks" from `X-Headroom-*` header leakage, `anthropic-beta` mutation and re-serialization on
+   OAuth/subscription CLIs. A hook-layer product carries none of this — it is invisible to the
+   upstream. This is a *deployable-where-the-proxy-can't-go* capability.
+3. **Proxy-free deployment.** Headroom's value today requires being in the API path
+   (`127.0.0.1:8787`). Verified live this session: with the proxy down, `headroom_stats` returns all
+   zeros and `headroom_compress` no-ops. Enterprises that cannot reroute model traffic (TLS trust,
+   egress policy, subscription auth) currently get nothing. context-mode's hook+MCP model needs no
+   interposition.
+
+Zero references to context-mode exist in the Headroom tree today — clean slate.
+
+---
+
+## 2. context-mode: portable IP inventory
+
+41,617 lines of TypeScript, 11 MCP tools, 18 host adapters, npm-distributed
+(`context-mode@1.0.169`, 8 runtime deps, esbuild-bundled).
+
+Ranked by *how hard it would be for Headroom to rebuild*:
+
+### Tier 1 — genuinely hard, no Headroom equivalent
+
+**1. Cross-host hook adapter layer** — `src/adapters/**` (~10K LOC), `src/adapters/types.ts`,
+`src/adapters/detect.ts` (737 lines), `configs/` (18 hosts).
+Normalizes three incompatible paradigms — `json-stdio` (Claude Code, Gemini/Qwen, Copilot, Codex,
+Kimi, Cursor, Kiro, Antigravity), `ts-plugin` (OpenCode, KiloCode, OpenClaw), `mcp-only` (Zed, Pi,
+OMP) — behind one contract: normalized `PreToolUse` / `PostToolUse` / `PreCompact` /
+`SessionStart` events, a `PlatformCapabilities` matrix, and a 5-way decision
+(`allow | deny | modify | context | ask`). Per-host install, config-format, and self-heal machinery
+included (`hooks/heal-partial-install.mjs`, `scripts/plugin-cache-integrity.mjs`).
+*Why hard to rebuild:* the value is entirely in the accumulated per-host quirks. There is no spec to
+implement against.
+
+**2. Tool-boundary policy engine** — `src/security.ts` (889 lines).
+A real policy decision point, not a regex list: glob→regex compilation, chained-command splitting
+(`&&`/`;`/`|` with escape awareness), subshell extraction, deny/ask pattern ingestion from host
+settings files, project-boundary containment (`evaluateProjectContainment` — Issue #852: an approved
+`ctx_execute_file` cannot escape the repo via a path the user couldn't see), and a
+**shell-escape scanner** (`SHELL_ESCAPE_PATTERNS`, `extractShellCommands`) that detects
+`execSync`/`subprocess`/etc. embedded inside sandboxed *non-shell* code and re-evaluates the escaped
+command against policy.
+*Why hard to rebuild:* this is the sandbox-escape prevention layer. Getting it wrong is a CVE.
+
+**3. Multi-language sandbox executor** — `src/executor.ts` (785), `src/runPool.ts`,
+`src/exit-classify.ts`, `src/truncate.ts`.
+12 languages, stdout-only egress, timeouts, background detach, output caps, exit classification.
+Enforces the "Think in Code" contract: the agent programs the analysis, only the answer enters
+context.
+
+**4. Lossless externalization store** — `src/store.ts` (2,071 lines).
+Dual SQLite FTS5 index — a tokenized `chunks` table *plus* a `chunks_trigram` table for
+substring/identifier search where BM25 tokenization fails on code — with a `vocabulary` table and
+schema migration path. Auto-externalizes any output >100 KB into FTS5 and returns a pointer.
+Nothing is discarded; the model queries on demand.
+
+### Tier 2 — valuable, but partially duplicated in Headroom
+
+**5. Counterfactual savings accounting** — `src/session/analytics.ts` (3,085 lines),
+`src/session/project-attribution.ts`, `src/session/db.ts` (1,726).
+`ContextSavings`, `ThinkInCodeComparison`, `RealBytesStats`, `MultiAdapterLifetimeStats`,
+`enumerateAdapterDirs()`. Measures *what would have entered context but didn't* — a different and
+harder quantity than Headroom's `savings_ledger.py`, which records actual compression deltas.
+Session event ledger + `tool_calls` + resume + per-project attribution.
+
+**6. Multi-vendor pricing catalog** — `src/session/pricing.ts` + `model-prices.json`.
+61 curated models × 4 rate buckets (input / output / cache-read / cache-write), refreshed from
+litellm, unknown model → `null` rather than a silently wrong Claude rate.
+**Overlaps `headroom/pricing/*` heavily. Do not port.**
+
+### Tier 3 — do not port
+
+Compression heuristics, memory/graph/relevance, telemetry transport, dashboard, install UX,
+update-check. Headroom has all of these, more mature, and Phase B/H is actively consolidating them.
+
+---
+
+## 3. Headroom's actual extension seams
+
+Verified entry-point groups (all `importlib.metadata`-discovered, all opt-in):
+
+| Seam | Group | Contract | Source |
+|---|---|---|---|
+| Proxy extension | `headroom.proxy_extension` | `install(app: FastAPI, config: ProxyConfig) -> None` | `headroom/proxy/extensions.py:52` |
+| Pipeline extension | `headroom.pipeline_extension` | `on_pipeline_event(PipelineEvent) -> PipelineEvent \| None` over 11 stages | `headroom/pipeline.py:13,68` |
+| Learn plugin | `headroom.learn_plugin` | — | `headroom/learn/registry.py:44` |
+| Memory text store | `headroom.memory_text` | — | `headroom/memory/config.py:41`, `factory.py:57` |
+| Memory vector store | `headroom.memory_vector` | — | `headroom/memory/config.py:34` |
+| Memory store | `headroom.memory_store` | — | `headroom/memory/config.py:25` |
+| CCR backend | `headroom.ccr_backend` | — | `headroom/cache/compression_store.py:981` |
+| Compression hooks | (subclass, not entry point) | `pre_compress` / `compute_biases` / `post_compress` | `headroom/hooks.py:1-31` |
+
+Two things worth noting:
+
+- `headroom/proxy/extensions.py:32` states an explicit **stability contract**: changing
+  `install(app, config)` or the group name requires a deprecation cycle. This is a supported public
+  seam, not an accident.
+- `headroom/hooks.py:16` says outright: *"Headroom SaaS implements position-aware compression and
+  cross-turn deduplication via these hooks."* The open-core split is already designed in.
+
+**The exemplar to copy:** `plugins/headroom-oauth2/` — own `pyproject.toml`, own `LICENSE`, own
+`SPEC.md`, registers on `headroom.proxy_extension`, dormant until `--proxy-extension oauth2`,
+all config via env, "zero core changes." That is the enterprise plugin template.
+
+**The precedent to copy:** `headroom/lean_ctx/installer.py` and `headroom/rtk/installer.py` —
+Headroom already ships thin installers that adopt sibling products. `plugins/headroom-agent-hooks`
+already installs startup hooks into Claude Code and Copilot CLI. The socket exists.
+
+**The gap:** Headroom has *no tool-boundary interception anywhere*. It sees `tool_use`/`tool_result`
+only as message content after the fact (`headroom/parser.py`, `headroom/tokenizers/*`). Its
+`PipelineStage` enum has no tool-result stage. Everything context-mode does is upstream of
+Headroom's earliest hook.
+
+---
+
+## 4. Proposed plugins & variants
+
+Ranked by value ÷ effort.
+
+### P1 — `headroom-recall`: FTS5+trigram lossless store as `headroom.memory_text`
+
+**What:** port `src/store.ts` behind the existing `headroom.memory_text` seam.
+
+**Why this first:** it is the smallest diff onto an *already-existing* contract, and it fixes a real
+product limitation. Today `headroom_retrieve(hash)` requires you to *know the hash* — the tool
+description literally says "hash comes from compression markers like `[N items compressed... hash=abc123]`".
+With an FTS5-backed store you get `retrieve-by-query`: "what did that build log say about OOM"
+instead of "paste hash abc123". The trigram index matters specifically because BM25 tokenization
+loses identifiers and stack frames.
+
+Composes rather than replaces: `compress` → return squeezed text + hash → store the *original* in
+FTS5 → rehydrate by hash **or** by query. Also a natural `headroom.ccr_backend` implementation —
+the realignment wants "CCR hardens: persistent backend" (Phase B), and this is one.
+
+**Enterprise variant:** shared team store, retention/TTL policy, per-project scoping (context-mode
+already has `project-attribution.ts`), audit of every retrieval.
+
+**Effort:** medium. Reimplement in Python/Rust against Headroom's memory interface, or ship the
+node store as a sidecar. Do not port the MCP tool surface — only the store.
+
+### P2 — `headroom-admission`: tool-boundary admission control across 18 hosts
+
+**What:** context-mode's adapter + hook layer, distributed the way `plugins/openclaw` and
+`plugins/opencode` already are (TS package under `plugins/`), reporting savings into Headroom's
+`savings_ledger.py` JSONL and emitting Headroom pipeline events.
+
+**Why:** this is the strategic piece. It gives Headroom:
+- a **pre-wire** enforcement point, upstream of Phase B's live-zone engine, with no cache-bust and
+  no token-validation fallback required;
+- coverage of **18 agent hosts** — the realignment's Phase G wants to "extend wrap CLIs (cline,
+  continue, goose, openhands)"; this is that work already done, and then some;
+- a deployment mode that works under **subscription auth**, where the proxy is a revocation risk.
+
+**Enterprise value — this is the DLP story Headroom cannot currently tell.** A `curl` inside a Bash
+tool call never touches the proxy, so Headroom is blind to it. context-mode blocks
+`curl`/`wget`/`WebFetch`/inline `fetch()`/`requests.get` at the tool boundary and forces network
+egress through `ctx_fetch_and_index`. That converts a token-savings feature into an
+**egress-control** feature — a different budget line and a different buyer.
+
+**Effort:** high, but it's mostly packaging + a reporting bridge, not a rewrite. Keep it TypeScript;
+Phase H retires Python *proxy* code but explicitly preserves "CLI wrappers, RTK installer" — the
+installer layer is the surviving Python, and it can shell out.
+
+### P3 — `headroom-policy` (Enterprise, license-gated): the PDP
+
+**What:** `src/security.ts` as a policy decision point, plus centrally-managed org rulesets.
+
+Two attach points: the hook layer from P2 (tool-level `allow/deny/ask`), and
+`headroom.pipeline_extension` at `PRE_SEND` (prompt-level policy). Feeds `headroom/audit/`.
+
+**Enterprise features that only make sense paid:** central policy service, org-wide allow/deny
+rulesets, project-boundary containment enforcement, shell-escape detection inside sandboxed code,
+tamper-evident audit trail, per-team reporting. Gate it with the ELv2 license key (see §6).
+
+**Effort:** medium. The engine exists and is tested (`tests/security/`, `src/security.ts` 889 lines);
+the work is the control plane.
+
+### P4 — `headroom-sandbox`: Think-in-Code execution
+
+**What:** `executor.ts` exposed as a Headroom MCP tool (`headroom_execute`), 12 languages,
+stdout-only.
+
+**Why:** this is the mechanism behind context-mode's largest measured savings —
+`ctx_execute_file` returns 98% savings across 315 KB of real fixtures (`BENCHMARK.md` Part 1),
+versus 82% for index+search (Part 2). Programming the analysis beats compressing the output.
+
+Must ship *with* P3: the shell-escape scanner is what stops the sandbox being an escape hatch.
+
+**Effort:** medium-high. Runtime isolation is the hard part; `headroom` already has a `sandbox` extra
+in `pyproject.toml` to build on.
+
+### P5 — `headroom-attribution`: counterfactual savings + per-project cost
+
+**What:** port the *methodology* from `session/analytics.ts` — `RealBytesStats`,
+`ThinkInCodeComparison`, `enumerateAdapterDirs`, `project-attribution.ts` — into Headroom's
+`savings_ledger` / `reporting` / `dashboard`.
+
+**Why:** Headroom measures compression deltas (what it squeezed). context-mode measures the
+counterfactual (what never entered). Enterprise buyers want the second number, sliced by team and
+repo. Do **not** port `pricing.ts` — `headroom/pricing/*` already does this with litellm resolution.
+
+**Merge, don't port.** `headroom/audit/reads.py` is already a counterfactual measurement tool over
+the same Claude Code transcript corpus (see §8). It has the better mechanism taxonomy — identical
+repeat, subset containment, write-readback, stale, line-number scaffolding, context residency,
+cache-death windows. `analytics.ts` has the multi-host coverage and per-project attribution it
+lacks. Combine the two rather than adding a third implementation.
+
+**Effort:** low-medium, mostly a metrics-definition merge.
+
+### Variants (packaging, not code)
+
+- **Headroom No-Proxy Edition** — P1+P2 only, zero API interposition. Sells to buyers who cannot
+  reroute model traffic and to every subscription-auth user. Removes the single biggest deployment
+  blocker Headroom has.
+- **Headroom Admission Control (Enterprise)** — P2+P3+P4 with a central policy plane and fleet
+  enrollment across 18 hosts. Positioned as AI-agent DLP/governance, not token savings.
+- **Headroom Fleet** — P5 + `enumerateAdapterDirs` for org-wide rollout state and cost reporting.
+
+---
+
+## 5. Evidence base
+
+context-mode's `BENCHMARK.md`: 21 scenarios, 376 KB raw → 16.5 KB context, **96% overall**, all
+fixtures captured from real tool invocations (Context7, Playwright, `gh`, vitest, tsc, nginx logs,
+`git log`, analytics CSV) rather than synthetic. Honest about its weak cases — 13% on a 0.4 KB
+Playwright network dump, and Part 2 openly explains why index+search only reaches 50-93% (it returns
+exact code blocks rather than summaries, by design).
+
+Test suite: 125 tests across executor/store/MCP-integration/ecosystem, plus 45 test dirs in `tests/`
+covering adapters, security, session, hooks, analytics.
+
+That's a defensible enough evidence base to reuse in Headroom's own materials, and the fixture corpus
+itself is reusable for Headroom's `benchmarks/`.
+
+---
+
+## 6. Blockers — resolve these before writing code
+
+**1. License incompatibility (hard blocker).**
+context-mode is **Elastic License 2.0**, "Copyright 2026 Mert Koseoglu". Headroom is
+**Apache-2.0**, "Copyright 2025 Headroom Contributors".
+
+- ELv2 code **cannot** be merged into the Apache-2.0 core. Not a technicality — it would relicense
+  Headroom's core.
+- ELv2 forbids providing the software "to third parties as a hosted or managed service." That
+  directly constrains `headroom-managed/`.
+- Different copyright holders means this needs an **IP arrangement between entities**, not an
+  engineering decision.
+
+The good news: Headroom's plugin architecture is exactly the boundary that makes this tractable.
+A separate package with its own `pyproject.toml` and its own `LICENSE`, registered on an entry
+point — the `plugins/headroom-oauth2/` shape — can carry ELv2 while core stays Apache-2.0. ELv2 is
+also the *right* license for a license-key-gated enterprise tier; it explicitly contemplates one.
+
+Recommendation: any context-mode-derived code ships as separately-licensed plugin packages under
+`plugins/`, never vendored into `headroom/`. Get the IP arrangement in writing first.
+
+**2. Realignment collision.**
+Phases A–I are ~40 PRs / 8–13 weeks and include deleting ~25K LOC. Do not open a new integration
+front mid-Phase-B. P1 (`headroom.memory_text` / `ccr_backend`) is the exception — it *serves* Phase
+B's "CCR hardens: persistent backend" goal rather than competing with it.
+
+**3. Phase H direction.**
+Python proxy code is being retired. Write nothing new in `headroom/proxy/`. Target the surviving
+layers: installers, memory writers, CLI wrappers, and Rust.
+
+---
+
+## 7. Sequencing
+
+| Order | Item | Gate |
+|---|---|---|
+| 0 | IP/licensing arrangement | before any code |
+| 1 | P1 `headroom-recall` — FTS5 store on `memory_text`/`ccr_backend` | lands inside Phase B, serves it |
+| 2 | P2 `headroom-admission` — 18-host hook layer under `plugins/` | after Phase A stabilizes |
+| 3 | Variant: **No-Proxy Edition** = P1+P2 | as soon as P2 works on 3+ hosts |
+| 4 | P3 `headroom-policy` (Enterprise, ELv2, key-gated) | after P2 |
+| 5 | P4 `headroom-sandbox` | with P3, never before |
+| 6 | P5 `headroom-attribution` | opportunistic |
+
+---
+
+## 8. Follow-up verification
+
+All four items flagged as open in the first pass are now resolved.
+
+**`headroom-managed/` is the SaaS arm, and it is unlicensed.**
+`headroom-managed/pyproject.toml`: `name = "headroom-managed"`, `description = "Headroom SaaS
+Platform - Managed context window optimization"`, `version = 0.1.0`. It has `app/auth.py`,
+`app/middleware/`, `app/routes/`, `app/services/`, `app/models.py`, alembic migrations, and a
+`pilot/`. There is **no `license` field and no LICENSE file** — i.e. proprietary by default.
+
+This *sharpens* the §6 blocker rather than easing it. ELv2 forbids providing the software "to third
+parties as a hosted or managed service." The product whose name is literally *Managed* is the one
+place context-mode-derived code cannot go without an explicit commercial grant from the copyright
+holder. Plan the plugin boundary so that `headroom-managed` consumes only Apache-2.0 core
+interfaces, never ELv2 implementations.
+
+**`headroom/audit/reads.py` does not overlap P3 — and it independently validates the whole thesis.**
+It is a *measurement* tool, not an audit trail: it streams Claude Code `*.jsonl` transcripts to size
+"the addressable bytes for each Read compression mechanism... so defaults are set from traffic, not
+theory." No policy, no tamper-evidence. P3's audit trail remains a gap.
+
+Two lines in its docstring are the most useful corroboration in either repo:
+
+- *"context residency — how many assistant turns each Read stays in context (the multiplier on its
+  prefix-cache read cost; **the case for compress-before-cache-entry**)"* — Headroom is already
+  arguing, from its own traffic, for moving earlier in the pipeline. context-mode is the terminus of
+  that argument: compress before **context** entry, not merely before cache entry.
+- *"identical repeat — a dedup mechanism for this was prototyped and removed: it measured 0.1% of
+  Read bytes on real traffic."* — Headroom has already empirically established that
+  message-history-level dedup is worthless. The addressable bytes are at the tool boundary, not in
+  history. That is the same conclusion the realignment reached from the cache side, arrived at
+  independently from the traffic side.
+
+It *does* overlap **P5** — `audit/reads.py` and context-mode's `session/analytics.ts` are two
+independent implementations of counterfactual measurement over the same transcript corpus. Merge
+them rather than porting; `audit/reads.py` has the better mechanism taxonomy, `analytics.ts` has
+multi-host coverage and per-project attribution.
+
+**No plugin-authoring docs exist.** `docs/` is a Next.js site (`app/`, `content/`, `components/`);
+`wiki/` has nothing on extension authoring (only `macos-deployment.md` matched). `plugins/headroom-oauth2/SPEC.md`
+remains the de-facto authoring reference — which means whichever plugin lands first sets the house
+style. Worth writing the authoring doc as part of P1.
+
+**Headroom publishes no benchmark results.** `benchmarks/` is 29 runner scripts with no committed
+results artifacts, so no like-for-like number exists to compare against context-mode's 96%. The
+comparison has to be run. The harness is there and is unusually strong on exactly the axis that
+matters: `prefix_cache_benchmark.py`, `cache_bust_trace_report.py`, `cache_validation_bundle.py`,
+`synthetic_token_cache_bust_report.py`, `proxy_mode_benchmark.py`, `agent_cost_benchmark.py`,
+`real_world_agent_benchmark.py`. Use it to *prove* the §1 cache-safety claim empirically rather than
+asserting it — a measured "zero cache-bust events" result is the strongest possible artifact for the
+No-Proxy Edition.
+
+**Bonus finding — the platform axes are orthogonal.**
+`docs/platform-feature-matrix.json` (schema v1, updated 2026-07-06) tracks coverage across
+`["linux", "macos", "windows"]` — Headroom's platform axis is **operating system**. context-mode's
+platform axis is **agent host** (18 of them). Headroom tracks no host-coverage matrix at all. P2
+therefore fills a dimension that does not currently exist in Headroom's own feature accounting,
+which also means it needs a second matrix rather than new rows in this one.
+
+*Process note:* six subagents were dispatched across this analysis and all six stalled at the
+600-second watchdog; one reported "Bash is temporarily unavailable" before dying, so the failures
+were tool-layer, not analytical. Every finding in this document was verified directly.

headroom/backends/litellm.py

@@ -413,6 +413,40 @@ PROVIDER_REGISTRY: dict[str, ProviderConfig] = {
 }
 
 
+# How long an upstream call may go silent before we give up on it.
+#
+# WHY THIS EXISTS. There was no timeout here at all, so a request the upstream
+# never answered blocked its caller forever. Observed 2026-08-07 under load:
+# four agent workers sat on ESTABLISHED connections for 36+ minutes while this
+# proxy answered /readyz in 0.11s. No error, no retry, no log line -- the
+# client just stops. That is the worst shape a failure can take, because it is
+# indistinguishable from slow work and no supervisor can tell the difference.
+#
+# A float, not an httpx.Timeout, on purpose: litellm expands a float into all
+# four httpx phases, so for a STREAMING call this becomes the maximum gap
+# BETWEEN CHUNKS rather than a cap on total generation time. A long answer
+# streaming steadily is never cut off; a stalled one dies. That is the
+# semantic we want, and it falls out of the simpler type.
+#
+# 600s is deliberately generous -- long enough that no healthy call is at
+# risk, short enough that a hang surfaces within a coffee break instead of
+# never.
+UPSTREAM_TIMEOUT_ENV = "HEADROOM_UPSTREAM_TIMEOUT"
+DEFAULT_UPSTREAM_TIMEOUT = 600.0
+
+
+def _upstream_timeout() -> float:
+    """Seconds. Never raises; a junk env value must not disable the timeout."""
+    import os
+
+    try:
+        v = float(os.getenv(UPSTREAM_TIMEOUT_ENV, DEFAULT_UPSTREAM_TIMEOUT))
+    except (TypeError, ValueError):
+        return DEFAULT_UPSTREAM_TIMEOUT
+    # 0 or negative would mean "no timeout" to httpx, which is the bug.
+    return v if v > 0 else DEFAULT_UPSTREAM_TIMEOUT
+
+
 def get_provider_config(provider: str) -> ProviderConfig:
     """Get provider config, with fallback for unknown providers."""
     if provider in PROVIDER_REGISTRY:
@@ -925,6 +959,9 @@ class LiteLLMBackend(Backend):
             logger.debug(f"LiteLLM request: model={litellm_model}")
 
             # Make the call
+            # Bounded, always: an upstream that never answers must not
+            # block the caller forever. setdefault so an explicit value wins.
+            kwargs.setdefault("timeout", _upstream_timeout())
             response = await acompletion(**kwargs)
 
             # Convert to Anthropic format
@@ -1055,6 +1092,9 @@ class LiteLLMBackend(Backend):
             kwargs["stream_options"] = {"include_usage": True}
 
             # Stream content — blocks emitted dynamically based on response
+            # Bounded, always: an upstream that never answers must not
+            # block the caller forever. setdefault so an explicit value wins.
+            kwargs.setdefault("timeout", _upstream_timeout())
             response = await acompletion(**kwargs)
             output_tokens = 0
             current_block_index = -1
@@ -1283,6 +1323,9 @@ class LiteLLMBackend(Backend):
             logger.debug(f"LiteLLM OpenAI request: model={litellm_model}")
 
             # Make the call
+            # Bounded, always: an upstream that never answers must not
+            # block the caller forever. setdefault so an explicit value wins.
+            kwargs.setdefault("timeout", _upstream_timeout())
             response = await acompletion(**kwargs)
 
             # Build the usage block. LiteLLM normalizes prompt-cache stats from
@@ -1452,6 +1495,9 @@ class LiteLLMBackend(Backend):
                 elif headers.get("x-api-key"):
                     kwargs["api_key"] = headers["x-api-key"]
 
+            # Bounded, always: an upstream that never answers must not
+            # block the caller forever. setdefault so an explicit value wins.
+            kwargs.setdefault("timeout", _upstream_timeout())
             response = await acompletion(**kwargs)
 
             async for chunk in response:

headroom/proxy/handlers/anthropic.py

@@ -2562,8 +2562,14 @@ class AnthropicHandlerMixin:
             # turn-hook fold (optimized_messages is post-hook). Runs unconditionally.
             try:
                 _orig_snapshot = original_client_messages  # noqa: F821 (bound at request start)
-                original_tokens = tokenizer.count_messages(_orig_snapshot)
+                # Off the event loop (#2810): both passes are CPU-bound real BPE
-                optimized_tokens = tokenizer.count_messages(optimized_messages)
+                # since #2543, and running them inline stalled every other
+                # in-flight request on the same process (~1s on a 2.3 MB body).
+                # Same tokenizer instance, so the reported values are unchanged.
+                original_tokens = await asyncio.to_thread(tokenizer.count_messages, _orig_snapshot)
+                optimized_tokens = await asyncio.to_thread(
+                    tokenizer.count_messages, optimized_messages
+                )
                 # Fold the tool-schema/desc compaction delta into BOTH endpoints so
                 # tok_before - tok_after == tok_saved stays coherent in the PERF line
                 # (count_messages never sees tool bytes). Same shape as the OpenAI chat
@@ -2657,7 +2663,11 @@ class AnthropicHandlerMixin:
                     parsed_original = json.loads(original_body_bytes)
                     if parsed_original != body:
                         body_mutation_tracker.mark_mutated("structural_diff_vs_original")
-                except (json.JSONDecodeError, ValueError):
+                # MemoryError is not a ValueError, so a re-parse spike on 1M-context
+                # bodies used to escape and abort an otherwise-fine request (#2768).
+                # This block is a safety net; marking mutated is already the safe
+                # outcome (it forces canonical re-serialization).
+                except (json.JSONDecodeError, ValueError, MemoryError, RecursionError):
                     body_mutation_tracker.mark_mutated("original_unparseable")
 
             if (

headroom/proxy/handlers/openai.py

@@ -3721,8 +3721,11 @@ class OpenAIHandlerMixin:
         # (result.tokens_before), which mismatches optimized_tokens (provider tokenizer)
         # and yields impossible tok_after>tok_before. Recount original from the
         # pre-compression snapshot so the message delta is on one scale.
-        original_tokens = tokenizer.count_messages(original_client_messages)
+        # Off the event loop (#2810): see the matching note in the Anthropic handler.
-        optimized_tokens = tokenizer.count_messages(body["messages"])
+        original_tokens = await asyncio.to_thread(
+            tokenizer.count_messages, original_client_messages
+        )
+        optimized_tokens = await asyncio.to_thread(tokenizer.count_messages, body["messages"])
         if tool_tokens_before_compaction > 0:
             try:
                 tool_tokens_after_compaction = tokenizer.count_text(_json_debug_dumps(tools))

headroom/proxy/prometheus_metrics.py

@@ -417,14 +417,14 @@ class PrometheusMetrics:
             return total_input_tokens, total_input_cost_usd
 
         try:
-            cost_stats = self.cost_tracker.stats()
+            # totals() rather than stats(): identical numbers, without the
+            # 31-day cost-record walk that stats()["budget_basis"] performs and
+            # this caller throws away. See CostTracker.totals.
+            tracked_input_tokens, tracked_input_cost_usd = self.cost_tracker.totals()
         except Exception:
             logger.debug("Failed to read cost tracker totals for savings history", exc_info=True)
             return total_input_tokens, total_input_cost_usd
 
-        tracked_input_tokens = cost_stats.get("total_input_tokens")
-        tracked_input_cost_usd = cost_stats.get("total_input_cost_usd")
-
         if tracked_input_tokens is not None:
             try:
                 total_input_tokens = self._savings_tracker_input_tokens_offset + max(

run-all-plugins.sh

@@ -0,0 +1,171 @@
+#!/usr/bin/env bash
+# ─────────────────────────────────────────────────────────────────────────────
+# run-all-plugins.sh — install + configure + run the Headroom proxy with ALL 5
+# enterprise plugins, the coding savings-profile, and ML compression offloaded
+# to the Kompress-v2 Modal endpoint. Then confirm everything loaded.
+#
+#   Plugins : lossless_guard, skill_search, observability, tier_router, tool_search
+#   Extra   : headroom-ai[sandbox]  (torch-free proxy; ML offloaded to Modal)
+#   Profile : coding  (HEADROOM_SAVINGS_PROFILE) + cache mode (prefix-cache safe)
+#
+# Secrets are SOURCED from ~/env.txt and ~/.headroom/plugins.env — never inlined.
+# Re-runnable: install is skipped when already satisfied (FORCE_INSTALL=1 forces).
+# ─────────────────────────────────────────────────────────────────────────────
+set -euo pipefail
+
+HR=/Users/tcms/demo/headroom
+VENV="$HR/.venv"
+PORT="${HEADROOM_PORT:-8787}"
+ENV_TXT="${ENV_TXT:-$HOME/env.txt}"
+PLUGINS_ENV="$HOME/.headroom/plugins.env"
+LOG="${HEADROOM_LOG:-$HOME/.headroom/logs/proxy-all-plugins.log}"
+mkdir -p "$(dirname "$LOG")"
+
+# ── 1. venv ──────────────────────────────────────────────────────────────────
+# `python`/`pip`/`uv` are broken system-wide on this box — always use the venv,
+# and `python -m pip` (the .venv/bin/pip shim is broken too).
+# shellcheck disable=SC1091
+source "$VENV/bin/activate"
+PY="$VENV/bin/python"
+
+# ── 2. install (guarded) ──────────────────────────────────────────────────────
+# headroom-ai[sandbox] pulls proxy,code,relevance,reports,otel,html,mcp,spreadsheet
+# (all torch-free — heavy ML is offloaded to the Modal Kompress endpoint below).
+# The 5 plugins install --no-deps so pip won't drag PyPI's headroom-ai over the
+# local editable one; headroom-license is their shared Ed25519 verifier.
+need_install=1
+if [ "${FORCE_INSTALL:-0}" != "1" ]; then
+  n=$("$PY" -c 'import opentelemetry; from headroom.proxy.extensions import discover; print(len(list(discover())))' 2>/dev/null || echo 0)
+  [ "$n" = "5" ] && need_install=0
+fi
+if [ "$need_install" = "1" ]; then
+  avail=$(df -g "$HR" 2>/dev/null | awk 'NR==2{print $4}')
+  echo "▶ disk: ${avail:-?}Gi free before install"
+  if [ -n "$avail" ] && [ "$avail" -lt 2 ]; then
+    echo "!! <2Gi free — aborting before heavy install (free space, then re-run)"; exit 1
+  fi
+  echo "▶ installing pip+maturin, then headroom-ai[sandbox] + license + 5 plugins (editable)…"
+  "$PY" -m pip install -U pip maturin
+  # litellm >=1.92 ships an sdist-only Rust bridge whose AWS-SDK crates need rustc>=1.94.1;
+  # the default rustup toolchain here is older (pip builds litellm in a temp dir that misses
+  # the repo's 1.95 pin), so pin to the last pure-Python wheel line (1.91.4). Satisfies
+  # headroom's litellm>=1.86.2,<2.0 and skips the Rust build entirely.
+  "$PY" -m pip install "litellm<1.92"
+  "$PY" -m pip install -e "${HR}[sandbox]" "litellm<1.92"
+  "$PY" -m pip install -e /Users/tcms/demo/headroom-license
+  for p in lossless-guard skill-search observability tier-router tool-search; do
+    "$PY" -m pip install -e "/Users/tcms/demo/headroom-${p}" --no-deps
+  done
+else
+  echo "▶ install satisfied (5 extensions discovered) — skipping (FORCE_INSTALL=1 to force)"
+fi
+
+# ── 3. secrets from ~/env.txt ─────────────────────────────────────────────────
+# Provides: OPENAI_API_KEY, ANTHROPIC_API_KEY, FIREWORKS_API_KEY (upstream creds);
+#           LANGFUSE_{PUBLIC,SECRET}_KEY + LANGFUSE_BASE_URL (observability sink);
+#           HEADROOM_KOMPRESS_ENDPOINT + _TOKEN (Modal ML offload).
+[ -f "$ENV_TXT" ] || { echo "!! $ENV_TXT not found"; exit 1; }
+set -a; # shellcheck disable=SC1090
+source "$ENV_TXT"; set +a
+
+# ── 4. plugin license (Ed25519, offline, wildcard) ────────────────────────────
+# HEADROOM_LICENSE + HEADROOM_LICENSE_PUBKEY. This is SEPARATE from the OSS cloud
+# key (HEADROOM_LICENSE_KEY) — the banner will still say "OSS (no license key)",
+# but each plugin prints "license accepted". Fallback: skip verification entirely.
+if [ -f "$PLUGINS_ENV" ]; then
+  set -a; # shellcheck disable=SC1090
+  source "$PLUGINS_ENV"; set +a
+else
+  echo "▶ $PLUGINS_ENV missing — using dev license bypass"
+  export HEADROOM_LICENSE_DEV=1
+fi
+
+# ── 5. Kompress ML offload → Modal ────────────────────────────────────────────
+# Setting HEADROOM_KOMPRESS_ENDPOINT (+_TOKEN) alone routes Kompress inference to
+# the Modal endpoint (content_router._get_kompress_remote). No other flag needed;
+# HEADROOM_COMPRESS_ALLOW_REMOTE is a different thing (remote upstreams, not this).
+: "${HEADROOM_KOMPRESS_ENDPOINT:?must be set in $ENV_TXT}"
+export HEADROOM_KOMPRESS_ENDPOINT_TOKEN="${HEADROOM_KOMPRESS_ENDPOINT_TOKEN:-}"
+
+# ── 6. observability sink → Langfuse + spend attribution ──────────────────────
+# HEADROOM_LANGFUSE_ENABLED must be explicitly truthy (LANGFUSE_* creds come from
+# env.txt). Traces (agent.turn / llm.turn spans with gen_ai.usage.cost) land in
+# Langfuse. Spend is opt-in: HEADROOM_MODEL_PRICES is {model-substr:{in,out}} in
+# USD per 1K tokens. (Per-request identity — org/team/user/session — is supplied
+# by the CLIENT via x-headroom-* headers, not settable here.)
+export HEADROOM_LANGFUSE_ENABLED=1
+export HEADROOM_LANGFUSE_SERVICE_NAME=headroom-proxy
+export HEADROOM_MODEL_PRICES='{"claude-opus":{"in":0.015,"out":0.075},"claude-sonnet":{"in":0.003,"out":0.015},"gpt-5":{"in":0.00125,"out":0.01},"gpt-4":{"in":0.003,"out":0.012}}'
+# Metrics (counters) need a separate OTLP endpoint — none in env.txt, so left off:
+# export HEADROOM_OTEL_METRICS_ENABLED=1 HEADROOM_OTEL_METRICS_ENDPOINT=http://localhost:4318
+
+# ── 7. tier_router ─────────────────────────────────────────────────────────────
+# Only stamps service_tier on the wire (no token delta). OpenAI 'flex' is only
+# auto-selected for models declared eligible here. Anthropic tiers are a no-op by
+# default. Clients force a tier with x-headroom-tier / x-headroom-background: 1.
+export HEADROOM_TIER_FLEX_MODELS="${HEADROOM_TIER_FLEX_MODELS:-gpt-5,gpt-4.1,o4-mini}"
+
+# ── 8. plugin tuning (defaults shown; override as needed) ─────────────────────
+# skill_search fires on Anthropic w/ >=min skills; tool_search on synthetic-tier
+# providers w/ >=min tools; lossless_guard lossy tier is opt-in (kept OFF).
+export HEADROOM_SKILL_SEARCH_MIN_SKILLS="${HEADROOM_SKILL_SEARCH_MIN_SKILLS:-8}"
+export HEADROOM_TOOL_SEARCH_MIN_TOOLS="${HEADROOM_TOOL_SEARCH_MIN_TOOLS:-5}"
+# export HEADROOM_LOSSLESS_GUARD_LOSSY=1   # opt-in irreversible Bash-noise drop
+
+# ── 9. coding profile + mode ───────────────────────────────────────────────────
+# savings_profile=coding tunes the pipeline for coding-agent traffic; cache mode
+# freezes prior turns to preserve the provider prefix-cache (what coding wants).
+export HEADROOM_SAVINGS_PROFILE=coding
+
+# ── 10. run + confirm ──────────────────────────────────────────────────────────
+cleanup() { [ -n "${PROXY_PID:-}" ] && kill "$PROXY_PID" 2>/dev/null || true; }
+trap cleanup INT TERM EXIT
+
+echo "▶ starting proxy on :$PORT  (profile=coding, mode=cache, all 5 extensions)…"
+headroom proxy --port "$PORT" --mode cache --proxy-extension '*' > "$LOG" 2>&1 &
+PROXY_PID=$!
+
+# wait for readiness (no foreground sleep on this harness)
+curl -s --retry 40 --retry-delay 1 --retry-all-errors --max-time 60 \
+  "http://127.0.0.1:$PORT/health" >/dev/null 2>&1 || true
+
+echo
+echo "══════════════════ CONFIRMATION ══════════════════"
+echo "── extensions loaded (from $LOG) ──"
+grep -iE "Extensions:|license accepted|installed \(" "$LOG" | sed 's/^/  /' || true
+
+echo "── Modal Kompress endpoint reachable? ──"
+code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 30 "$HEADROOM_KOMPRESS_ENDPOINT" || echo "unreachable")
+echo "  $HEADROOM_KOMPRESS_ENDPOINT -> HTTP $code (any response = up; offload runs on real traffic)"
+
+echo "── /stats surfaces (empty until traffic flows) ──"
+curl -s --max-time 5 "http://127.0.0.1:$PORT/stats" | "$PY" -c '
+import sys,json
+d=json.load(sys.stdin)
+print("  extension_savings   :", d.get("extension_savings"))
+print("  by_layer            :", list(d.get("savings",{}).get("by_layer",{})))
+print("  tokens_saved_by_strat:", d.get("tokens_saved_by_strategy"))
+print("  otel.enabled        :", d.get("otel",{}).get("enabled"))
+print("  langfuse.enabled    :", d.get("langfuse",{}).get("enabled"))
+' 2>/dev/null || echo "  (stats not ready)"
+
+cat <<EOF
+
+── where each effect shows up ──
+  lossless_guard  -> dashboard (compression layer)  + /stats.tokens_saved_by_strategy
+  skill_search    -> /stats.extension_savings  (NOT dashboard) — Anthropic client, >=8 skills
+  tool_search     -> /stats.extension_savings  (NOT dashboard) — synthetic-tier client, >=5 tools
+  observability   -> Langfuse UI (spans + gen_ai.usage.cost)   — send x-headroom-org/user/session
+  tier_router     -> service_tier on the wire / provider bill  (no token delta)
+
+── drive traffic (two clients — they exercise different plugins) ──
+  Claude Code : ANTHROPIC_BASE_URL=http://localhost:$PORT claude        # lossless_guard + skill_search
+  OpenAI/opencode: OPENAI_BASE_URL=http://localhost:$PORT/v1 <client>    # tool_search
+  Dashboard   : headroom dashboard   (http://127.0.0.1:$PORT/dashboard)
+  Raw stats   : curl -s localhost:$PORT/stats | python3 -m json.tool
+
+Proxy is running (pid $PROXY_PID). Ctrl-C to stop. Logs: $LOG
+═══════════════════════════════════════════════════
+EOF
+
+wait "$PROXY_PID" || true

tests/test_litellm_upstream_timeout.py

@@ -0,0 +1,70 @@
+"""Every upstream call must be bounded.
+
+There was no timeout in this backend at all. Observed 2026-08-07 under load:
+four agent workers blocked on ESTABLISHED connections for 36+ minutes while
+the proxy answered /readyz in 0.11s. No error, no retry, no log line -- the
+caller simply stops, forever, and that is indistinguishable from slow work.
+"""
+
+from __future__ import annotations
+
+import ast
+from pathlib import Path
+
+import pytest
+
+from headroom.backends.litellm import (
+    DEFAULT_UPSTREAM_TIMEOUT,
+    UPSTREAM_TIMEOUT_ENV,
+    _upstream_timeout,
+)
+
+_SRC = Path(__file__).resolve().parents[1] / "headroom" / "backends" / "litellm.py"
+
+
+def test_every_acompletion_call_is_bounded():
+    """A new dispatch path added without a timeout reintroduces the hang.
+
+    Checked structurally rather than by mocking, because the failure mode is a
+    call site someone ADDS later -- which no mock of the existing paths sees.
+    """
+    tree = ast.parse(_SRC.read_text())
+    calls, guards = 0, 0
+    for node in ast.walk(tree):
+        if not isinstance(node, ast.Call):
+            continue
+        fn = node.func
+        if isinstance(fn, ast.Name) and fn.id == "acompletion":
+            calls += 1
+        if (
+            isinstance(fn, ast.Attribute)
+            and fn.attr == "setdefault"
+            and node.args
+            and isinstance(node.args[0], ast.Constant)
+            and node.args[0].value == "timeout"
+        ):
+            guards += 1
+    assert calls > 0, "no acompletion call sites found -- test is stale"
+    assert guards >= calls, (
+        f"{calls} acompletion call site(s) but only {guards} timeout guard(s); "
+        "an unbounded upstream call blocks its caller forever"
+    )
+
+
+def test_a_junk_env_value_cannot_disable_the_timeout(monkeypatch):
+    """`0` means 'no timeout' to httpx, i.e. exactly the bug. So does junk."""
+    for bad in ("", "0", "-1", "nonsense", "None"):
+        monkeypatch.setenv(UPSTREAM_TIMEOUT_ENV, bad)
+        assert _upstream_timeout() == DEFAULT_UPSTREAM_TIMEOUT, bad
+
+
+def test_an_operator_can_still_tune_it(monkeypatch):
+    monkeypatch.setenv(UPSTREAM_TIMEOUT_ENV, "42.5")
+    assert _upstream_timeout() == pytest.approx(42.5)
+
+
+def test_the_default_is_generous_enough_for_real_work():
+    """Streaming: litellm expands a float across all httpx phases, so this is
+    the max gap BETWEEN CHUNKS, not a cap on total generation. A steady long
+    answer is never cut off."""
+    assert 60.0 <= DEFAULT_UPSTREAM_TIMEOUT <= 1800.0

tests/test_proxy/test_anthropic_recount_and_reparse_safety.py

@@ -0,0 +1,158 @@
+"""Two request-path safety nets in ``handle_anthropic_messages``.
+
+1. #2810 — the consistency re-count runs ``count_messages`` twice. Both passes
+   are CPU-bound real BPE (since #2543) and used to run directly on the event
+   loop, stalling every other in-flight request on the process (~1s on a 2.3 MB
+   body). They must run off the loop.
+2. #2768 — the byte-faithful forwarder's verification re-parse of the original
+   body is best-effort, but ``MemoryError`` is not a ``ValueError``, so on
+   1M-context payloads it escaped and aborted an otherwise-fine request. The
+   block must never be able to fail the request.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+fastapi = pytest.importorskip("fastapi")
+
+from fastapi.testclient import TestClient  # noqa: E402
+
+from headroom.proxy.server import ProxyConfig, create_app  # noqa: E402
+
+MESSAGES = "/v1/messages"
+MODEL = "claude-sonnet-4-6"
+
+# Only ever present in the PRE-compression snapshot, never in the outbound body.
+# Long enough to clear the handler's min-token floors.
+SENTINEL = "presnapshot-sentinel " * 500
+
+
+def _config(**overrides) -> ProxyConfig:
+    base = {
+        "optimize": True,
+        "cache_enabled": False,
+        "rate_limit_enabled": False,
+        "cost_tracking_enabled": False,
+        "mode": "token",
+    }
+    base.update(overrides)
+    return ProxyConfig(**base)
+
+
+def _upstream_200() -> MagicMock:
+    payload = {
+        "id": "msg_1",
+        "type": "message",
+        "role": "assistant",
+        "content": [{"type": "text", "text": "ok"}],
+        "model": MODEL,
+        "usage": {"input_tokens": 10, "output_tokens": 2},
+    }
+    resp = MagicMock()
+    resp.status_code = 200
+    resp.headers = {"content-type": "application/json"}
+    resp.content = json.dumps(payload).encode()
+    resp.text = json.dumps(payload)
+    resp.json.return_value = payload
+    return resp
+
+
+def test_consistency_recount_runs_off_the_event_loop(monkeypatch):
+    """No ``count_messages`` pass over the pre-compression snapshot may run on
+    the loop thread. The snapshot is identified by SENTINEL, which the pipeline
+    strips, so this pins the re-count specifically: the already-offloaded count
+    at request start also sees the sentinel and passes either way, while the
+    two re-count passes ran inline before #2810 and would fail here.
+    """
+    import headroom.tokenizers as tokenizers_mod
+
+    seen: list[bool] = []  # one entry per snapshot count: True == ran on the loop
+
+    # Patch the class, not the cached instance, so pytest restores it for us.
+    tokenizer_cls = type(tokenizers_mod.get_tokenizer(MODEL))
+    real_count = tokenizer_cls.count_messages
+
+    def counting(self, messages):  # noqa: ANN001, ANN202
+        if SENTINEL in json.dumps(messages, default=str):
+            try:
+                asyncio.get_running_loop()
+            except RuntimeError:
+                seen.append(False)  # worker thread — no running loop here
+            else:
+                seen.append(True)  # blocking the event loop
+        return real_count(self, messages)
+
+    monkeypatch.setattr(tokenizer_cls, "count_messages", counting)
+
+    def stripping_apply(**kwargs):  # noqa: ANN003, ANN202
+        """Return genuinely-changed messages with the sentinel removed."""
+        from types import SimpleNamespace
+
+        compressed = [{**m, "content": "compressed"} for m in kwargs["messages"]]
+        return SimpleNamespace(
+            messages=compressed,
+            transforms_applied=["test_strip"],
+            timing={},
+            tokens_before=100,
+            tokens_after=80,
+            waste_signals=None,
+        )
+
+    app = create_app(_config())
+    with TestClient(app) as client:
+        proxy = client.app.state.proxy
+        proxy.anthropic_pipeline.apply = MagicMock(side_effect=stripping_apply)
+        proxy._retry_request = AsyncMock(return_value=_upstream_200())
+        r = client.post(
+            MESSAGES,
+            json={
+                "model": MODEL,
+                "max_tokens": 16,
+                "messages": [{"role": "user", "content": SENTINEL}],
+            },
+        )
+
+    assert r.status_code == 200, r.text
+    assert seen, "no count_messages pass saw the snapshot; test is not exercising #2810"
+    assert not any(seen), f"{sum(seen)}/{len(seen)} snapshot counts blocked the event loop"
+
+
+def test_memoryerror_in_verification_reparse_does_not_abort_the_request(monkeypatch):
+    """A ``MemoryError`` from the best-effort original-body re-parse must be
+    swallowed (the safe fallback marks the body mutated, forcing canonical
+    re-serialization) rather than escaping and killing the request.
+    """
+    import headroom.proxy.handlers.anthropic as anthropic_mod
+
+    real_loads = json.loads
+    raised = {"n": 0}
+
+    def exploding_loads(s, *args, **kwargs):  # noqa: ANN001, ANN002, ANN003, ANN202
+        # Only the verification re-parse passes the raw original body bytes.
+        if isinstance(s, (bytes, bytearray)) and b"reparse-bomb" in s:
+            raised["n"] += 1
+            raise MemoryError("simulated re-parse spike")
+        return real_loads(s, *args, **kwargs)
+
+    monkeypatch.setattr(anthropic_mod.json, "loads", exploding_loads)
+
+    app = create_app(_config(optimize=False))
+    with TestClient(app) as client:
+        proxy = client.app.state.proxy
+        proxy._retry_request = AsyncMock(return_value=_upstream_200())
+        r = client.post(
+            MESSAGES,
+            json={
+                "model": MODEL,
+                "max_tokens": 16,
+                "messages": [{"role": "user", "content": "reparse-bomb"}],
+            },
+        )
+
+    assert raised["n"] > 0, "the verification re-parse never ran; test is not exercising #2768"
+    assert r.status_code == 200, r.text