PR #484 added branch-awareness to ``scripts/sync-plugin-versions.py``
(no-op on feature branches unless ``HEADROOM_SYNC_VERSIONS=1``). The
existing ``test_main_runs_plugin_only_version_sync`` test broke
because it didn't account for the new ``_should_sync`` gate — on a
feature branch the main() function early-returns and the subprocess
mock was never invoked, but actually the test broke earlier because
``_current_branch`` calls ``subprocess.run(..., capture_output=True,
text=True, check=False)`` and the test's lambda only accepted
``(command, cwd, check)``.
Fix: force ``_should_sync`` True in the existing test so it locks
the run-path, then add 4 new tests covering the branch-aware logic
itself (env override, main vs feature, git-unavailable defensive
no-op).
Three independent contract-pattern follow-ons bundled into one PR.
Same frozen-dataclass + factory + apply_to_tags + Rust-portable
shape that PR #473 / #477 / #483 established.
## (1) MemoryRanker + RecencyBoostRanker
Pre-this-PR Headroom ranked memory candidates by pure cosine
similarity. Every other memory system we surveyed (Letta, Mem0,
Cognee, Supermemory) re-ranks beyond cosine.
* ``MemoryRanker`` Protocol — pluggable re-ranker; future PRs add
source-weight + access-count rankers behind the same interface.
* ``RecencyBoostRanker`` — first concrete impl. Final score is
``cosine × exp(-age_days / decay_days)``. Default decay 30 days
(half-life ~21 days; 60-day-old factor 0.135, 90-day-old 0.050).
* ``MemoryCandidate`` — backend-agnostic frozen value type that
flows through the ranker. ``MemoryCandidate.from_backend_result``
adapter converts the existing ``MemoryResult`` shape (with nested
``memory.created_at``) into the ranker's flatter form.
* Wired into ``memory_handler.search_and_format_context`` as an
optional ``ranker=`` kwarg — backwards-compat: ``None`` (default)
preserves the pure-cosine path identically.
Defensive:
* ``created_at=None`` → factor 1.0 (recency-neutral, back-compat with
legacy rows / migrating backends)
* Negative age (clock skew) → clamped to factor 1.0 (a future-dated
row can't outrank a real fresh memory)
* Sort is stable on ties — same input → same output every turn, so
consecutive turns inject memories in the same order (prefix-cache
friendly)
Performance: O(N) over candidates where N=top_k≈10. One ``math.exp``
per candidate. Sub-microsecond. Zero new I/O.
## (2) ImageCompressionDecision
Mirror of :class:`CompressionDecision` for image compression. Two
sites today (``openai.py:1203``, ``anthropic.py:868``) gate inline;
both already respect bypass (no Gemini-class drift bug like text
compression had), but consolidating into a value type:
* Locks bypass-respect via AST contract test — future sites can't
drift on it
* Surfaces ``image_skip_reason`` in ``RequestOutcome.tags`` for
dashboard slicing (same observability surface as
``passthrough_reason`` and ``memory_skip_reason``)
* Same Rust-port shape as the other decision types
Precedence: ``bypass_header`` > ``image_optimize_disabled`` >
``no_messages`` > ``should_compress=True``.
Anthropic's extra ``is_cache_mode`` check stays inline because it's
Anthropic-specific (openai/gemini don't have it). Documented in a
code comment.
## (3) Branch-aware sync-plugin-versions hook
Pre-this-fix the pre-commit ``sync-plugin-versions`` hook ran on
every commit and bumped manifests to the predicted-next-release
version. Every PR ended up carrying the prediction as collateral
("Why are we bumping ``.claude-plugin/marketplace.json`` — we
should not, right??" - user, on PR #483).
Fix: the hook is now a NO-OP unless EITHER:
* We're on the ``main`` branch, OR
* ``HEADROOM_SYNC_VERSIONS=1`` is set explicitly (release workflow)
On feature branches the hook prints a single line explaining the
skip and exits cleanly. The release workflow opts in via the env
var; behaviour on main / at release time is unchanged.
## Test coverage
* 16 new tests on ``MemoryRanker`` / ``RecencyBoostRanker``
(frozen, equal cosine wins by recency, decay configurable, NULL
timestamp neutral, no-mutation contract, Rust-port shape)
* 17 new tests on ``ImageCompressionDecision`` (frozen, all 3
skip reasons, precedence, observability fields, apply_to_tags)
* 1 new AST invariant test (extends
``test_handler_outcome_tag_invariant.py``) — locks "no raw
``if self.config.image_optimize and messages and not _bypass:``
conjunction in any handler"
All existing memory + cache-stability + handler tests pass (203 ✓).
``make ci-precheck`` clean.
## Rust portability
All three new value types port cleanly to frozen Rust structs +
pure functions. Same migration pattern as ``CompressionDecision``
(already locked in for the SmartCrusher Rust port).
## Zero-regression contract
* Default ``ranker=None`` → memory_handler behaves identically to
pre-this-PR (pure cosine; no perf change)
* Image decision migration is identity at the bypass/optimize/messages
gate — no behaviour change, just contract consolidation
* Hook fix is no-op on feature branches (less churn) and unchanged
on main (release flow preserved)
PR-this removed the 500-char query truncation in
``memory_handler._extract_user_query``. The pre-existing assertion at
``test_memory_handler_native_ops.py:845`` was testing the old buggy
behaviour (output truncated at 500) and CI caught it. Updated to
assert the full-fidelity return.
Three bug classes fixed plus three architectural extension points,
together making the memory subsystem uniform across all five sites
and ready for future Mem0/Letta/Cognee backend integration.
## Bug fixes
* **3 sites silently ignored `x-headroom-bypass: true`** —
``anthropic.py:1303``, ``openai.py:1620`` (chat), ``gemini.py:382``
injected memory under bypass, mutating request bytes when the user
explicitly asked for byte-faithful passthrough. Now gated on
``MemoryDecision.decide(...)`` which honours bypass uniformly.
* **500-char query truncation** — ``memory_handler._extract_user_query``
capped at 500 chars, silently throwing away signal. None of Letta /
Mem0 / Cognee / Supermemory truncate. Removed; the embedding model
handles its own window.
* **Gemini had no timeout** on ``search_and_format_context`` — the
only chat handler without one. A slow backend could stall requests.
Added ``asyncio.wait_for`` matching Anthropic + OpenAI Chat +
Responses.
* **WS injected into ``body["instructions"]``** — the system /
cache-hot-zone field, violating invariant I2 (all other handlers
inject at user-message tail). Switched to ``ws_response_body["input"]``
for string-shaped input; list-shaped input deferred to the Rust
handler with a clear log.
## New value types (extension points)
* ``MemoryDecision`` — frozen dataclass + factory. Five-way skip
reason enum (``bypass_header`` / ``no_handler`` / ``no_user_id`` /
``mode_disabled`` / ``mode_tool``). ``apply_to_tags()`` surfaces
the skip reason in ``RequestOutcome.tags["memory_skip_reason"]``
— dashboards can now slice memory-blind traffic by cause.
* ``MemoryQuery`` — multi-source retrieval query. ``from_messages()``
walks the conversation and extracts latest user text + recent tool
outputs + recent assistant turns at FULL fidelity (no truncation).
Handles both OpenAI-shape ``role: tool`` and Anthropic-shape
``tool_result`` content blocks. ``to_embedding_input()`` produces
a delimited concatenation the embedder sees as structured context.
* ``MemoryInjectionBudget`` — uniform token / entry / similarity
bound on the formatted injection block. Pre-this-PR no cap (~4000
tokens could land per request). Default 1024 tokens / 10 entries /
0.3 similarity floor. ``apply_to_text()`` truncates at line
boundaries so dashboard renders intact bullet points.
## Migration scope — all 5 sites uniform at the GATE level
| Site | Handler | Pre-PR gate | Post-PR gate |
|---|---|---|---|
| 1 | anthropic.py | `memory_handler and memory_user_id` | `memory_decision.inject` |
| 2 | gemini.py | `memory_handler and memory_user_id` | `memory_decision.inject` |
| 3 | openai.py chat | `memory_handler and memory_user_id` | `memory_decision.inject` |
| 4 | openai.py Responses | `memory_handler and memory_user_id and not _bypass` | `responses_memory_decision.inject` |
| 6 | openai.py WS | `memory_handler and body and not _ws_bypass` | `ws_memory_decision.inject` |
Site 5 (Responses bypass-elif log-only branch) is preserved verbatim.
## Deliberately deferred (separate PRs)
* **Memory injection order inversion** — sites 4 and 6 inject
BEFORE compression; sites 1/2/3 inject AFTER. Moving 4 + 6 to
post-compression needs its own focused cache-stability testing.
* **Importance scoring** — recency × source × access-count.
* **Per-memory atomize-and-split** — Mem0/Supermemory pattern.
* **AST-aware code chunking for tool outputs** — Supermemory's
code-chunk approach.
The contracts shipped here (``MemoryQuery`` + ``MemoryInjectionBudget``)
are the extension points those will plug into.
## Test coverage
* 20 new tests on ``MemoryDecision``
* 14 new tests on ``MemoryQuery`` (full-fidelity, multi-source)
* 10 new tests on ``MemoryInjectionBudget``
* 3 new AST contract tests (no raw gate; no system writes; every
search call passes ``query=``)
* All existing memory + cache-stability tests still pass (222 passed)
## Rust portability
Every new value type ports cleanly to a frozen Rust struct. Pure
functions, no I/O, no global state. Same Python ↔ Rust parity-test
pattern that ``CompressionDecision`` already uses.
## Zero-regression contract
Existing chat/completion harnesses (Claude Code, Codex, Cursor,
Continue, Aider) see ZERO wire-byte changes when bypass is NOT set.
When bypass IS set, the 3 chat handlers now correctly skip memory
injection — that's the bug fix, not a regression.
ContentRouterConfig.exclude_tools already existed as the intended override
hook for the never-compress tool list, but nothing ever set it — it stayed
None, so the consumer always fell back to the hardcoded DEFAULT_EXCLUDE_TOOLS.
There was no way to extend the exclusion list without editing source.
- Add ProxyConfig.exclude_tools field.
- Add _parse_exclude_tools() reading the comma-separated --exclude-tools CLI
flag and the HEADROOM_EXCLUDE_TOOLS env var (mirrors how --tool-profile
pairs with HEADROOM_TOOL_PROFILES).
- Merge the parsed set with DEFAULT_EXCLUDE_TOOLS into router_config
(a non-None exclude_tools replaces the default set, so merge not assign).
- Names added in original and lowercase form, mirroring DEFAULT_EXCLUDE_TOOLS.
Unset leaves behavior unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three fixes bundled; all in admin / cache-hit paths where tests didn't
catch the regression.
## (A) 13 RequestOutcome sites missing tags=
An AST audit found that 13 of 21 ``RequestOutcome(...)`` construction
sites across the four handler files emitted outcomes without threading
``tags=``. Affected paths:
* ``handle_anthropic_messages`` — the ``from_response_cache=True``
early-return outcome (Claude Code cache-hit turns dashboard-blind)
* ``handle_openai_chat`` — same cache-hit early-return (Codex +
Cursor + Continue cache-hit turns dashboard-blind)
* ``handle_openai_responses_ws`` — the per-turn outcome inside the
Codex WS session. The stale comment that said "ws_session_tags is
not yet bound" was wrong — ``ws_tags`` was already extracted at
handler entry
* ``handle_anthropic_batch_create / batch_passthrough / batch_results``
* ``handle_passthrough`` (OpenAI Models / Files / List-Batches)
* ``handle_google_batch_create / batch_passthrough / batch_results``
* ``_google_batch_passthrough`` (internal helper)
* ``handle_batch_create`` (OpenAI batch entry)
* ``handle_gemini_count_tokens`` (also fixed in #479; identical)
Pattern of the fix is uniform: pull tags from headers and thread
them into the ``RequestOutcome`` construction.
New contract test ``test_handler_outcome_tag_invariant.py`` walks each
handler file's AST and asserts every ``RequestOutcome`` site inside any
``handle_*`` or ``*_passthrough`` method passes both ``tags=`` and
``client=``. Future handlers get a clear test failure with file +
line + method name if they regress.
## (B) Issue #478 — /v1/models 403 under Codex ChatGPT auth
Codex Desktop with ChatGPT-subscription OAuth polls ``/v1/models`` to
populate its model picker. Forwarding to ``chatgpt.com/backend-api/
models`` returned 403 to OAuth tokens. Fix: synthesize an OpenAI-
compatible payload locally from a known-supported model set
(``gpt-5.5`` through ``gpt-5``). All other ChatGPT-auth paths still
forward as before — only model-metadata gets the local response.
## (C) Move _extract_tags to free function (mixin-isolation test compat)
Handlers called ``self._extract_tags(headers)``. That worked in
production where ``HeadroomProxy`` composes every mixin and defines
the method, but broke tests that instantiate a single mixin via
``object.__new__(OpenAIHandlerMixin)``. The free-function form
removes that coupling — handlers import ``extract_tags`` from
``headroom.proxy.helpers`` and call directly. ``HeadroomProxy.
_extract_tags`` is kept as a thin wrapper for any external caller
still using the method form. 17 call sites migrated.
## Zero behavior change for existing users
Claude Code, Codex, Cursor, Continue, Aider, Gemini-routed harnesses
all hit handlers that already extracted tags. Their wire bytes to
upstream LLMs are byte-identical. Only the dashboard view gains tags
on previously-blind paths.
Closes#478.
Adds ``CompressionDecision.apply_to_tags(tags)`` — a one-liner mutator
that stamps the passthrough reason into a tags dict for downstream
observability. Each migrated handler now calls
``_decision.apply_to_tags(tags)`` immediately after
``CompressionDecision.decide(...)``. The tags dict flows unchanged
into every downstream ``RequestOutcome(tags=tags, ...)`` construction,
which the funnel surfaces in ``RequestLog.tags`` — same mechanism the
funnel already uses for ``client``.
Dashboards can now slice passthrough traffic by cause:
* tags["passthrough_reason"] == "bypass_header"
* tags["passthrough_reason"] == "compression_disabled"
* tags["passthrough_reason"] == "no_messages"
* tags["passthrough_reason"] == "license_denied"
No-op when ``should_compress=True`` — compressing requests don't
carry the tag, so absence vs presence is itself the signal.
Bonus fix: ``handle_gemini_count_tokens`` was the one Gemini handler
that never pulled tags out of headers, so its emitted
``RequestOutcome`` reached the dashboard without any of the per-
request slicing keys. Added the missing ``tags = self._extract_tags
(request.headers)`` and threaded ``tags=tags`` into its outcome.
Closes the observability loop opened by PR #477: the four Gemini-
bypass-bug fixes are now visible in the request-log feed the moment
they fire.
Pre-this-PR, four handler files computed "should this request be
compressed?" inline at five sites with subtle drift. Three Gemini
sites silently ignored ``x-headroom-bypass: true``; one of those
also ignored the license gate. Anthropic and OpenAI got the full
conjunction right, but encoded it inline.
``CompressionDecision.decide(headers, config, usage_reporter,
messages)`` is the single canonical factory. Precedence:
1. ``bypass_header`` — user's explicit opt-out (highest)
2. ``compression_disabled`` — operator config.optimize=False
3. ``no_messages`` — nothing to compress
4. ``license_denied`` — commercial gate
The factory exposes every constituent boolean
(``bypass_header_set``, ``config_optimize_enabled``,
``license_allows``, ``has_messages``) so debug tooling can answer
"what did the decision see?" without re-running it.
Each migrated site now logs ``Compression skipped: reason=<X>`` on
passthrough — new structured observability.
Bug fixes that came with the consolidation:
* ``handlers/gemini.py:handle_gemini_generate_content`` — now
respects ``x-headroom-bypass``
* ``handlers/gemini.py:handle_google_cloudcode_stream`` — now
respects ``x-headroom-bypass``
* ``handlers/gemini.py:handle_gemini_count_tokens`` — now respects
``x-headroom-bypass`` AND the license gate (was missing both)
Three streaming finalizers — ``_finalize_stream_response``,
``_stream_response_bedrock``, ``_stream_openai_via_backend`` — each
duplicated the same set of body- and config-derived fields when
constructing a ``RequestOutcome``:
* ``attempted_input_tokens = optimized_tokens + tokens_saved``
* ``num_messages = len(body.get("messages", []))``
* ``request_messages`` conditional on ``config.log_full_messages``
* ``transforms_applied`` list → tuple (frozen-dataclass contract)
* ``tags or {}`` normalization
* ``turn_id`` via ``compute_turn_id``
The last one was a real bug. Only the Bedrock site computed
``turn_id`` — sites 1 and 3 silently dropped it, breaking the
dashboard's multi-turn-session grouping for every Anthropic-SSE and
OpenAI-via-backend request. The new ``RequestOutcome.from_stream``
classmethod computes it uniformly so the three finalizers cannot
drift apart on derivation logic again.
Each call site now hands ``from_stream`` the body + provider-specific
cache/timing fields and gets a fully-constructed outcome back. The
funnel call after it stays identical (``await
self._record_request_outcome(outcome)``).
The wrap-e2e harness passed `--startup-timeout-ms 5000` to `headroom
wrap openclaw`, leaving zero slack for the openclaw plugin's auto-start
launcher to bring up the headroom proxy before the 5s health-check
deadline. On a busy CI runner, cold Python import of `headroom.cli` plus
pyo3 dlopen plus FastAPI app boot routinely lands in the 4–8s range, so
this was always a coin-flip.
Evidence: run 25897154424 failed on main with the exact code that
passed pre-merge on PR #474's docker-wrap-e2e check (run 25897085244).
Both logs show identical openclaw "Config warnings" output — that's
normal noise, not the cause. The differentiating line is
`[plugins] Headroom proxy started and reachable` (pass) vs
`[plugins] Headroom proxy unavailable: health check failed` (fail).
30s matches what other wrap-e2e callers already use as a working margin
for the headroom proxy boot path; the runtime default for `headroom
wrap openclaw --startup-timeout-ms` is 20s.
Cashes in the RequestOutcome refactor with a typed-field surface that
gives EVERY handler per-harness visibility — Codex / Claude Code /
aider / Cursor / Zed / opencode / DROID / antigravity / etc. — for one
field-add across the contract.
The one-field-add proof
Headroom went from "what fraction of OUR requests come from which
harness?" being unanswerable (handlers logged ad-hoc User-Agent strings
in heterogeneous tag dicts at 18 sites, with 9 of 18 not even
populating them) to one structured ``client: str | None`` value on
every observation flowing through the funnel. No new bookkeeping at
call sites; every handler picks it up via a single
``classify_client(headers)`` call at request entry.
Implementation
* New ``CLIENT_UA_MAP`` + ``classify_client()`` in
``headroom/proxy/auth_mode.py``. Substring match against
User-Agent; ``X-Client`` header overrides UA. Returns ``str | None``
so ``None`` is the loud "unidentified" signal rather than a silent
empty bucket.
* New ``RequestOutcome.client: str | None = None`` field.
* Funnel updates (in ``outcome.py``):
- Appends ``client=X`` to the PERF log line ONLY when set, so
``headroom perf --client X`` parsing stays clean for
unidentified traffic (no bogus ``client=`` token).
- Copies ``client`` into ``RequestLog.tags["client"]`` so the
dashboard's existing tag-based filtering surfaces per-harness
slicing with zero new columns.
* Every handler that constructs a RequestOutcome now passes
``client=client`` — wired across streaming.py (3 finalizers,
with ``_finalize_stream_response`` gaining a new optional kwarg
since it doesn't have direct access to headers), anthropic.py
(6 sites), openai.py (8 sites including Codex WS), gemini.py
(2 emitting sites), batch.py (5 sites).
Harnesses recognised
Anthropic ecosystem: claude-code, claude-cli, claude-vscode,
anthropic-cli
OpenAI ecosystem: codex-cli
Editors: cursor, zed
AI coding harnesses: aider, droid, opencode, github-copilot
Other: antigravity (Google experimental)
Adding a new client is a one-line edit to ``CLIENT_UA_MAP``.
Tests
* 8 new tests in ``test_request_outcome.py`` covering:
- ``client`` field round-trips on the value type
- ``classify_client`` against every recognised UA prefix
- ``X-Client`` header override beats UA match
- ``None`` for unknown traffic (the loud signal)
- Funnel appends ``client=X`` to PERF when set
- Funnel OMITS ``client=`` from PERF when None (no bogus empty)
- Funnel stamps ``client`` into ``RequestLog.tags``
* All 228 existing tests still pass (full sweep across streaming,
cache, Codex, Anthropic, OpenAI, Gemini, batch, auth-mode).
* ruff + ruff-format + mypy clean.
What's now true that wasn't before
Once this lands, the dashboard can answer:
* "Show me cache hit rate by harness"
→ ``GROUP BY tags.client FROM request_log``
* "Which harness contributes the most cache writes?"
→ same
* "Per-harness savings ratio"
→ same
* ``headroom perf --client codex`` / ``--client claude-code``
→ analyzer filters PERF log lines on ``client=X`` token
Zero new bookkeeping in handlers. Zero changes to Prometheus label
cardinality (kept the client dimension out of Prometheus on purpose —
the tags route is the right surface). The "what's our traffic split
by harness?" question is now answerable in three places (PERF log,
RequestLog tags, dashboard widgets that already filter on tags)
without any per-provider work.
Full regression sweep found 7 failures in test dummies (out of 4242
tests) that didn't have the production handler interface my refactor
now requires. All same root cause: the dummies need
``_record_request_outcome`` to delegate to the funnel; the
copilot-auth passthrough dummy also needs ``_next_request_id``
because the migrated passthrough handler now allocates an ID at
record-time.
Failures:
test_proxy_handlers_batch.py (6 sites — all DummyBatchHandler)
test_proxy_copilot_auth_hooks.py (1 site — Dummy in passthrough test)
Fix is the same pattern used in the earlier dummy fixes
(test_anthropic_pre_upstream_backpressure, test_openai_codex_routing,
test_openai_codex_ws_lifecycle):
async def _record_request_outcome(self, outcome):
from headroom.proxy.outcome import emit_request_outcome
await emit_request_outcome(self, outcome)
After the fix: 22 / 22 in the previously-failing tests; full
regression sweep 4242 / 4242 with zero failures (179 skipped, all
opt-in real-API).
Unrelated env issues observed in the same sweep but skipped:
* tests/test_memory/* — huggingface-hub<2.0 / transformers version
drift in local venv. Pre-existing, not caused by this refactor.
* tests/integrations/* — same env class.
* tests/test_realignment_live_multi_turn.py — opt-in live tests
needing API keys.
Completes the migration of every ``metrics.record_request`` call site
in ``headroom/proxy/handlers/`` onto the canonical funnel. After this
commit, **zero ad-hoc record_request calls remain** across the entire
handler subtree. Every request — regardless of provider, harness, or
transport — flows through ``emit_request_outcome``.
Migrated sites (this commit):
* **handle_openai_responses_ws** (Codex WS) — 2 sites:
- per-turn record (per ``response.completed``)
- session-end residual (leftover tokens not captured per-turn)
Pre-refactor these sites emitted only metrics + cost_tracker — no
RequestLog, no PERF — so Codex traffic was invisible to
``headroom perf`` and the recent-requests feed. Funnel restores all
four effects uniformly per turn. (Closes the visibility half of
what #471's sibling PR addressed for the scheduler half.)
The explicit session-summary RequestLog at session-end stays as a
separate explicit log entry — it's a session-cumulative summary,
distinct from per-turn observations.
* **handle_openai_chat** — 3 sites:
- response-cache hit (uses ``from_response_cache=True``)
- backend-routed (LiteLLM/AnyLLM) non-streaming success
- direct OpenAI non-streaming success
* **handle_openai_responses** HTTP (Codex HTTP transport) — 1 site
* **handle_passthrough** (OpenAI passthrough endpoints) — 1 site
* **batch.py** handlers — 5 sites:
- handle_google_batch_create
- handle_google_batch_passthrough (Files API forward)
- handle_google_batch_passthrough (list/get/cancel)
- handle_google_batch_results (CCR-processed)
- handle_batch_create (OpenAI batches)
All converge on the funnel. Several gain request_id allocation
they didn't have before (passthrough sites previously emitted
``request_id=None`` in logs).
**Deleted: handle_databricks_invocations + its route + test cases.**
Databricks was a 57-line thin wrapper at openai.py that parsed JSON,
injected the model from URL into body, and delegated to
``handle_openai_chat``. It enabled
``databricks serving-endpoints query <model> --profile HEADROOM``
direct CLI use. No evidence of active users (no docs, no issues, no
mentions). Databricks-hosted models still work via the standard
``/v1/chat/completions`` surface; LiteLLM has its own Databricks
support too. If a user complains, this PR is a 30-minute revert.
Architectural note: also updated 2 more test dummies
(``_DummyOpenAIHandler`` in routing + WS lifecycle tests) to bind
``_record_request_outcome`` via the free function
``emit_request_outcome`` — same pattern as ``_run_compression_in_executor``.
Final migration tally (from P0 audit + extensions):
* **18 audit sites** + **5 batch.py sites discovered during migration** = 23 sites migrated
* **1 site deleted** (Databricks)
* **0 sites remaining** anywhere under ``handlers/``
Surface impact (this commit):
* openai.py: −168 LOC (315 deletions − 147 insertions)
* batch.py: +35 LOC (124 ins − 89 del; mostly comments)
* proxy_routes.py: −4 LOC (Databricks route gone)
* tests: +11 LOC (dummy `_record_request_outcome` bindings, 2 sites)
* Net: ~−126 LOC in production handler code
Tests
* All 157 existing streaming/cache/Codex/anthropic/openai/backpressure/
routes tests pass with zero regressions.
* ruff + ruff-format + mypy clean.
This brings the cumulative refactor delta (across all 3 commits on
this branch) to:
contract introduced (outcome.py + funnel): ~+200 LOC fixed cost
handler migrations (streaming + anthropic +
gemini + openai + batch + WS): ~−700 LOC
Databricks deletion: −57 LOC
────────────────────────────────────────────── ─────────
Net production code delta: ~−557 LOC
Plus +474 LOC of test coverage (RequestOutcome unit tests +
funnel contract assertions).
And every handler now emits identical observable outputs per
request: same metrics shape, same cost_tracker shape, same
RequestLog shape, same PERF format. The wire is uniform.
Builds on the RequestOutcome contract introduced in the previous commit.
This commit collapses **8 more record_request sites** across two
providers, demonstrating that the contract works across the
provider-shape diversity it was designed for:
* `handle_gemini_generate_content` (1 site) — read-only cache, no
write counter, no TTL splits. The funnel's optional fields default
to 0 for everything Gemini doesn't have; no special-casing needed.
* `handle_gemini_count_tokens` (1 site) — sizing helper, no output
tokens, no cache. Funnel handles the "minimal observation" shape
with zero ceremony.
* `handle_anthropic_messages` — **6 sites collapse to 1 funnel call
per site**, including the response-cache-hit path, the
Bedrock/Vertex non-streaming backend path, the main native
Anthropic non-streaming path, and three batch handlers
(create / passthrough / CCR-processed results).
Bug fixes that fall out of the migration:
* The non-streaming Anthropic main site was missing
`attempted_input_tokens=` (one of the 7-of-18 sites flagged in the
P0 audit). Dashboards showing 0% active-savings on non-streaming
Anthropic traffic will now show the correct ratio (= #454/#455
silently retired for this surface).
* Bedrock/Vertex non-streaming site was missing cache args entirely,
hardcoding `cache_hit=False` on RequestLog. Now `cache_hit` is
derived from the outcome correctly. Cache extraction itself is
still a follow-up — but the wire shape is now uniform.
* Three batch handlers (create / passthrough / CCR-processed) were
emitting only `record_request` — no RequestLog, no PERF log. They
now flow through the canonical funnel so batch traffic appears in
`headroom perf` and the recent-requests feed for the first time.
Architectural changes:
* **Extracted the funnel from `HeadroomProxy._record_request_outcome`
into a free function `emit_request_outcome(handler, outcome)`** in
`outcome.py`. The proxy method becomes a thin two-line wrapper.
Reason: test dummies (e.g. `_DummyAnthropicHandler` in
`test_anthropic_pre_upstream_backpressure.py`) need to call the
funnel from their mixin tests without inheriting from
`HeadroomProxy`. A free function with structurally-typed `handler`
arg satisfies both production and test paths without a typing.Protocol
ceremony.
* **Added `from_response_cache: bool = False` to `RequestOutcome`**
to model Headroom's semantic-cache hits separately from
upstream-prompt-cache hits. Both still collapse to the unified
`cache_hit` derived property for downstream consumers, but
dashboards can split them. Previously the cache-hit path
hardcoded `cached=True` to `record_request`; now it's a typed,
explicit signal.
* **Two batch handlers (`handle_anthropic_batch_passthrough`,
`handle_anthropic_batch_results`) now allocate a `request_id`** at
entry. They didn't have one before (they logged
`request_id=None`), but the funnel requires it. Minor logging
improvement.
Tests
* `tests/test_anthropic_pre_upstream_backpressure.py::_DummyAnthropicHandler`
gets a 5-line `_record_request_outcome` that delegates to
`emit_request_outcome`. Same pattern the dummy uses for
`_run_compression_in_executor` / `_next_request_id`.
* All 140 streaming/cache/Codex/anthropic/backpressure tests pass:
- test_request_outcome.py (14)
- test_backend_streaming_cache_metrics.py (4)
- test_proxy_streaming_request_logger.py (8)
- test_proxy_streaming_resilience.py (24)
- test_proxy_anthropic_cache_stability.py (22)
- test_anthropic_pre_upstream_backpressure.py (20)
- test_openai_codex_routing.py (11)
- test_openai_codex_ws_lifecycle.py (10)
- test_responses_ws_pyo3_compression.py (27)
* ruff + mypy clean.
Surface impact
* `anthropic.py`: 6 record_request sites → 0 (all go through funnel).
Net 315 insertions, 273 deletions, but **the insertions are mostly
comments explaining the migration** — actual code change is closer
to a net wash. The wins compound in next migrations.
* `gemini.py`: 2 sites → 0. Net +30 LOC (mostly comments).
* `server.py`: −90 LOC (funnel extracted to free function).
* `outcome.py`: +110 LOC (free function + comments).
Remaining migrations from P0 audit §6 (still pending):
* handle_openai_responses_ws (Codex WS, 2 sites)
* handle_openai_chat non-streaming
* handle_openai_responses HTTP
* handle_gemini_stream_generate_content + handle_google_cloudcode_stream
* handle_databricks_invocations
P0 audit (docs/superpowers/specs/P0-proxy-pipeline-audit.md) catalogued
**18 metrics.record_request call sites** across 4 handler files with **4
distinct argument shapes**: 9 of 18 omitted `cached=`, 7 of 18 omitted
`attempted_input_tokens=` (= bug #454/#455's "headline 0%"), only 4 sites
emitted a `PERF` log line (= bug #327's "msgs=0" sibling — Codex traffic
invisible to `headroom perf`), and `cache_hit` was hardcoded `False` at
9 of 18 RequestLog sites.
The cause was structural, not tactical: every site was independently
deciding what "record this completed request" meant. This PR puts a
single value type + a single function between the handlers and the
metrics layer.
Two new files:
* `headroom/proxy/outcome.py` — `RequestOutcome` frozen dataclass.
Captures everything we ever need to record about one completed
request: identity, tokens, cache stats (per-TTL splits + inferred
flag for OpenAI), timing, transforms, diagnostics. Provider-specific
fields default to neutral values so non-Anthropic handlers don't have
to know about 5m/1h splits, non-OpenAI handlers don't have to know
about inferred writes, etc. Computed properties (`cache_hit`,
`cache_hit_pct`, `savings_pct`) make "forgot to compute it" mistakes
structurally impossible.
* `HeadroomProxy._record_request_outcome` in `server.py` — the single
funnel. Owns the four downstream effects in canonical order:
1. `metrics.record_request(...)` with the FULL kwarg set
2. `cost_tracker.record_tokens(...)` with `(model, tokens_saved,
optimized_tokens)` positional + all cache kwargs
3. `logger.log(RequestLog(...))` with `cache_hit` correctly derived
4. structured `PERF` log line in the canonical key=value shape
Migrated three streaming finalizers in this PR:
* `_finalize_stream_response` (Anthropic native + OpenAI HTTP streaming)
* `_stream_response_bedrock` (Bedrock-native Anthropic streaming)
* `_stream_openai_via_backend` (OpenAI/Azure backend via LiteLLM/AnyLLM)
All three previously had inline, drifted versions of the four-call
sequence. Each is now ~70 fewer lines: build a `RequestOutcome` from
local context, call `self._record_request_outcome(outcome)`. The
prefix-tracker mutation (Anthropic-specific) stays outside the funnel —
different concern.
Six more migrations queued for follow-up PRs (handle_anthropic_messages
6 sites, handle_openai_chat, handle_openai_responses, handle_openai_
responses_ws 2 sites, handle_gemini_*, handle_databricks_invocations).
Each is mechanical now.
Tests
* New: `tests/test_request_outcome.py` — 14 tests covering value-type
contract (frozen, derived properties, neutral defaults) + funnel
contract (full record_request kwargs, canonical record_tokens shape,
derived cache_hit in RequestLog, PERF log key=value format,
optional cost_tracker/logger). Bind the real production method via
descriptor binding so the test exercises the real implementation, not
a fork.
* All 135 existing streaming/cache/Codex tests pass with zero
regressions (`tests/test_backend_streaming_cache_metrics.py`,
`test_proxy_streaming_request_logger.py`, `test_proxy_streaming_resilience.py`,
`test_proxy_anthropic_cache_stability.py`, `test_openai_codex_*`,
`test_responses_ws_pyo3_compression.py`, `test_anthropic_pre_upstream_backpressure.py`).
* `mypy headroom/proxy/{outcome,server,handlers/streaming}.py` clean.
* `ruff check` clean.
Surface impact
* −238 lines from `handlers/streaming.py` (deduplication).
* +92 lines in `server.py` (the funnel — counted ONCE, not 18×).
* +130 lines in new `outcome.py` (frozen dataclass + docstrings).
* Net production code: ~−16 lines today, ~−500 lines after the
remaining six migrations land.
Forward design constraints (per
docs/superpowers/specs/P0-proxy-pipeline-audit.md §7)
* KISS: one value type, one function, no factory hierarchies.
* No regex in routing — handlers stay provider-specific in their
upstream contract. Output unification only.
* No silent fallbacks — `cache_hit` is computed, not defaulted.
`cache_inferred=True` is the loud signal when OpenAI write count
came from `_infer_openai_cache_write_tokens`.
* PERF format frozen so `headroom/perf/analyzer.py` keeps parsing
cleanly; P3 follow-up replaces the free-text shape with a
structured event.
PyPI rejected the v0.21.37 release publish with:
HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
Project size too large. Limit for project 'headroom-ai' total size is 10 GB.
PyPI inventory check confirmed: **191 versions × ~213 MB/release =
10.00 GB exactly** — at the cumulative project storage ceiling. Each
recent release ships 12 wheels × ~16-18 MB each.
Post-mortem inspection of a production wheel
(``headroom_ai-0.21.36-cp311-cp311-manylinux_2_28_x86_64.whl``)
showed the binary was ``not stripped``:
.text 18.3 MB (code)
.rodata 11.4 MB (Magika model + ONNX runtime data)
.strtab 4.9 MB (debug strings — strippable)
.eh_frame 1.9 MB (unwind tables)
.symtab 1.5 MB (debug symbols — strippable)
.gcc_except_table 1.2 MB
This commit adds a release profile:
[profile.release]
strip = "symbols"
lto = "thin"
codegen-units = 1
That:
* Strips ``.symtab`` + ``.strtab`` (~6.4 MB direct savings per wheel)
* Enables thin link-time optimization for cross-crate dead-code
elimination (~5-10% ``.text`` savings)
* Single codegen unit for better inlining + DCE at the cost of
~30-50% slower release builds (acceptable for CI)
Deliberately NOT setting ``panic = "abort"``:
* The proxy is a long-lived async process. A panic on one bad
request triggering process abort would disconnect every concurrent
client. Accept the smaller savings; keep unwind behaviour.
Estimated impact
* Per wheel: ~16-18 MB → ~10-11 MB (40% smaller)
* Per release (12 wheels): ~213 MB → ~130 MB
* PyPI capacity: ~30+ more releases before hitting 10 GB again
Verification
* Local build of ``headroom._core`` with new profile:
``.so`` size 29 MB on macOS arm64 (was ~45 MB pre-fix; final wheel
compressed will be smaller on Linux which also benefits from the
``strip`` directive).
* 77 Rust-parity tests pass — extension still functional.
* Single-codegen-unit slows build by ~30-50% but maturin/cibuildwheel
build time was never the bottleneck.
Forward strategy (separate work)
* Submit a PyPI project-size-limit-increase request to unblock the
immediate release.
* Adopt a release-deprecation policy: yank versions older than N
patches per minor; consider dropping Python 3.10 wheels (EOL'd
October 2026) and manylinux_2_28_aarch64 wheels (niche audience,
largest at 18.75 MB).
* Investigate runtime-download for Magika model (~10 MB further
savings) — same pattern Kompress already uses.
Second CI failure on the same stress test, this time with the ratio
threshold:
AssertionError: p99/p50 ratio is 7.4× (p50=28406ms, p99=210468ms).
Expected < 5× — wall=651s.
Root cause: previous iteration used MIXED frame sizes (200 B → 16 KB)
across 30 concurrent sessions on a 2-vCPU CI runner. The p99/p50
ratio captured TWO things:
1. The contention-tail signature we want to catch (≈27× pre-fix).
2. Size-variance compute spread (≈3–8× depending on hardware).
On dev hardware the (2) component was small relative to the
contention signal. On CI it dominated, masking the (1) detection.
The fix is to remove (2) from the measurement entirely:
* All 60 frames are now identical 4 KB plain-text payloads.
* Concurrency dropped from 30 to 12 — still > the deleted 10-slot
semaphore (so the bug pattern, if reintroduced, surfaces), but
doesn't oversaturate the 2-vCPU CI runner with OS-scheduler
noise.
* Frames per session dropped from 12 to 5 → 60 total samples,
still enough to compute a meaningful p99, with bounded runtime.
* Threshold tightened from 5× to 4×. On uniform workload the only
legitimate source of p99/p50 spread is OS-level scheduling
noise (≈2–3×). 4× sits comfortably between that and the bug
signature (≈27×).
Local re-run: 60 frames, 0.59s wall, p50=108ms p99=198ms ratio=1.83×
— well under the 4× ceiling, captures the bug shape unambiguously.
Test design note added to docstring explaining the why so future
CI hardware changes don't trip the threshold again.
CI failure on first attempt at the stress test:
p99 per-frame elapsed_ms = 214020; expected < 1000
GitHub Actions runners (2 vCPU, shared) are 5–50× slower in absolute
terms than the 12-CPU dev box this PR's baseline numbers were taken on.
The absolute thresholds (p99<1000ms, wall<5s) intentionally caught the
bug on dev hardware but force CI either to skip the test or to use
thresholds so loose they stop catching the regression.
The bug being guarded against creates a *bimodal* latency distribution
(most fast, some catastrophic) via the deleted
``_CODEX_WS_UNIT_ROUTER_SEMAPHORE``. Pre-fix on dev: p50=91ms,
p99=2433ms → ratio=27×. The contention *pattern* is invariant — if the
semaphore tail comes back, the ratio explodes regardless of CPU speed.
This commit:
* Removes the machine-dependent absolute thresholds (p99<1000ms,
wall<5s).
* Keeps the p99/p50 ratio test (now strictly < 5×, no special floor).
* Adds a `print()` of the full distribution so CI logs always show
numbers — useful both for diagnosing failures and tracking drift.
Local re-run: p50=264ms p99=492ms ratio=1.87× — well under the 5×
ceiling and the test still proves the contention tail is gone.
CI ran the Codex compression scheduler stress test on python 3.10/3.11/
3.12/3.13 and all four versions failed with:
ModuleNotFoundError: No module named 'scripts.replay_codex_ws_load'
The test imports ``boot_proxy``, ``warmup``, ``replay_session``, ``Frame``,
and ``Scenario`` from the replay tool to drive 30 concurrent compression
calls and assert no p99 contention tail (the very regression this PR
fixes). The tool was untracked because ``scripts/*`` is gitignored by
allowlist — local-only tools work fine for measurement but CI cannot
import them.
Add the replay tool to the allowlist so it ships. Same pattern as
``scripts/smoke_issue_327.py`` and other single-purpose scripts already
on the allowlist. The tool is genuinely useful beyond this PR: it lets
any contributor reproduce the Codex slowness baseline numbers and
measure their own fix against the same workload shape.
Local re-run with the tool committed: 3 passed, 1 skipped — identical
to the pre-fix branch run.
Production proxy logs (2026-05-14) showed 305 `TimeoutError: forwarding
original frame` warnings and 12,905 `slow compression unit elapsed_ms>1s`
log entries, with p99 unit elapsed_ms = 587 SECONDS, max = 1987 seconds,
and WS session p90 duration = 48 minutes. The cause was a two-layer
concurrency bug in `_compress_openai_responses_payload`:
* `_CODEX_WS_UNIT_ROUTER_SEMAPHORE = threading.BoundedSemaphore(10)` — a
process-global gate over every compression unit in every frame across
every concurrent session. At ~3+ active Codex users it saturates;
subsequent units block on acquisition. The 30s parent timeout fires;
uncompressed frames forward but the user already waited 30s.
* `time.perf_counter()` started BEFORE semaphore acquisition, so
`elapsed_ms` conflated wait time with compute. A `strategy=passthrough`
unit on 148 bytes (a no-op) showed `elapsed_ms=60917` in the log — 60
seconds of "compression" that was actually 60 seconds of queueing.
* `concurrent.futures.ThreadPoolExecutor(max_workers=worker_count)` was
created and torn down per frame, layered on top of the
`self._compression_executor` proxy-wide pool. Pool-on-pool plus the
global semaphore made the bug self-amplifying.
Fix: delete all three. Process routed units serially within the frame-
level worker thread. Frame-level parallelism is already provided by the
existing `self._compression_executor` (32 workers, sized `min(32,
cpu*4)`, instrumented). Bonus: add a structured PERF log emit from
`handle_openai_responses_ws` so Codex traffic is no longer invisible to
`headroom perf` — same visibility bug class as #327, fixed for Codex.
Tier 3 replay against `scripts/replay_codex_ws_load.py` (30 concurrent
sessions × 30 frames = 900 frames, 4.6MB) — same machine, before vs
after:
| metric | pre-fix (main) | post-fix | Δ |
|---------------------|-----------------|----------------|------------|
| p50 per-frame | 91 ms | 258 ms | +183 % |
| p99 per-frame | 2 434 ms | 275 ms | −89 % |
| max per-frame | 2 681 ms | 368 ms | −86 % |
| p99 / p50 ratio | 27 × | 1.06 × | tail gone |
| wall time | 7.54 s | 7.09 s | −6 % |
| errors | 0 | 0 | — |
The median rises modestly at high load (the cost of KISS: serial units
instead of intra-frame parallelism, documented in EC2 of the design).
That trade is right: the catastrophic p99 contention tail is what users
felt, and it collapses 9×. At low load (10c × 20f) the fix is strictly
equal-or-better on every metric — the trade is invisible until the
semaphore was actually the binding constraint.
Tests
* tests/test_codex_ws_compression_scheduler.py — three regression
guards: source-level assertions that `_CODEX_WS_UNIT_ROUTER_SEMAPHORE`
and `concurrent.futures.ThreadPoolExecutor` cannot reappear in
handlers/openai.py, plus a concurrency stress test asserting p99 <
1000ms and p99/p50 < 5× at 30 concurrent sessions.
* All 95 existing Codex/streaming/cache tests pass with zero
regressions.
Removed surface
* Deleted `_CODEX_WS_UNIT_ROUTER_MAX_WORKERS`,
`_CODEX_WS_UNIT_ROUTER_SEMAPHORE`, `_codex_ws_unit_worker_count`,
and the `HEADROOM_CODEX_WS_UNIT_WORKERS` env knob. Net −13 module-
level lines + one undocumented env var gone from the public surface.
Two regressions surfaced as "Cache write: 0" in `headroom perf` and the
dashboard for every backend-routed streaming request (e.g. SvenMeyer's
DROID CLI > headroom > Azure GPT-5.5 setup):
* `_stream_openai_via_backend` parsed only `completion_tokens` and never
read `prompt_tokens_details.cached_tokens` from the upstream usage
frame. It also emitted no PERF log line at all, so `headroom perf`
couldn't even count the request to report numbers. Now buffers SSE
bytes, drains via `_parse_sse_usage_from_buffer(provider="openai")`,
infers writes via `_infer_openai_cache_write_tokens` (only when the
upstream actually reported usage — mirrors `_extract_responses_usage`),
threads cache values into `record_request`, `cost_tracker.record_tokens`,
the RequestLog, and a real PERF log line.
* `_stream_response_bedrock` hardcoded `cache_read=0 cache_write=0
cache_hit_pct=0` in its PERF line regardless of what `message_start.usage`
reported. Extended `stream_state` with `cache_read_input_tokens` and
`cache_creation_input_tokens` (plus 5m/1h TTL buckets), captures them
from `message_start`, threads through `record_request(cached=...)`,
`cost_tracker.record_tokens(...)`, and `RequestLog(cache_hit=...)`.
Tests: four new tests in `test_backend_streaming_cache_metrics.py` cover
both paths plus a source-level regression guard against the hardcoded
zero string reappearing.
Several signals AI agents and search engines use to discover and
install a project were misaligned or missing:
* ``docs/app/layout.tsx`` set ``metadataBase`` to
``https://chopratejas.github.io/headroom/`` while the live docs run
on Vercel — every page's ``og:url`` and ``twitter:url`` resolved to
a URL that returns 404 for ``/llms.txt``. Now points at the live
Vercel host (overridable via ``NEXT_PUBLIC_SITE_URL`` for a future
custom domain). Adds explicit ``openGraph`` and ``twitter`` metadata
so social shares render a card with the project's pitch.
* No ``llms.txt`` at the GitHub repo root. AI agents crawling
``github.com/chopratejas/headroom/`` saw only the README. The new
``llms.txt`` follows the llmstxt.org convention: 1-line pitch,
canonical docs links, copy-paste install commands (pip / npm /
Docker / proxy / ``headroom wrap``), and entry points for the
library, proxy, MCP server, and SDK integrations. Points at the
Fumadocs-generated ``/llms.txt`` and ``/llms-full.txt`` for the
full picture.
* ``pyproject.toml`` ``Documentation`` URL pointed at the GitHub
README anchor. Updated to point at the docs site so PyPI visitors
land on searchable docs, and adds an ``AI / LLM Index`` URL
pointing at the Fumadocs ``/llms.txt``.
* No explicit AI-bot allow list. Added ``docs/app/robots.ts`` (Next
13+ App Router convention) with explicit allows for GPTBot,
ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot,
ChatGPT-User, Cohere-AI, CCBot, and Applebot-Extended. Wildcard
allow as the catch-all. Advertises the sitemap.
* No ``sitemap.xml`` route. Added ``docs/app/sitemap.ts`` that pulls
every Fumadocs page out of ``source`` (same source backing
``/llms.txt``, search, and OG images) so search and AI crawlers
can enumerate doc pages without scraping HTML.
* README didn't tell AI agents where to look. Added a 2-line
pointer near the top nav row: read ``/llms.txt`` here, or fetch
the live index / full docs blob.
Also tightened the GitHub repo description and added five topics
(``claude-code``, ``cursor``, ``tokens``, ``prompt-engineering``,
``typescript``) via ``gh repo edit`` — that's already live on the
repo, not part of this commit.
No Python or Rust code changes; ``make ci-precheck`` was run to
confirm the test slice still passes.
`TrafficLearner._extract_preferences` ran three regex patterns over raw
user-message text and saved any match as a `User preference: <captured>`
memory. Two compounding bugs made ~10% of the reporter's saved memories
(187 of 1796) garbage:
1. **System-reminder content was matched.** Claude Code injects
`<system-reminder>…</system-reminder>` blocks into user-role
messages — scaffolding ("don't mention this reminder", "use colgrep
instead of Grep", "never bypass signing") that hits every correction
trigger. The learner happily persisted scaffolding as authoritative
user preferences.
2. **Capture groups were fixed-length windows.** `(.{10,100})` grabbed
the next 10–100 chars with no boundary awareness, producing
mid-word truncations like `User preference: of Grep, Glob. When
spawning agents, mention colgrep features a`.
This change rewrites `_extract_preferences` to be **regex-free** and
adds two layered defences:
- `_strip_system_reminders` (literal `str.find` scan, no regex)
removes `<system-reminder>…</system-reminder>` blocks from user
text before any pattern matching. Unclosed reminders drop to
end-of-string. Case-insensitive on the tag name only. ~95% of the
reporter's noise sample comes from this single layer.
- A token-based correction scanner replaces the three `re.compile`
patterns. It tokenises on whitespace (lowercasing once, up front),
matches trigger sequences as ordered token lists (`don't`, `do not`,
`stop`, `never`, `avoid`, `no use`, `no try`, `no do`, `instead`),
and captures the trailing content until a sentence terminator
(`.!?\n`) or end-of-input. Captures shorter than 10 chars are
rejected (stray triggers), and captures that hit the 78/98-char cap
without finding a terminator are rejected (rambling fragments). The
former noise — `colgrep instead of Grep, Glob. When spawning…` —
fails this gate; short complete user utterances
(`don't use git push, I'll push manually`) still pass because
end-of-input counts as a boundary.
Net regex count in this file: -3, +0.
`_hydrate_persisted_state` already runs in `start()` and seeds
`_saved_hashes`/`_persisted_ids` from prior rows, so cross-restart
dedup is already wired up — the reporter's "doesn't survive restarts"
note was partially outdated. The narrow remaining edge (in-process
`_dedup_window=100` eviction within a single very-long-running
process) self-heals on next restart and is left as a separate
follow-up.
Tests: 17 new across `TestStripSystemReminders`,
`TestExtractPreferencesSystemReminderFiltering`,
`TestExtractPreferencesRealCorrections`, and
`TestExtractPreferencesSentenceBoundary`. Full traffic_learner suite:
139 passing. ci-precheck green.
Memory retrieval was partitioned only by `x-headroom-user-id`. Claude
Code never sets that header, so every project a user worked on landed
in one global `default` bucket; the proxy then injected semantically
similar memories from that mixed bucket into every `/v1/messages`
request, regardless of which repo the session was actually about. The
injected `## Relevant Memories` block reads like a prompt-injection
payload and Claude has been seen to refuse to act on it, defeating the
feature.
This change makes leakage structurally impossible by giving each
resolved workspace its own SQLite database file. The wrong DB is
simply not open during a request.
- `headroom/memory/storage_router.py` (new) — `MemoryStorageMode`
(project/user/global), `ProjectResolver` (x-headroom-project-id →
x-headroom-cwd → --memory-project-root CLI override → env-block
parse: `Primary working directory:` / `Working directory:` / `cwd:`,
no regex), and `BackendRouter` with an LRU of open `LocalBackend`s
keyed by db_path.
- `proxy/memory_handler.py` — `MemoryConfig.storage_mode` defaults to
`PROJECT`. Provider handlers build a `RequestContext` once and pass
it through; `search_and_format_context`, `handle_memory_tool_calls`,
and the `_execute_*` methods route save/search/update/delete on the
per-project backend. Qdrant-neo4j gets a composite
`user::project_key` partition so external Mem0-style deployments
also isolate per project without a parallel collection.
- Fix C — injected block carries provenance:
`## Relevant Memories (workspace: <basename>, scope: project)`.
CCR proactive-expansion block gets a matching workspace tag.
- `memory/factory.py` — process-wide embedder cache so opening N
project DBs doesn't load the embedder N times. OpenAI key
validation runs ahead of the cache.
- CLI — `--memory-storage={project,user,global}` (default `project`),
`--memory-project-root` override, rewritten `--memory` help text,
banner reports storage mode.
- Migration UX — if the legacy single-file DB has content while
project mode is active, an INFO log points users at
`--memory-storage=global`. Bridge currently only syncs the legacy
DB; a WARN fires when bridge + project mode are combined.
Backward-compatible: legacy `~/.headroom/memory.db` untouched and
reachable via `--memory-storage=global`. `request_context` is
keyword-only on entry points so existing tests/mocks keep working.
Tests: 24 new (resolver tiers, LRU eviction, two-cwd isolation,
user-mode partition, legacy fallback, provenance headers); full
suite 5260 passing, ci-precheck green.
