1 commit

Author	SHA1	Message	Date
chopratejas	71d5a7b545	fix(proxy): MemoryDecision contract + 3 bypass bugs + drop 500-char query cap ... Three bug classes fixed plus three architectural extension points, together making the memory subsystem uniform across all five sites and ready for future Mem0/Letta/Cognee backend integration. ## Bug fixes * **3 sites silently ignored `x-headroom-bypass: true`** — ``anthropic.py:1303``, ``openai.py:1620`` (chat), ``gemini.py:382`` injected memory under bypass, mutating request bytes when the user explicitly asked for byte-faithful passthrough. Now gated on ``MemoryDecision.decide(...)`` which honours bypass uniformly. * **500-char query truncation** — ``memory_handler._extract_user_query`` capped at 500 chars, silently throwing away signal. None of Letta / Mem0 / Cognee / Supermemory truncate. Removed; the embedding model handles its own window. * **Gemini had no timeout** on ``search_and_format_context`` — the only chat handler without one. A slow backend could stall requests. Added ``asyncio.wait_for`` matching Anthropic + OpenAI Chat + Responses. * **WS injected into ``body["instructions"]``** — the system / cache-hot-zone field, violating invariant I2 (all other handlers inject at user-message tail). Switched to ``ws_response_body["input"]`` for string-shaped input; list-shaped input deferred to the Rust handler with a clear log. ## New value types (extension points) * ``MemoryDecision`` — frozen dataclass + factory. Five-way skip reason enum (``bypass_header`` / ``no_handler`` / ``no_user_id`` / ``mode_disabled`` / ``mode_tool``). ``apply_to_tags()`` surfaces the skip reason in ``RequestOutcome.tags["memory_skip_reason"]`` — dashboards can now slice memory-blind traffic by cause. * ``MemoryQuery`` — multi-source retrieval query. ``from_messages()`` walks the conversation and extracts latest user text + recent tool outputs + recent assistant turns at FULL fidelity (no truncation). Handles both OpenAI-shape ``role: tool`` and Anthropic-shape ``tool_result`` content blocks. ``to_embedding_input()`` produces a delimited concatenation the embedder sees as structured context. * ``MemoryInjectionBudget`` — uniform token / entry / similarity bound on the formatted injection block. Pre-this-PR no cap (~4000 tokens could land per request). Default 1024 tokens / 10 entries / 0.3 similarity floor. ``apply_to_text()`` truncates at line boundaries so dashboard renders intact bullet points. ## Migration scope — all 5 sites uniform at the GATE level | Site | Handler | Pre-PR gate | Post-PR gate | |---|---|---|---| | 1 | anthropic.py | `memory_handler and memory_user_id` | `memory_decision.inject` | | 2 | gemini.py | `memory_handler and memory_user_id` | `memory_decision.inject` | | 3 | openai.py chat | `memory_handler and memory_user_id` | `memory_decision.inject` | | 4 | openai.py Responses | `memory_handler and memory_user_id and not _bypass` | `responses_memory_decision.inject` | | 6 | openai.py WS | `memory_handler and body and not _ws_bypass` | `ws_memory_decision.inject` | Site 5 (Responses bypass-elif log-only branch) is preserved verbatim. ## Deliberately deferred (separate PRs) * **Memory injection order inversion** — sites 4 and 6 inject BEFORE compression; sites 1/2/3 inject AFTER. Moving 4 + 6 to post-compression needs its own focused cache-stability testing. * **Importance scoring** — recency × source × access-count. * **Per-memory atomize-and-split** — Mem0/Supermemory pattern. * **AST-aware code chunking for tool outputs** — Supermemory's code-chunk approach. The contracts shipped here (``MemoryQuery`` + ``MemoryInjectionBudget``) are the extension points those will plug into. ## Test coverage * 20 new tests on ``MemoryDecision`` * 14 new tests on ``MemoryQuery`` (full-fidelity, multi-source) * 10 new tests on ``MemoryInjectionBudget`` * 3 new AST contract tests (no raw gate; no system writes; every search call passes ``query=``) * All existing memory + cache-stability tests still pass (222 passed) ## Rust portability Every new value type ports cleanly to a frozen Rust struct. Pure functions, no I/O, no global state. Same Python ↔ Rust parity-test pattern that ``CompressionDecision`` already uses. ## Zero-regression contract Existing chat/completion harnesses (Claude Code, Codex, Cursor, Continue, Aider) see ZERO wire-byte changes when bypass is NOT set. When bypass IS set, the 3 chat handlers now correctly skip memory injection — that's the bug fix, not a regression.	2026-05-19 11:13:52 -05:00