1 commit

Author	SHA1	Message	Date
Raúl	9c30b62962	fix: skip cross-turn dedup pointers on OpenAI chat streaming (#3191) ... ## Description Cross-turn dedup (`HEADROOM_DEDUPE` / `enable_cross_turn_dedup`, plus the cold-prefix recompaction router) folds a repeated tool-output span into a one-line in-context pointer, `[↑NL same as msg M: 'anchor']`. That pointer is only recoverable where the model can resolve the reference. On the OpenAI chat-completions STREAMING path (what `headroom wrap copilot` serves) it cannot, for two independent reasons: 1. The proxy itself logs `CCR: skipping retrieval-tool injection for OpenAI chat streaming; this path cannot intercept tool calls`, so no `headroom_retrieve` tool exists on this path and nothing can mechanically resolve a fold. 2. The pointer names its source as `msg M`, Headroom's internal message index. OpenAI-compatible chat clients never show the model numbered messages, so the reference is unresolvable even though the original bytes are technically still earlier in the same request. Observed with Kimi k2.7-code / k3 via `wrap copilot`: the model treats the pointer as deleted output, reports "the renderer is deduplicating/compressing", and retry-loops near-identical reads (one session burned ~200 turns; a folded conflicted-files listing hid 4 of 5 conflicted files and the agent committed unresolved `<<<<<<<` markers). The router already keeps unrecoverable LOSSY output verbatim (`lossy_unrecoverable_skipped`). Dedup folds are lossless in theory but unrecoverable in practice on this path; this PR gives them the same recoverability gate. Closes #3190 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/transforms/content_router.py`: `ContentRouter.apply()` accepts a per-request `cross_turn_dedup_recoverable` kwarg (default `True`, so every existing caller is byte-identical). When `False`, the cross-turn dedup pass is skipped and repeated spans stay verbatim, mirroring the recoverability posture of the lossy `lossy_unrecoverable_skipped` guard. Config comment on `enable_cross_turn_dedup` documents the gate. - `headroom/proxy/handlers/openai.py`: `handle_openai_chat` computes the gate from the same predicate that already gates CCR retrieval-tool injection, `_should_inject_openai_chat_ccr_tool(ccr_inject_tool, stream)`, and threads it into both `openai_pipeline.apply(...)` call sites (token-mode and non-token-mode branches). Streaming chat requests skip the fold; buffered (non-streaming) chat, which can inject and redeem the retrieval tool, keeps folding. - `headroom/transforms/cold_prefix.py`: `cold_recompact_messages` no longer hardcodes pointer emission; new keyword-only `cross_turn_dedup_recoverable: bool = True` is forwarded to the router gate. The only caller (Anthropic cache-mode cold turn) keeps the default and is unchanged. - `tests/test_cross_turn_dedup.py`: router-gate regression tests (unrecoverable path keeps verbatim bytes for both the OpenAI `role:tool` string shape and the Anthropic `tool_result` block shape; default/explicit-`True` still folds). - `tests/test_cold_prefix.py` (new): recompaction folds by default (Anthropic path unchanged) and keeps verbatim bytes with `cross_turn_dedup_recoverable=False`. - `tests/test_openai_chat_dedup_recoverability.py` (new): end-to-end through the real `/v1/chat/completions` handler with `HEADROOM_DEDUPE=1`, capturing the exact upstream request body: `stream=True` keeps both copies byte-verbatim with no `[↑` pointer; `stream=False` still folds; `stream=False` under `--lossless` (which forces `ccr_inject_tool=False`) also keeps verbatim bytes, locking the intended coupling of "no retrieval tool" to "no bare pointer". ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # BEFORE (branch base, fix reverted): the streaming regression test fails, # the upstream body carries the unresolvable pointer and drops the bytes. $ git stash push headroom/ && uv run pytest -q \ tests/test_openai_chat_dedup_recoverability.py::test_streaming_chat_keeps_verbatim_bytes_no_dedup_pointer E assert '[↑' not in "fix the ove...t merge.py']" E '[↑' is contained here: E [↑14L same as msg 2: '$ cat merge.py'] FAILED tests/test_openai_chat_dedup_recoverability.py::test_streaming_chat_keeps_verbatim_bytes_no_dedup_pointer (same run: test_cold_recompact_unrecoverable_path_keeps_verbatim_bytes also fails pre-fix; both recoverable-path legs pass before and after) # AFTER (full diff applied): $ uv run pytest tests/test_cross_turn_dedup.py tests/test_cold_prefix.py \ tests/test_openai_chat_dedup_recoverability.py \ tests/test_proxy/test_openai_chat_ccr_injection.py tests/test_no_ccr_lossy.py \ tests/test_openai_chat_turn_hooks.py tests/test_openai_chat_tool_desc_compaction.py \ tests/test_responses_cross_turn_dedup.py -q 45 passed, 2 warnings in 8.75s $ uv run pytest tests/test_proxy/ tests/test_openai_codex_routing.py \ tests/test_openai_chat_turn_hooks.py tests/test_openai_chat_tool_desc_compaction.py \ tests/test_openai_beta_session_sticky.py tests/test_openai_max_completion_tokens.py \ tests/test_no_ccr_lossy.py tests/test_netcost_gate.py tests/test_agent_savings.py \ tests/test_cross_turn_dedup.py tests/test_cold_prefix.py \ tests/test_openai_chat_dedup_recoverability.py -q 424 passed, 2 warnings in 73.52s $ uv run ruff format --check <touched files> && uv run ruff check <touched files> All checks passed! $ uv run mypy headroom/transforms/cold_prefix.py headroom/transforms/content_router.py headroom/proxy/handlers/openai.py Success: no issues found in 3 source files $ cargo fmt --all -- --check # FMT_OK $ cargo clippy --all-targets # 2 pre-existing warnings in untouched lib-test code, no errors $ cargo test # all targets green; see Additional Notes for the one environmental exception ``` ## Real Behavior Proof - Environment: macOS (Darwin), Python 3.13, repo tip `upstream/main` 5e0ce242 (v0.36.2). No secrets, no external network: the proof drives the real proxy handler in-process via FastAPI `TestClient` with the upstream send stubbed, capturing the exact request body the provider would receive. - Exact command / steps (copy-pasteable, self-contained): next lines ```sh # 1. The bug, on the branch base (pointer emitted on the streaming path): git stash push headroom/ # or check out upstream/main uv run pytest -q tests/test_openai_chat_dedup_recoverability.py # streaming leg FAILS git stash pop # 2. The fix: uv run pytest -q tests/test_openai_chat_dedup_recoverability.py # both legs pass ``` The test posts a chat-completions request whose history contains two identical multi-line tool outputs (the shape that folds), with `HEADROOM_DEDUPE=1`, and asserts on the captured upstream body: - `stream=True` (the `wrap copilot` shape): both copies forwarded byte-verbatim, no `[↑NL same as msg M]` pointer anywhere. - `stream=False` (buffered, retrieval tool injectable): the repeated span still folds to a pointer; the earliest copy stays verbatim as the in-context original. - Observed result: BEFORE, the streaming leg fails with the pointer present in the upstream body (same `transforms=router:cross_turn_dedup:N` evidence seen in proxy.log when the bug bit). AFTER, streaming keeps verbatim bytes and buffered keeps folding; the full touched-module suite (423 tests) is green. - Not tested: a live `wrap copilot` session against the real Copilot API (needs a subscription token; the in-process test captures the identical upstream body the handler produces). The Responses API path (`_dedup_responses_output_items`, Codex) is intentionally untouched: Responses streaming has a separate buffered-CCR path that can intercept tool calls. `/v1/compress` derived pipelines keep the default (recoverable) behavior. Separately worth verifying in a follow-up: whether `headroom_retrieve` resolves `msg M` dedup pointers on the paths that keep folding, or only CCR `hash=` content markers (the Anthropic-path fold is retained per the issue's scope, where it has not been observed to cause retry loops). ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: N/A - Stable/default behavior changed: only the OpenAI chat-completions request path, and only when cross-turn dedup is active (opt-in `HEADROOM_DEDUPE=1`, or cold-prefix recompaction): streaming chat now keeps repeated tool-output bytes verbatim instead of emitting `[↑NL same as msg M]` pointers, and (because `--lossless` forces `ccr_inject_tool=False`) buffered chat in lossless mode does the same. Buffered chat with CCR on, Anthropic, Responses, and `/v1/compress` are byte-identical to before (default `cross_turn_dedup_recoverable=True`; the Responses fold is covered by the untouched, still-green `tests/test_responses_cross_turn_dedup.py`). - Kill switch / disable path: dedup remains opt-in via `HEADROOM_DEDUPE`; the gate itself can be overridden per request by passing `cross_turn_dedup_recoverable=True`. - Unsafe override required: no - Qualification impact: none - Rollback path: revert the single commit; no state, schema, or config migration involved. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation (docstrings + config comments) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - Mirrors the existing recoverability precedent: the lossy path already refuses to emit unrecoverable output (`lossy_unrecoverable_skipped`, issue #1307); this extends the same posture to cross-turn dedup folds. - The gate reuses `_should_inject_openai_chat_ccr_tool`, the predicate that already decides whether the chat path can redeem an injected retrieval tool, so the two can never drift apart. - Prefer-false-negatives posture: a skipped fold only ever means bytes stay verbatim; no content is dropped, reordered, or lossy-transformed by this change. - Secondary operational bug noticed while diagnosing (NOT fixed here, separate issue candidate): all concurrent proxy processes write the same `~/.headroom/logs/proxy.log` with independent rotating handlers, so rotation stomps history across `wrap` instances on different ports. - Local environment note: `cargo test` on this machine hangs inside `crates/headroom-core/tests/kompress_parity.rs` (both tests stall in `ort` ONNX-runtime environment init, reproducible on the untouched branch base; this PR changes no Rust). With those two tests skipped, the full Rust suite is green (all targets `ok`, 0 failed). `cargo clippy --all-targets` and `cargo fmt --all -- --check` pass as-is.	2026-08-21 15:51:05 -07:00