From 7784bb184614bdcd7ba1afd93a23e8d1458ecc96 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ra=C3=BAl?= Date: Sun, 23 Aug 2026 20:53:17 +0200 Subject: [PATCH] fix(transforms): stop folding datetime-prefixed user messages as search results (#3221) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Description Interactive `headroom wrap copilot` sessions intermittently lose the user's message: the model answers "How can I help you today?" to a real task prompt. Root cause: Copilot CLI prepends `…` to every interactive user turn; the ISO-8601 timestamp matches the grep `file:line:` detector, so a datetime + one-line prompt (1 match / 2 non-empty lines = 50% ≥ 30%) classifies as `SEARCH_RESULTS`, and `SearchCompressor` — which keeps only detector-matching lines — deletes the prompt before upstream. On the OpenAI chat streaming path there is no retrieval tool, so the loss is unrecoverable. Fix: `_try_detect_search` now (a) requires the pre-colon segment to look like a file path (no `<`, `>`, `=`), and (b) requires at least two matching lines, so one coincidental `word:digits:` line can no longer classify a whole payload. A genuine one-line grep result loses nothing: all its lines match, so the compressor would have kept it verbatim anyway. Closes #3220 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/transforms/content_detector.py`: new `_is_search_result_line` helper (path-like prefix gate); `_try_detect_search` gains a two-matching-line absolute floor. - `tests/test_transforms_content_detection.py`: regression tests — datetime-prefixed one-liner not search; two-line floor; tag-like / `key=value` prefixes rejected; genuine grep output still detected. - `tests/test_transforms_content_router.py`: router-level regression — the incident payload never routes to SEARCH and the prose survives `ContentRouter().compress()`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ .venv/bin/python -m pytest tests/test_transforms_content_detection.py tests/test_transforms_content_router.py tests/test_mixed_content_sections.py tests/test_text_compressors.py tests/test_transforms_tabular.py -q 135 passed in 21.06s $ .venv/bin/ruff check headroom/transforms/content_detector.py tests/test_transforms_content_detection.py tests/test_transforms_content_router.py All checks passed! $ .venv/bin/mypy headroom/transforms/content_detector.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS 26.5 arm64, Python 3.13, editable source build 0.37.0-dev; upstream `api.githubcopilot.com`, cheapest subscription model `kimi-k2.7-code`. - Exact command / steps: standalone copilot-routed proxy (`OPENAI_TARGET_API_URL=https://api.githubcopilot.com headroom proxy --port 8899`) + `.overlay/e2e-copilot-content-probe.sh --port 8899 --model kimi-k2.7-code`, which sends the real interactive wire shape (`…` + one-line sentinel prompt, streaming) and a multi-line control. - Observed result: BEFORE the fix, probe 1 FAIL — model replied "Hello! I see the current datetime is … How can I assist you today?" with proxy log `transforms=router:search:0.50` (prompt deleted). AFTER the fix, both probes PASS — the sentinel echoes verbatim, proving the user message reached upstream intact. - Not tested: other harnesses' interactive wrappers (claude/droid/auggie send different shapes; the detector fix is generic); the mixed-content section splitter has its own grep pattern (out of scope — its 1-line "search" sections are kept verbatim, no data loss). ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: N/A (no flag). - Stable/default behavior changed: content with exactly one `path:line:`-shaped line no longer classifies as search results (stays uncompressed instead — safe direction; compression only ever engages on ≥2 matching lines now). - Kill switch / disable path: N/A. - Unsafe override required: no. - Qualification impact: none. - Rollback path: revert; prior behavior restores (with the bug). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A — proxy transform change; no UI. ## Additional Notes Detection-precision tradeoff is documented in code comments: single-line genuine grep output is no longer folded (no data loss either way — the compressor keeps all-matching content verbatim). A residual edge (prose with ≥2 coincidental `x:1:` lines in ≤6 lines) is accepted and documented in the issue. --- headroom/transforms/content_detector.py | 27 +++++++++++++- tests/test_transforms_content_detection.py | 43 ++++++++++++++++++++++ tests/test_transforms_content_router.py | 20 ++++++++++ 3 files changed, 88 insertions(+), 2 deletions(-) diff --git a/headroom/transforms/content_detector.py b/headroom/transforms/content_detector.py index 5e364fed6..bb8325e42 100644 --- a/headroom/transforms/content_detector.py +++ b/headroom/transforms/content_detector.py @@ -457,6 +457,23 @@ def _try_detect_html(content: str) -> DetectionResult | None: ) +def _is_search_result_line(line: str) -> bool: + """True when a line looks like ``path:line:content`` grep output. + + The bare ``^[^\\s:]+:\\d+:`` shape also matches ISO-8601 timestamps + (``…T09:57:59…``) and XML-ish wrappers harnesses prepend to user turns + (Copilot CLI's ``…`` line), which misroutes prose to + the SearchCompressor — and that compressor keeps only matching lines, + deleting the rest. So the pre-colon segment must additionally look like + a file path: no angle brackets and no ``=`` (rules out markup tags and + ``key=value:12:`` log lines). + """ + if not _SEARCH_RESULT_PATTERN.match(line): + return False + prefix = line.split(":", 1)[0] + return "<" not in prefix and ">" not in prefix and "=" not in prefix + + def _try_detect_search(content: str) -> DetectionResult | None: """Try to detect grep/ripgrep search results.""" lines = content.split("\n")[:100] # Check first 100 lines @@ -465,10 +482,16 @@ def _try_detect_search(content: str) -> DetectionResult | None: matching_lines = 0 for line in lines: - if line.strip() and _SEARCH_RESULT_PATTERN.match(line): + if line.strip() and _is_search_result_line(line): matching_lines += 1 - if matching_lines == 0: + # Absolute floor: a single coincidental `word:digits:` line (a timestamp, + # a URL, a time literal inside prose) must not classify a whole payload as + # search results — the SearchCompressor drops every non-matching line, so + # a false positive is data loss. A genuine one-line grep result loses + # nothing by staying uncompressed: all of its lines match, so the + # compressor would have kept it verbatim anyway. + if matching_lines < 2: return None # Calculate confidence based on proportion of matching lines diff --git a/tests/test_transforms_content_detection.py b/tests/test_transforms_content_detection.py index 1201c2a60..e74776292 100644 --- a/tests/test_transforms_content_detection.py +++ b/tests/test_transforms_content_detection.py @@ -170,6 +170,49 @@ def test_search_detection_uses_match_ratio() -> None: assert _try_detect_search("\n\n") is None +def test_search_detection_rejects_datetime_prefixed_user_message() -> None: + """Regression: wrap-copilot ate one-line interactive prompts (2026-08-23). + + Copilot CLI prepends ``…`` to every + interactive user turn. The ISO-8601 ``T09:57:59`` matched the grep + ``file:line:`` pattern, so a datetime + one-line prompt classified as + SEARCH_RESULTS (1 match / 2 lines = 50% ≥ 30%) and the SearchCompressor + deleted the prompt line — the model received only the timestamp. + """ + incident = ( + "2026-08-23T09:57:59.792+02:00\n" + "\n" + "Please update the PR desc and check .overlay/ for hints." + ) + assert _try_detect_search(incident) is None + assert detect_content_type(incident).content_type is not ContentType.SEARCH_RESULTS + + +def test_search_detection_requires_two_matching_lines() -> None: + """A single coincidental ``word:digits:`` line must not classify prose.""" + assert _try_detect_search("src/foo.py:12:def foo():") is None + assert ( + _try_detect_search( + "Meeting at 09:30:00 tomorrow.\nBring the reports.\nDo not forget coffee." + ) + is None + ) + # Two genuine grep lines still classify. + two = "src/foo.py:12:def foo():\nsrc/bar.py:34: foo()" + result = _try_detect_search(two) + assert result is not None + assert result.content_type is ContentType.SEARCH_RESULTS + + +def test_search_detection_rejects_tag_like_and_key_value_prefixes() -> None: + """Markup / key=value lines are not file paths even with ``:\\d+:`` inside.""" + assert ( + _try_detect_search('started\nstopped') + is None + ) + assert _try_detect_search("timeout=30:12:retried\ntimeout=31:12:retried") is None + + def test_log_detection_prefers_build_output_patterns() -> None: log_output = "\n".join( [ diff --git a/tests/test_transforms_content_router.py b/tests/test_transforms_content_router.py index 4f76c6386..74ce9b5c9 100644 --- a/tests/test_transforms_content_router.py +++ b/tests/test_transforms_content_router.py @@ -1785,3 +1785,23 @@ def test_detect_content_overrides_html_misroute_for_grep_and_logs( "" ) assert _detect_content(html).content_type is ContentType.HTML + + +def test_datetime_prefixed_user_prompt_survives_router() -> None: + """Regression (2026-08-23): interactive wrap-copilot prompts were deleted. + + Copilot CLI prepends ``…`` to every + interactive user turn; the ISO timestamp matched the grep ``file:line:`` + detector, the one-line prompt classified as SEARCH_RESULTS, and + SearchCompressor kept only the datetime line — the model received no + request and answered "How can I help you today?". The router must never + route this shape to the search line-filter and must keep the prose. + """ + prompt = ( + "2026-08-23T09:57:59.792+02:00\n" + "\n" + "Please update the PR desc and check .overlay/ for hints." + ) + result = ContentRouter().compress(prompt) + assert result.strategy_used is not CompressionStrategy.SEARCH + assert "Please update the PR desc" in result.compressed