fix(transforms): stop folding datetime-prefixed user messages as search results (#3221)

## Description

Interactive `headroom wrap copilot` sessions intermittently lose the
user's message: the model answers "How can I help you today?" to a real
task prompt. Root cause: Copilot CLI prepends
`<current_datetime>…</current_datetime>` to every interactive user turn;
the ISO-8601 timestamp matches the grep `file:line:` detector, so a
datetime + one-line prompt (1 match / 2 non-empty lines = 50% ≥ 30%)
classifies as `SEARCH_RESULTS`, and `SearchCompressor` — which keeps
only detector-matching lines — deletes the prompt before upstream. On
the OpenAI chat streaming path there is no retrieval tool, so the loss
is unrecoverable.

Fix: `_try_detect_search` now (a) requires the pre-colon segment to look
like a file path (no `<`, `>`, `=`), and (b) requires at least two
matching lines, so one coincidental `word:digits:` line can no longer
classify a whole payload. A genuine one-line grep result loses nothing:
all its lines match, so the compressor would have kept it verbatim
anyway.

Closes #3220

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/transforms/content_detector.py`: new
`_is_search_result_line` helper (path-like prefix gate);
`_try_detect_search` gains a two-matching-line absolute floor.
- `tests/test_transforms_content_detection.py`: regression tests —
datetime-prefixed one-liner not search; two-line floor; tag-like /
`key=value` prefixes rejected; genuine grep output still detected.
- `tests/test_transforms_content_router.py`: router-level regression —
the incident payload never routes to SEARCH and the prose survives
`ContentRouter().compress()`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/python -m pytest tests/test_transforms_content_detection.py tests/test_transforms_content_router.py tests/test_mixed_content_sections.py tests/test_text_compressors.py tests/test_transforms_tabular.py -q
135 passed in 21.06s

$ .venv/bin/ruff check headroom/transforms/content_detector.py tests/test_transforms_content_detection.py tests/test_transforms_content_router.py
All checks passed!

$ .venv/bin/mypy headroom/transforms/content_detector.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS 26.5 arm64, Python 3.13, editable source build
0.37.0-dev; upstream `api.githubcopilot.com`, cheapest subscription
model `kimi-k2.7-code`.
- Exact command / steps: standalone copilot-routed proxy
(`OPENAI_TARGET_API_URL=https://api.githubcopilot.com headroom proxy
--port 8899`) + `.overlay/e2e-copilot-content-probe.sh --port 8899
--model kimi-k2.7-code`, which sends the real interactive wire shape
(`<current_datetime>…` + one-line sentinel prompt, streaming) and a
multi-line control.
- Observed result: BEFORE the fix, probe 1 FAIL — model replied "Hello!
I see the current datetime is … How can I assist you today?" with proxy
log `transforms=router:search:0.50` (prompt deleted). AFTER the fix,
both probes PASS — the sentinel echoes verbatim, proving the user
message reached upstream intact.
- Not tested: other harnesses' interactive wrappers (claude/droid/auggie
send different shapes; the detector fix is generic); the mixed-content
section splitter has its own grep pattern (out of scope — its 1-line
"search" sections are kept verbatim, no data loss).

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A (no flag).
- Stable/default behavior changed: content with exactly one
`path:line:`-shaped line no longer classifies as search results (stays
uncompressed instead — safe direction; compression only ever engages on
≥2 matching lines now).
- Kill switch / disable path: N/A.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert; prior behavior restores (with the bug).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — proxy transform change; no UI.

## Additional Notes

Detection-precision tradeoff is documented in code comments: single-line
genuine grep output is no longer folded (no data loss either way — the
compressor keeps all-matching content verbatim). A residual edge (prose
with ≥2 coincidental `x:1:` lines in ≤6 lines) is accepted and
documented in the issue.
This commit is contained in:
Raúl 2026-08-23 20:53:17 +02:00 committed by GitHub
parent 7550efb68f
commit 7784bb1846
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 88 additions and 2 deletions

View file

@ -457,6 +457,23 @@ def _try_detect_html(content: str) -> DetectionResult | None:
)
def _is_search_result_line(line: str) -> bool:
"""True when a line looks like ``path:line:content`` grep output.
The bare ``^[^\\s:]+:\\d+:`` shape also matches ISO-8601 timestamps
(``T09:57:59``) and XML-ish wrappers harnesses prepend to user turns
(Copilot CLI's ``<current_datetime>…`` line), which misroutes prose to
the SearchCompressor and that compressor keeps only matching lines,
deleting the rest. So the pre-colon segment must additionally look like
a file path: no angle brackets and no ``=`` (rules out markup tags and
``key=value:12:`` log lines).
"""
if not _SEARCH_RESULT_PATTERN.match(line):
return False
prefix = line.split(":", 1)[0]
return "<" not in prefix and ">" not in prefix and "=" not in prefix
def _try_detect_search(content: str) -> DetectionResult | None:
"""Try to detect grep/ripgrep search results."""
lines = content.split("\n")[:100] # Check first 100 lines
@ -465,10 +482,16 @@ def _try_detect_search(content: str) -> DetectionResult | None:
matching_lines = 0
for line in lines:
if line.strip() and _SEARCH_RESULT_PATTERN.match(line):
if line.strip() and _is_search_result_line(line):
matching_lines += 1
if matching_lines == 0:
# Absolute floor: a single coincidental `word:digits:` line (a timestamp,
# a URL, a time literal inside prose) must not classify a whole payload as
# search results — the SearchCompressor drops every non-matching line, so
# a false positive is data loss. A genuine one-line grep result loses
# nothing by staying uncompressed: all of its lines match, so the
# compressor would have kept it verbatim anyway.
if matching_lines < 2:
return None
# Calculate confidence based on proportion of matching lines

View file

@ -170,6 +170,49 @@ def test_search_detection_uses_match_ratio() -> None:
assert _try_detect_search("\n\n") is None
def test_search_detection_rejects_datetime_prefixed_user_message() -> None:
"""Regression: wrap-copilot ate one-line interactive prompts (2026-08-23).
Copilot CLI prepends ``<current_datetime></current_datetime>`` to every
interactive user turn. The ISO-8601 ``T09:57:59`` matched the grep
``file:line:`` pattern, so a datetime + one-line prompt classified as
SEARCH_RESULTS (1 match / 2 lines = 50% 30%) and the SearchCompressor
deleted the prompt line the model received only the timestamp.
"""
incident = (
"<current_datetime>2026-08-23T09:57:59.792+02:00</current_datetime>\n"
"\n"
"Please update the PR desc and check .overlay/ for hints."
)
assert _try_detect_search(incident) is None
assert detect_content_type(incident).content_type is not ContentType.SEARCH_RESULTS
def test_search_detection_requires_two_matching_lines() -> None:
"""A single coincidental ``word:digits:`` line must not classify prose."""
assert _try_detect_search("src/foo.py:12:def foo():") is None
assert (
_try_detect_search(
"Meeting at 09:30:00 tomorrow.\nBring the reports.\nDo not forget coffee."
)
is None
)
# Two genuine grep lines still classify.
two = "src/foo.py:12:def foo():\nsrc/bar.py:34: foo()"
result = _try_detect_search(two)
assert result is not None
assert result.content_type is ContentType.SEARCH_RESULTS
def test_search_detection_rejects_tag_like_and_key_value_prefixes() -> None:
"""Markup / key=value lines are not file paths even with ``:\\d+:`` inside."""
assert (
_try_detect_search('<log time="10:00:00">started</log>\n<log time="10:00:01">stopped</log>')
is None
)
assert _try_detect_search("timeout=30:12:retried\ntimeout=31:12:retried") is None
def test_log_detection_prefers_build_output_patterns() -> None:
log_output = "\n".join(
[

View file

@ -1785,3 +1785,23 @@ def test_detect_content_overrides_html_misroute_for_grep_and_logs(
"</section></main></body></html>"
)
assert _detect_content(html).content_type is ContentType.HTML
def test_datetime_prefixed_user_prompt_survives_router() -> None:
"""Regression (2026-08-23): interactive wrap-copilot prompts were deleted.
Copilot CLI prepends ``<current_datetime></current_datetime>`` to every
interactive user turn; the ISO timestamp matched the grep ``file:line:``
detector, the one-line prompt classified as SEARCH_RESULTS, and
SearchCompressor kept only the datetime line the model received no
request and answered "How can I help you today?". The router must never
route this shape to the search line-filter and must keep the prose.
"""
prompt = (
"<current_datetime>2026-08-23T09:57:59.792+02:00</current_datetime>\n"
"\n"
"Please update the PR desc and check .overlay/ for hints."
)
result = ContentRouter().compress(prompt)
assert result.strategy_used is not CompressionStrategy.SEARCH
assert "Please update the PR desc" in result.compressed