mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
6413cc75a2
## Description Fixes #1126. On a Windows (cp1252) locale, `headroom wrap` crashes with `UnicodeDecodeError` the first time it injects guidance into a user instruction file that contains non-ASCII prose (e.g. typographic quotes `“happy places”` or an em-dash). `_inject_rtk_instructions` and `_inject_memory_agents_md` both read the existing file and append/create it with a bare `read_text()` / `open()` / `write_text()`, so the default codec (cp1252, not UTF-8) chokes on the multi-byte characters. This is the same bug class already fixed for the `learn` pipeline (#1202) and earlier for other wrap paths — here it's the instruction-file injectors. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: in `_inject_rtk_instructions` and `_inject_memory_agents_md`, read the existing instruction file as `encoding="utf-8", errors="replace"` and append/create with `encoding="utf-8"`. The read only feeds the marker-existence check and the append doesn't rewrite existing bytes, so replacement can't corrupt the file. - `tests/test_cli/test_wrap_encoding.py`: new regression tests. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_cli/test_wrap_encoding.py tests/test_cli/test_wrap_hintfile_agents.py -q 16 passed $ ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_encoding.py All checks passed! ``` The new tests are **red on the old code, green with the fix**: injecting into a file with a typographic quote plus a stray `0x9d` byte (undefined in cp1252 and invalid UTF-8, so a bare `open()` fails on any locale) — the append and idempotent paths fail before the fix (4 failed) and pass after (6 passed). ## Real Behavior Proof - Environment: Windows 11, Python 3.10, against the real `headroom.cli.wrap` injectors (no live agent launch; the decode failure is at file read time). - Exact command / steps: `write_bytes` an `AGENTS.md` containing `"Be in “happy places” — really.\n"` plus a stray `0x9d` byte, then call `_inject_rtk_instructions(path)` / `_inject_memory_agents_md(path)`. - Observed result: **before** the fix → `UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9d` (and on a real cp1252 locale, the same on the typographic quotes alone); **after** → both return `True`, the marker is present, the pre-existing prose is preserved, and re-running is idempotent. - Not tested: a full end-to-end `headroom wrap copilot` against a live Copilot CLI (verified at the injector level, which is where the decode crash lives). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
62 lines
2.2 KiB
Python
62 lines
2.2 KiB
Python
"""Regression tests for #1126 — `headroom wrap` instruction injection must read
|
|
and write user instruction files as UTF-8, so non-ASCII prose (typographic
|
|
quotes, em-dashes) does not crash on a cp1252 (Windows) locale.
|
|
|
|
The reads use `errors="replace"`, so a stray non-UTF-8 byte (e.g. `0x9d`, which
|
|
fails a bare `open()` on *any* locale) cannot abort the wrap either.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from headroom.cli.wrap import (
|
|
_MEMORY_AGENTS_MARKER,
|
|
_RTK_MARKER,
|
|
_inject_memory_agents_md,
|
|
_inject_rtk_instructions,
|
|
)
|
|
|
|
# (inject_fn, marker) for the two prose injectors that share the bug.
|
|
INJECTORS = [
|
|
pytest.param(_inject_rtk_instructions, _RTK_MARKER, id="rtk"),
|
|
pytest.param(_inject_memory_agents_md, _MEMORY_AGENTS_MARKER, id="memory_agents"),
|
|
]
|
|
|
|
# A typographic quote / em-dash (valid UTF-8) plus a stray byte that is
|
|
# undefined in cp1252 and an invalid UTF-8 start byte.
|
|
_EXISTING = "Be in “happy places” — really.\n".encode() + b"legacy \x9d byte\n"
|
|
|
|
|
|
@pytest.mark.parametrize("inject, marker", INJECTORS)
|
|
def test_inject_appends_into_file_with_non_ascii_and_stray_byte(inject, marker, tmp_path: Path):
|
|
target = tmp_path / "AGENTS.md"
|
|
target.write_bytes(_EXISTING)
|
|
|
|
# Before the fix this raised UnicodeDecodeError reading the existing file.
|
|
assert inject(target) is True
|
|
|
|
text = target.read_text(encoding="utf-8", errors="replace")
|
|
assert marker in text
|
|
# The pre-existing prose is preserved (append, not rewrite).
|
|
assert "happy places" in text
|
|
|
|
|
|
@pytest.mark.parametrize("inject, marker", INJECTORS)
|
|
def test_inject_creates_file_when_absent(inject, marker, tmp_path: Path):
|
|
target = tmp_path / "nested" / "AGENTS.md"
|
|
|
|
assert inject(target) is True
|
|
assert marker in target.read_text(encoding="utf-8")
|
|
|
|
|
|
@pytest.mark.parametrize("inject, marker", INJECTORS)
|
|
def test_inject_is_idempotent(inject, marker, tmp_path: Path):
|
|
target = tmp_path / "AGENTS.md"
|
|
target.write_bytes(_EXISTING)
|
|
|
|
assert inject(target) is True
|
|
assert inject(target) is True # marker already present -> no duplicate / no crash
|
|
assert target.read_text(encoding="utf-8", errors="replace").count(marker) == 1
|