headroom/tests/test_cli/test_wrap_codex.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

1979 lines
77 KiB
Python
Raw Normal View History

fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
"""Tests for `headroom wrap codex` and `headroom unwrap codex`.
These exercise the Codex-specific ``config.toml`` injection and restoration
helpers that route Codex through the Headroom proxy. They are deliberately
end-to-end-ish: the unit tests call the helpers directly against a temp
``$HOME``, and the integration tests invoke the real Click commands the same
way a user would from the shell.
"""
from __future__ import annotations
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
import shutil
fix(codex): retag thread providers so history menu stays whole across the proxy boundary (#1034) ## Description Codex stamps every thread with the `model_provider` it ran under and filters its history/projects menu by the currently active provider set. When Headroom rewrites Codex's config to route through the custom `headroom` provider, threads created through Headroom are tagged `headroom` while native threads keep `openai` — so the two sets never appear in the same menu. The visible symptom: enabling Headroom appears to "lose" the entire native Codex history, and disabling it hides everything created while wrapped. This reconciles the thread tags in Codex's SQLite store alongside the existing config edits, so the menu stays whole across the proxy boundary in both directions: `openai -> headroom` on enable/wrap, `headroom -> openai` on revert/unwrap. Only rows matching the source provider are touched, so third-party providers (e.g. `anthropic`) are left alone. The provider key cannot be unified by config — Codex rejects naming a custom provider `openai` ("reserved built-in provider IDs") — so retagging the store is the only path. A DB-only retag is sufficient: resuming a retagged session still routes completions through the active provider; the rollout `.jsonl` files do not need rewriting. Every operation is best-effort: a missing store, a missing `threads` table, or a corrupt store is logged and skipped, never raised, so install/uninstall and wrap/unwrap never fail on account of the history menu. The store is WAL-mode, so the update succeeds even while Codex is running; the short busy timeout only covers a transient checkpoint lock. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - New `headroom/providers/codex/threads.py`: best-effort retag of Codex thread provider tags across both known stores (`<codex_home>/sqlite/state_5.sqlite` for the GUI and `<codex_home>/state_5.sqlite` for the CLI/TUI). `retag_to_headroom` / `retag_to_native` wrap the directional helper. `codex_home` is passed in by callers (never re-derived from `Path.home()`), so tests stay pointed at a temp dir. - `providers/codex/install.py`: `apply_provider_scope` calls `retag_to_headroom` after writing the provider block; `revert_provider_scope` calls `retag_to_native` after stripping it. - `cli/wrap.py`: `_inject_codex_provider_config` calls `retag_to_headroom`; `unwrap_codex` calls `retag_to_native` once the config restore reports a `restored`/`cleaned`/`removed` status. - Tests: `tests/test_provider_codex_threads.py` (retag direction, threads-table no-op, missing/corrupt store best-effort) and a wrap/unwrap round-trip integration test in `tests/test_cli/test_wrap_codex.py`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --extra dev pytest tests/test_provider_codex_threads.py tests/test_cli/test_wrap_codex.py tests/test_install/test_providers.py -q ... 88 passed, 2 failed # The 2 failures are TestInjectAvoidsDuplicateTopLevelKeys::* — pre-existing on a clean # upstream/main checkout, unrelated to this change: they `import tomllib`, which is # stdlib only on Python 3.11+, and this environment runs Python 3.10.18. $ uv run --extra dev ruff check headroom/cli/wrap.py headroom/providers/codex/threads.py \ headroom/providers/codex/install.py tests/test_cli/test_wrap_codex.py tests/test_provider_codex_threads.py All checks passed! $ uv run --extra dev mypy headroom/providers/codex/threads.py headroom/providers/codex/install.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - Environment: macOS, Codex GUI v148 + Codex CLI, Python 3.10.18. - Exact command / steps: The root cause and fix were confirmed live in the Headroom desktop app, which performs the identical SQLite retag. Connecting Codex to Headroom hid ~140 native (`openai`) threads from the history menu; running `UPDATE threads SET model_provider='headroom' WHERE model_provider='openai'` on the live store (`~/.codex/sqlite/state_5.sqlite`) made the full menu reappear, and resuming a retagged session still routed completions through the active provider. This Python port is a 1:1 of that logic, exercised by the unit + wrap/unwrap integration tests above. - Observed result: full Codex history menu restored across enable/disable; third-party provider rows untouched; the real `~/.codex` stores were snapshotted before/after the test run and were not mutated by the tests. - Not tested: an end-to-end `headroom wrap codex` run against a live Codex GUI in this CI environment (no Codex install here); covered instead by the integration test invoking the real `wrap`/`unwrap` Click commands against a temp `$HOME`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — behavior is in Codex's own history menu; covered by the proof above. ## Additional Notes - Documentation / CHANGELOG: N/A — internal behavior with no user-facing surface beyond the restored menu. - The 2 failing `TestInjectAvoidsDuplicateTopLevelKeys` tests are pre-existing on upstream/main and fail only because this environment runs Python 3.10 (no `tomllib`); they are unrelated to this change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 22:13:21 +02:00
import sqlite3
fix(codex): skip sockets in session home overlay (#2104) ## Description Prevent `headroom wrap codex` from failing when the active `CODEX_HOME` contains a Unix socket. The session overlay copied every entry with `shutil.copytree()`, which raises `shutil.Error` when it reaches Git's `fsmonitor--daemon.ipc` socket. The overlay now skips socket entries while continuing to copy regular Codex state and surface unrelated copy errors. Closes #2103 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Ignore filesystem sockets while seeding the temporary Codex session home. - Add a regression test with a real nested `fsmonitor--daemon.ipc` socket and a regular sibling file. ## Testing - [x] Focused unit tests pass (`pytest tests/test_cli/test_wrap_codex.py -q`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New regression test added - [ ] Manual interactive testing performed ### Test Output ```text Docker, Linux arm64, Python 3.12.12 pytest tests/test_cli/test_wrap_codex.py -q 88 passed in 5.92s ruff check . All checks passed! ruff format --check . 1191 files already formatted mypy headroom --ignore-missing-imports Success: no issues found in 469 source files ``` ## Real Behavior Proof - Environment: isolated Docker container on Linux arm64 with Python 3.12.12 and Rust 1.95.0 - Exact command / steps: bind a real Unix socket at `vendor_imports/skills/.git/fsmonitor--daemon.ipc`, then enter `_codex_session_home_overlay()` through the focused pytest regression - Observed result: the regular sibling file is copied, the socket is omitted, the source socket remains active, and the overlay exits cleanly - Not tested: an interactive Codex launch against the live host `~/.codex` ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused Codex wrapper tests pass with my changes ## Additional Notes The filter is intentionally limited to socket entries. Permission errors and failures involving regular files still propagate from `shutil.copytree()`.
2026-07-13 16:09:32 +02:00
import sys
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
from pathlib import Path
from unittest.mock import patch
import pytest
from click.testing import CliRunner
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
if sys.version_info >= (3, 11):
import tomllib
else: # pragma: no cover - exercised in the Python 3.10 test job
import tomli as tomllib
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
from headroom.cli import wrap as wrap_mod
from headroom.cli.main import main
fix(mcp): register managed installs with a resolvable headroom command (#1386) ## Description Managed Headroom installs can register the MCP server with a bare `headroom mcp serve` command even when the active runtime lives in a venv outside `PATH`. That leaves Claude and Codex with a registration they cannot re-launch reliably, and Claude eventually fails with `Failed to reconnect to headroom: ENOENT`. This PR reuses the existing runtime command resolver when building the shared Headroom MCP spec, so the generated registration follows the active install instead of assuming `headroom` is globally discoverable. It also updates the shared-builder and registrar tests so the proof rows now flow through `build_headroom_spec()` and prove the same resolved command contract on both the Claude CLI path and the Codex TOML path. A follow-up CI fix keeps the Docker init E2E expectation aligned with that same resolver-backed contract. Closes #487 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/mcp_registry/install.py`: build the Headroom MCP server spec from the canonical runtime command resolver instead of hardcoding `headroom mcp serve` - `tests/test_mcp_registry/test_install.py`: cover the shared builder's direct-binary and module-fallback command shapes - `tests/test_mcp_registry/test_claude_registrar.py`: prove the Claude CLI registration forwards the resolved command vector end to end - `tests/test_mcp_registry/test_codex_registrar.py`: prove the Codex registrar writes the same resolved command vector into TOML - `e2e/init/run.py`: derive the Docker init E2E's expected Claude MCP registration argv from `resolve_headroom_command()` so the CI harness follows the same runtime contract - `CHANGELOG.md`: note the managed-install MCP registration fix ## Testing - [x] Unit tests pass (`uv run pytest tests/test_mcp_registry/test_install.py -v`, `uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v`, `uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v`) - [x] Linting passes (`uv run ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) or explain N/A truthfully - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_mcp_registry/test_install.py -v ============================= 12 passed in 0.13s ============================== $ uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v ============================= 24 passed in 0.18s ============================== $ uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v ============================= 25 passed in 0.20s ============================== $ uv run python -c "from e2e.init.run import _expected_headroom_mcp_call; print(_expected_headroom_mcp_call('http://127.0.0.1:9011'))" ['mcp', 'add', 'headroom', '-s', 'user', '-e', 'HEADROOM_PROXY_URL=http://127.0.0.1:9011', '--', '.../headroom', 'mcp', 'serve'] $ uv run ruff check e2e/init/run.py All checks passed! $ uv run ruff format e2e/init/run.py --check 1 file already formatted $ uv run ruff check . All checks passed! $ uv run ruff format . --check 987 files already formatted ``` `uv run mypy headroom` was not run locally; this repo's focused local gate for the touched Python registry path is the targeted pytest set plus Ruff. ## Real Behavior Proof - Environment: managed-install-safe MCP registration path, Python 3.11+, no provider required - Exact command / steps: run the focused MCP registry pytest files, inspect the captured Claude CLI argv and rendered Codex TOML block, and verify the Docker init E2E expectation derives its Claude MCP argv from the same runtime helper - Observed result: the persisted MCP registration uses a resolvable command tied to the active Headroom runtime instead of bare `headroom`, while `HEADROOM_PROXY_URL` handling stays unchanged - Not tested: full live Claude reconnect against a real managed venv, unless that is run during implementation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Scoped to the MCP registration slice in `#487`. The RTK hook rewriting thread from the same issue is intentionally out of scope here. - `@erikpr1994` isolated the managed-install `ENOENT` failure mode in the issue thread and narrowed it to the bare-command MCP registration path. - If existing owned registrations with the old bare-command contract need an in-place upgrade path, that should be handled explicitly in the final diff rather than left implicit.
2026-06-27 00:39:00 -04:00
from headroom.mcp_registry.install import build_headroom_spec
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
def _set_test_home(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
home = str(tmp_path)
monkeypatch.setenv("HOME", home)
monkeypatch.setenv("USERPROFILE", home)
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
monkeypatch.delenv("CODEX_HOME", raising=False)
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
@pytest.fixture
def runner() -> CliRunner:
return CliRunner()
# ---------------------------------------------------------------------------
# Unit tests: helpers operating on ~/.codex/config.toml
# ---------------------------------------------------------------------------
class TestStripCodexHeadroomBlocks:
"""Tests for the regex-based cleanup helper."""
def test_empty_content_returns_empty(self) -> None:
assert wrap_mod._strip_codex_headroom_blocks("") == ""
def test_returns_content_unchanged_when_no_markers(self) -> None:
original = '[profiles.default]\nmodel = "gpt-4o"\n'
cleaned = wrap_mod._strip_codex_headroom_blocks(original)
# Trailing whitespace normalization only — semantic content preserved.
assert 'model = "gpt-4o"' in cleaned
assert "[profiles.default]" in cleaned
def test_removes_complete_headroom_block(self) -> None:
wrapped = (
f"{wrap_mod._CODEX_TOP_LEVEL_MARKER}\n"
'model_provider = "headroom"\n'
"\n"
"[model_providers.headroom]\n"
'base_url = "http://127.0.0.1:8787/v1"\n'
f"{wrap_mod._CODEX_END_MARKER}\n"
)
assert wrap_mod._strip_codex_headroom_blocks(wrapped) == ""
def test_preserves_user_content_around_block(self) -> None:
user_pre = '[profiles.default]\nmodel = "gpt-4o"\n'
user_post = '[mcp_servers.foo]\ncommand = "echo"\n'
wrapped = (
f"{wrap_mod._CODEX_TOP_LEVEL_MARKER}\n"
'model_provider = "headroom"\n'
f"{wrap_mod._CODEX_END_MARKER}\n" + user_pre + "\n"
f"{wrap_mod._CODEX_TOP_LEVEL_MARKER}\n"
"[model_providers.headroom]\n"
'base_url = "http://127.0.0.1:8787/v1"\n'
f"{wrap_mod._CODEX_END_MARKER}\n" + user_post
)
cleaned = wrap_mod._strip_codex_headroom_blocks(wrapped)
assert wrap_mod._CODEX_TOP_LEVEL_MARKER not in cleaned
assert wrap_mod._CODEX_END_MARKER not in cleaned
assert 'model = "gpt-4o"' in cleaned
assert "[mcp_servers.foo]" in cleaned
def test_removes_stray_top_level_model_provider_line(self) -> None:
# Old wrap versions left `model_provider = "headroom"` outside markers.
content = 'foo = 1\nmodel_provider = "headroom"\nbar = 2\n'
cleaned = wrap_mod._strip_codex_headroom_blocks(content, remove_mcp=True)
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert 'model_provider = "headroom"' not in cleaned
assert "foo = 1" in cleaned
assert "bar = 2" in cleaned
def test_removes_codex_mcp_blocks(self) -> None:
content = (
'[profiles.default]\nmodel = "gpt-4o"\n\n'
f"{wrap_mod._CODEX_MCP_MARKER}\n"
"[mcp_servers.headroom]\n"
'command = "headroom"\n'
f"{wrap_mod._CODEX_MCP_END}\n\n"
2026-05-09 22:57:35 -07:00
"# --- Headroom MCP server: serena ---\n"
"[mcp_servers.serena]\n"
'command = "uvx"\n'
"# --- end Headroom MCP server: serena ---\n\n"
f"{wrap_mod._MEMORY_MCP_MARKER}\n"
"[mcp_servers.headroom_memory]\n"
'command = "python"\n'
f"{wrap_mod._MEMORY_MCP_END}\n"
)
cleaned = wrap_mod._strip_codex_headroom_blocks(content, remove_mcp=True)
assert "[mcp_servers.headroom]" not in cleaned
2026-05-09 22:57:35 -07:00
assert "[mcp_servers.serena]" not in cleaned
assert "[mcp_servers.headroom_memory]" not in cleaned
assert 'model = "gpt-4o"' in cleaned
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
def test_preserves_named_mcp_blocks_when_remove_named_mcp_false(self) -> None:
content = (
"# --- Headroom MCP server: serena ---\n"
"[mcp_servers.serena]\n"
'command = "uvx"\n'
"# --- end Headroom MCP server: serena ---\n\n"
f"{wrap_mod._MEMORY_MCP_MARKER}\n"
"[mcp_servers.headroom_memory]\n"
'command = "python"\n'
f"{wrap_mod._MEMORY_MCP_END}\n"
)
cleaned = wrap_mod._strip_codex_headroom_blocks(
content, remove_mcp=True, remove_named_mcp=False
)
assert "[mcp_servers.serena]" in cleaned
assert "[mcp_servers.headroom_memory]" not in cleaned
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
class TestSnapshotCodexConfig:
"""Tests for ``_snapshot_codex_config_if_unwrapped``."""
def test_creates_backup_on_first_call(self, tmp_path: Path) -> None:
config_file = tmp_path / "config.toml"
backup_file = tmp_path / "config.toml.headroom-backup"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text('model = "gpt-4o"\n', encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
wrap_mod._snapshot_codex_config_if_unwrapped(config_file, backup_file)
assert backup_file.exists()
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert backup_file.read_text(encoding="utf-8") == 'model = "gpt-4o"\n'
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
def test_does_not_overwrite_existing_backup(self, tmp_path: Path) -> None:
config_file = tmp_path / "config.toml"
backup_file = tmp_path / "config.toml.headroom-backup"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text("second-wrap content\n", encoding="utf-8")
backup_file.write_text("original-pre-wrap content\n", encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
wrap_mod._snapshot_codex_config_if_unwrapped(config_file, backup_file)
# Backup must still contain the *original* pre-wrap content.
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert backup_file.read_text(encoding="utf-8") == "original-pre-wrap content\n"
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
def test_no_backup_when_config_missing(self, tmp_path: Path) -> None:
config_file = tmp_path / "config.toml"
backup_file = tmp_path / "config.toml.headroom-backup"
wrap_mod._snapshot_codex_config_if_unwrapped(config_file, backup_file)
assert not backup_file.exists()
def test_no_backup_when_config_already_wrapped(self, tmp_path: Path) -> None:
config_file = tmp_path / "config.toml"
backup_file = tmp_path / "config.toml.headroom-backup"
config_file.write_text(
f"{wrap_mod._CODEX_TOP_LEVEL_MARKER}\n"
'model_provider = "headroom"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
f"{wrap_mod._CODEX_END_MARKER}\n",
encoding="utf-8",
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
)
wrap_mod._snapshot_codex_config_if_unwrapped(config_file, backup_file)
# Pre-wrap snapshot must never snapshot an already-wrapped file.
assert not backup_file.exists()
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
def test_no_backup_when_config_already_contains_memory_mcp_block(self, tmp_path: Path) -> None:
config_file = tmp_path / "config.toml"
backup_file = tmp_path / "config.toml.headroom-backup"
config_file.write_text(
f"{wrap_mod._MEMORY_MCP_MARKER}\n"
"[mcp_servers.headroom_memory]\n"
'command = "python"\n'
'args = ["-m", "headroom.memory.mcp_server", "--user", "codex-user"]\n'
f"{wrap_mod._MEMORY_MCP_END}\n"
)
wrap_mod._snapshot_codex_config_if_unwrapped(config_file, backup_file)
assert not backup_file.exists()
def test_backup_when_config_contains_named_mcp_marker(self, tmp_path: Path) -> None:
config_file = tmp_path / "config.toml"
backup_file = tmp_path / "config.toml.headroom-backup"
original = (
"# --- Headroom MCP server: headroom ---\n"
"[mcp_servers.headroom]\n"
'command = "headroom"\n'
"# --- end Headroom MCP server: headroom ---\n"
)
config_file.write_text(original)
wrap_mod._snapshot_codex_config_if_unwrapped(config_file, backup_file)
assert backup_file.exists()
assert backup_file.read_text() == original
class TestCodexMemoryMcpConfig:
"""Tests for the persisted Codex memory MCP block."""
def test_inject_omits_db_and_replaces_existing_memory_block(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_file = tmp_path / ".codex" / "config.toml"
config_file.parent.mkdir(parents=True)
config_file.write_text(
'[profiles.default]\nmodel = "gpt-4o"\n\n'
f"{wrap_mod._MEMORY_MCP_MARKER}\n"
"[mcp_servers.headroom_memory]\n"
'command = "python"\n'
'args = ["-m", "headroom.memory.mcp_server", "--db", "/tmp/project-a/.headroom/memory.db", "--user", "old-user"]\n'
f"{wrap_mod._MEMORY_MCP_END}\n"
)
wrap_mod._inject_memory_mcp_config("codex-user")
content = config_file.read_text()
assert content.count(wrap_mod._MEMORY_MCP_MARKER) == 1
assert "[mcp_servers.headroom_memory]" in content
assert '"--user", "codex-user"' in content
assert "--db" not in content
assert 'model = "gpt-4o"' in content
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
class TestInjectAndRestoreRoundTrip:
"""End-to-end wrap → unwrap cycle operating directly on a temp $HOME."""
def test_wrap_unwrap_restores_empty_state(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_file = tmp_path / ".codex" / "config.toml"
wrap_mod._inject_codex_provider_config(8787)
assert config_file.exists()
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert 'model_provider = "headroom"' in config_file.read_text(encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
status, _ = wrap_mod._restore_codex_provider_config()
# No prior config existed → the injected file is fully removed.
assert status == "removed"
assert not config_file.exists()
assert not (tmp_path / ".codex" / "config.toml.headroom-backup").exists()
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
def test_wrap_unwrap_respects_codex_home(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
codex_home = tmp_path / "custom-codex-home"
monkeypatch.setenv("CODEX_HOME", str(codex_home))
config_file = codex_home / "config.toml"
wrap_mod._inject_codex_provider_config(8787)
assert config_file.exists()
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert 'model_provider = "headroom"' in config_file.read_text(encoding="utf-8")
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
assert not (tmp_path / ".codex" / "config.toml").exists()
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "removed"
assert not config_file.exists()
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
def test_wrap_unwrap_restores_prior_model_provider(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
original = (
'model_provider = "openai"\n'
"\n"
"[model_providers.openai]\n"
'name = "OpenAI"\n'
'base_url = "https://api.openai.com/v1"\n'
)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text(original, encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
wrapped = config_file.read_text(encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert 'model_provider = "headroom"' in wrapped
assert "[model_providers.headroom]" in wrapped
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "restored"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert config_file.read_text(encoding="utf-8") == original
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert not (config_dir / "config.toml.headroom-backup").exists()
def test_wrap_is_idempotent(self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
original = '[profiles.default]\nmodel = "gpt-4o"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text(original, encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
wrap_mod._inject_codex_provider_config(8787)
wrap_mod._inject_codex_provider_config(8787)
wrap_mod._inject_codex_provider_config(9999) # port change
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = config_file.read_text(encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
# Exactly two Headroom blocks — a top-level-key block and the
# provider-table block. Re-wrapping must not duplicate them.
assert content.count(wrap_mod._CODEX_TOP_LEVEL_MARKER) == 2
assert content.count(wrap_mod._CODEX_END_MARKER) == 2
# Latest port is honoured in both keys.
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert 'base_url = "http://127.0.0.1:9999/v1"' in content
assert 'openai_base_url = "http://127.0.0.1:9999/v1"' in content
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert 'base_url = "http://127.0.0.1:8787/v1"' not in content
assert 'openai_base_url = "http://127.0.0.1:8787/v1"' not in content
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
# User's original content is preserved.
assert 'model = "gpt-4o"' in content
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "restored"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert config_file.read_text(encoding="utf-8") == original
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
def test_unwrap_is_noop_when_never_wrapped(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "noop"
def test_unwrap_cleans_block_without_backup(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Handles crash-case where wrap injected but backup was wiped."""
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
user_content = '[profiles.default]\nmodel = "gpt-4o"\n'
config_file.write_text(
user_content + f"{wrap_mod._CODEX_TOP_LEVEL_MARKER}\n"
'model_provider = "headroom"\n\n'
"[model_providers.headroom]\n"
'base_url = "http://127.0.0.1:8787/v1"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
f"{wrap_mod._CODEX_END_MARKER}\n",
encoding="utf-8",
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
)
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "cleaned"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
cleaned = config_file.read_text(encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert wrap_mod._CODEX_TOP_LEVEL_MARKER not in cleaned
assert wrap_mod._CODEX_END_MARKER not in cleaned
assert 'model_provider = "headroom"' not in cleaned
assert 'model = "gpt-4o"' in cleaned
def test_unwrap_without_backup_removes_provider_and_mcp_blocks(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
config_file.write_text(
'[profiles.default]\nmodel = "gpt-4o"\n\n'
f"{wrap_mod._CODEX_TOP_LEVEL_MARKER}\n"
'model_provider = "headroom"\n'
f"{wrap_mod._CODEX_END_MARKER}\n\n"
f"{wrap_mod._CODEX_MCP_MARKER}\n"
"[mcp_servers.headroom]\n"
'command = "headroom"\n'
f"{wrap_mod._CODEX_MCP_END}\n\n"
f"{wrap_mod._MEMORY_MCP_MARKER}\n"
"[mcp_servers.headroom_memory]\n"
'command = "python"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
f"{wrap_mod._MEMORY_MCP_END}\n",
encoding="utf-8",
)
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "cleaned"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
cleaned = config_file.read_text(encoding="utf-8")
assert 'model = "gpt-4o"' in cleaned
assert "headroom" not in cleaned
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
def test_memory_only_wrap_restores_preexisting_named_mcp_block(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
original = (
"# --- Headroom MCP server: headroom ---\n"
"[mcp_servers.headroom]\n"
'command = "headroom"\n'
"# --- end Headroom MCP server: headroom ---\n"
)
config_file.write_text(original)
wrap_mod._inject_memory_mcp_config("codex-user")
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "restored"
assert config_file.read_text() == original
def test_memory_only_wrap_without_backup_preserves_named_mcp_block(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
backup_file = config_dir / "config.toml.headroom-backup"
original = (
"# --- Headroom MCP server: headroom ---\n"
"[mcp_servers.headroom]\n"
'command = "headroom"\n'
"# --- end Headroom MCP server: headroom ---\n"
)
config_file.write_text(original)
wrap_mod._inject_memory_mcp_config("codex-user")
backup_file.unlink()
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "cleaned"
assert config_file.read_text() == original
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
def test_unwrap_handles_malformed_prior_config(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Unwrap preserves backup content verbatim — TOML validity isn't required."""
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
malformed = 'this is not valid toml ][ "" \x00\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text(malformed, encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
wrap_mod._inject_codex_provider_config(8787)
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "restored"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert config_file.read_text(encoding="utf-8") == malformed
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
fix(wrap): remove rtk instructions from Codex AGENTS.md on unwrap (#1604) ## Description `headroom wrap codex` injects Headroom's marker-fenced rtk instruction block into the Codex **global** `AGENTS.md` (`_codex_home_dir() / "AGENTS.md"`), so Codex voluntarily prefixes shell commands with `rtk`. But `headroom unwrap codex` only restored `config.toml` and cleaned up the MCP/Serena servers — it never removed that `AGENTS.md` block. The result: after unwrapping, a plain `codex` launch still inherits Headroom's behavior and keeps trying to run `rtk`. If the managed rtk binary directory is no longer on `PATH`, commands fail outright: ```text rtk : The term 'rtk' is not recognized as the name of a cmdlet, function, script file, or operable program. Conversation interrupted ``` `unwrap copilot` already calls `_remove_rtk_instructions(...)`; Codex was simply missing the same cleanup step. Closes #1421 ## Fix Call the existing `_remove_rtk_instructions` helper on the Codex global `AGENTS.md` inside `unwrap_codex`, right after the MCP-server cleanup: ```python if _remove_rtk_instructions(_codex_home_dir() / "AGENTS.md"): click.echo(" Removed Headroom rtk instructions from Codex AGENTS.md.") ``` The helper strips only the marker-fenced block and rewrites the rest of the file (deleting it only if nothing else remains), so user-authored `AGENTS.md` content is preserved. The call is unconditional and best-effort, matching the existing MCP-server cleanup in the same function. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/cli/wrap.py`: `unwrap_codex` now removes the marker-fenced rtk block from the Codex global `AGENTS.md` via `_remove_rtk_instructions`, with a status echo. - `tests/test_cli/test_wrap_codex.py`: regression tests — block removed on unwrap, surrounding user content preserved, and a no-op when `AGENTS.md` is absent. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output Before the fix the two removal tests fail (the no-AGENTS.md safety test passes either way); after the fix the whole file is green: ```text # before the fix (wrap.py reverted, tests kept) FAILED tests/test_cli/test_wrap_codex.py::...::test_unwrap_removes_rtk_block_from_global_agents FAILED tests/test_cli/test_wrap_codex.py::...::test_unwrap_preserves_user_content_in_global_agents ================= 2 failed, 1 passed, 66 deselected in 1.00s ================== # after the fix tests\test_cli\test_wrap_codex.py ...................................... ............................... ============================= 69 passed in 7.45s ============================== ``` ```text $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom built from this branch (`uv sync --extra dev`). - Exact command / steps: set `CODEX_HOME` to a temp dir, wrote a user `AGENTS.md`, injected the rtk block with the same helper `wrap codex` uses, then ran the real `unwrap codex` command (`unwrap_codex.callback(port=8787, no_stop_proxy=True)`) and re-read the file. No mocking of the code under test. - Observed result: the command printed `Removed Headroom rtk instructions from Codex AGENTS.md.`, the rtk marker is gone, and the user's own content survived: ```text === AGENTS.md BEFORE unwrap === # My rules Always write tests. <!-- headroom:rtk-instructions --> # RTK (Rust Token Killer) - Token-Optimized Commands ... <!-- /headroom:rtk-instructions --> rtk marker present before: True --- running: headroom unwrap codex --no-stop-proxy --- Removed Headroom rtk instructions from Codex AGENTS.md. ✓ Codex is no longer routed through the Headroom proxy. === AGENTS.md AFTER unwrap === # My rules Always write tests. rtk marker present after: False user content preserved: True ``` - Not tested: did not run a full real `codex` binary session end-to-end (not installed in this environment); the global-`AGENTS.md` state is the durable thing the bug was about, and it's exercised here for real. Did not run the full `mypy headroom` pass (one-line cleanup call, no new types). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Single logical change, no new dependencies. Reuses the existing `_remove_rtk_instructions` helper, so there's no new removal logic to maintain. - @chopratejas this mirrors the `unwrap copilot` cleanup; flagging you since you've been triaging the wrap/unwrap issues. Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-07-03 10:25:41 +05:30
def test_unwrap_is_safe_when_no_global_agents(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""No Codex AGENTS.md → unwrap is a clean no-op, not a crash."""
_set_test_home(monkeypatch, tmp_path)
wrap_mod.unwrap_codex.callback(port=8787, no_stop_proxy=True)
assert not (tmp_path / ".codex" / "AGENTS.md").exists()
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
fix(codex): retag thread providers so history menu stays whole across the proxy boundary (#1034) ## Description Codex stamps every thread with the `model_provider` it ran under and filters its history/projects menu by the currently active provider set. When Headroom rewrites Codex's config to route through the custom `headroom` provider, threads created through Headroom are tagged `headroom` while native threads keep `openai` — so the two sets never appear in the same menu. The visible symptom: enabling Headroom appears to "lose" the entire native Codex history, and disabling it hides everything created while wrapped. This reconciles the thread tags in Codex's SQLite store alongside the existing config edits, so the menu stays whole across the proxy boundary in both directions: `openai -> headroom` on enable/wrap, `headroom -> openai` on revert/unwrap. Only rows matching the source provider are touched, so third-party providers (e.g. `anthropic`) are left alone. The provider key cannot be unified by config — Codex rejects naming a custom provider `openai` ("reserved built-in provider IDs") — so retagging the store is the only path. A DB-only retag is sufficient: resuming a retagged session still routes completions through the active provider; the rollout `.jsonl` files do not need rewriting. Every operation is best-effort: a missing store, a missing `threads` table, or a corrupt store is logged and skipped, never raised, so install/uninstall and wrap/unwrap never fail on account of the history menu. The store is WAL-mode, so the update succeeds even while Codex is running; the short busy timeout only covers a transient checkpoint lock. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - New `headroom/providers/codex/threads.py`: best-effort retag of Codex thread provider tags across both known stores (`<codex_home>/sqlite/state_5.sqlite` for the GUI and `<codex_home>/state_5.sqlite` for the CLI/TUI). `retag_to_headroom` / `retag_to_native` wrap the directional helper. `codex_home` is passed in by callers (never re-derived from `Path.home()`), so tests stay pointed at a temp dir. - `providers/codex/install.py`: `apply_provider_scope` calls `retag_to_headroom` after writing the provider block; `revert_provider_scope` calls `retag_to_native` after stripping it. - `cli/wrap.py`: `_inject_codex_provider_config` calls `retag_to_headroom`; `unwrap_codex` calls `retag_to_native` once the config restore reports a `restored`/`cleaned`/`removed` status. - Tests: `tests/test_provider_codex_threads.py` (retag direction, threads-table no-op, missing/corrupt store best-effort) and a wrap/unwrap round-trip integration test in `tests/test_cli/test_wrap_codex.py`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --extra dev pytest tests/test_provider_codex_threads.py tests/test_cli/test_wrap_codex.py tests/test_install/test_providers.py -q ... 88 passed, 2 failed # The 2 failures are TestInjectAvoidsDuplicateTopLevelKeys::* — pre-existing on a clean # upstream/main checkout, unrelated to this change: they `import tomllib`, which is # stdlib only on Python 3.11+, and this environment runs Python 3.10.18. $ uv run --extra dev ruff check headroom/cli/wrap.py headroom/providers/codex/threads.py \ headroom/providers/codex/install.py tests/test_cli/test_wrap_codex.py tests/test_provider_codex_threads.py All checks passed! $ uv run --extra dev mypy headroom/providers/codex/threads.py headroom/providers/codex/install.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - Environment: macOS, Codex GUI v148 + Codex CLI, Python 3.10.18. - Exact command / steps: The root cause and fix were confirmed live in the Headroom desktop app, which performs the identical SQLite retag. Connecting Codex to Headroom hid ~140 native (`openai`) threads from the history menu; running `UPDATE threads SET model_provider='headroom' WHERE model_provider='openai'` on the live store (`~/.codex/sqlite/state_5.sqlite`) made the full menu reappear, and resuming a retagged session still routed completions through the active provider. This Python port is a 1:1 of that logic, exercised by the unit + wrap/unwrap integration tests above. - Observed result: full Codex history menu restored across enable/disable; third-party provider rows untouched; the real `~/.codex` stores were snapshotted before/after the test run and were not mutated by the tests. - Not tested: an end-to-end `headroom wrap codex` run against a live Codex GUI in this CI environment (no Codex install here); covered instead by the integration test invoking the real `wrap`/`unwrap` Click commands against a temp `$HOME`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — behavior is in Codex's own history menu; covered by the proof above. ## Additional Notes - Documentation / CHANGELOG: N/A — internal behavior with no user-facing surface beyond the restored menu. - The 2 failing `TestInjectAvoidsDuplicateTopLevelKeys` tests are pre-existing on upstream/main and fail only because this environment runs Python 3.10 (no `tomllib`); they are unrelated to this change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 22:13:21 +02:00
# ---------------------------------------------------------------------------
# Thread retag: wrap pulls native threads into the headroom menu, unwrap hands
# them back, so the Codex history list stays whole across the proxy boundary.
# ---------------------------------------------------------------------------
class TestWrapRetagsThreadProviders:
"""``wrap codex`` retags ``openai`` threads to ``headroom`` and back."""
@staticmethod
def _seed_threads(db: Path, rows: list[tuple[str, str]]) -> None:
db.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(str(db))
try:
conn.execute("CREATE TABLE threads (id TEXT PRIMARY KEY, model_provider TEXT NOT NULL)")
conn.executemany("INSERT INTO threads (id, model_provider) VALUES (?, ?)", rows)
conn.commit()
finally:
conn.close()
@staticmethod
def _count(db: Path, provider: str) -> int:
conn = sqlite3.connect(str(db))
try:
(n,) = conn.execute(
"SELECT COUNT(*) FROM threads WHERE model_provider = ?", (provider,)
).fetchone()
return n
finally:
conn.close()
def test_wrap_unwrap_round_trips_thread_providers(
self, runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
gui_db = tmp_path / ".codex" / "sqlite" / "state_5.sqlite"
cli_db = tmp_path / ".codex" / "state_5.sqlite"
self._seed_threads(gui_db, [("a", "openai"), ("b", "headroom"), ("c", "anthropic")])
self._seed_threads(cli_db, [("d", "openai")])
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
wrap_result = runner.invoke(main, ["wrap", "codex", "--prepare-only", "--port", "8787"])
fix(codex): retag thread providers so history menu stays whole across the proxy boundary (#1034) ## Description Codex stamps every thread with the `model_provider` it ran under and filters its history/projects menu by the currently active provider set. When Headroom rewrites Codex's config to route through the custom `headroom` provider, threads created through Headroom are tagged `headroom` while native threads keep `openai` — so the two sets never appear in the same menu. The visible symptom: enabling Headroom appears to "lose" the entire native Codex history, and disabling it hides everything created while wrapped. This reconciles the thread tags in Codex's SQLite store alongside the existing config edits, so the menu stays whole across the proxy boundary in both directions: `openai -> headroom` on enable/wrap, `headroom -> openai` on revert/unwrap. Only rows matching the source provider are touched, so third-party providers (e.g. `anthropic`) are left alone. The provider key cannot be unified by config — Codex rejects naming a custom provider `openai` ("reserved built-in provider IDs") — so retagging the store is the only path. A DB-only retag is sufficient: resuming a retagged session still routes completions through the active provider; the rollout `.jsonl` files do not need rewriting. Every operation is best-effort: a missing store, a missing `threads` table, or a corrupt store is logged and skipped, never raised, so install/uninstall and wrap/unwrap never fail on account of the history menu. The store is WAL-mode, so the update succeeds even while Codex is running; the short busy timeout only covers a transient checkpoint lock. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - New `headroom/providers/codex/threads.py`: best-effort retag of Codex thread provider tags across both known stores (`<codex_home>/sqlite/state_5.sqlite` for the GUI and `<codex_home>/state_5.sqlite` for the CLI/TUI). `retag_to_headroom` / `retag_to_native` wrap the directional helper. `codex_home` is passed in by callers (never re-derived from `Path.home()`), so tests stay pointed at a temp dir. - `providers/codex/install.py`: `apply_provider_scope` calls `retag_to_headroom` after writing the provider block; `revert_provider_scope` calls `retag_to_native` after stripping it. - `cli/wrap.py`: `_inject_codex_provider_config` calls `retag_to_headroom`; `unwrap_codex` calls `retag_to_native` once the config restore reports a `restored`/`cleaned`/`removed` status. - Tests: `tests/test_provider_codex_threads.py` (retag direction, threads-table no-op, missing/corrupt store best-effort) and a wrap/unwrap round-trip integration test in `tests/test_cli/test_wrap_codex.py`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --extra dev pytest tests/test_provider_codex_threads.py tests/test_cli/test_wrap_codex.py tests/test_install/test_providers.py -q ... 88 passed, 2 failed # The 2 failures are TestInjectAvoidsDuplicateTopLevelKeys::* — pre-existing on a clean # upstream/main checkout, unrelated to this change: they `import tomllib`, which is # stdlib only on Python 3.11+, and this environment runs Python 3.10.18. $ uv run --extra dev ruff check headroom/cli/wrap.py headroom/providers/codex/threads.py \ headroom/providers/codex/install.py tests/test_cli/test_wrap_codex.py tests/test_provider_codex_threads.py All checks passed! $ uv run --extra dev mypy headroom/providers/codex/threads.py headroom/providers/codex/install.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - Environment: macOS, Codex GUI v148 + Codex CLI, Python 3.10.18. - Exact command / steps: The root cause and fix were confirmed live in the Headroom desktop app, which performs the identical SQLite retag. Connecting Codex to Headroom hid ~140 native (`openai`) threads from the history menu; running `UPDATE threads SET model_provider='headroom' WHERE model_provider='openai'` on the live store (`~/.codex/sqlite/state_5.sqlite`) made the full menu reappear, and resuming a retagged session still routed completions through the active provider. This Python port is a 1:1 of that logic, exercised by the unit + wrap/unwrap integration tests above. - Observed result: full Codex history menu restored across enable/disable; third-party provider rows untouched; the real `~/.codex` stores were snapshotted before/after the test run and were not mutated by the tests. - Not tested: an end-to-end `headroom wrap codex` run against a live Codex GUI in this CI environment (no Codex install here); covered instead by the integration test invoking the real `wrap`/`unwrap` Click commands against a temp `$HOME`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — behavior is in Codex's own history menu; covered by the proof above. ## Additional Notes - Documentation / CHANGELOG: N/A — internal behavior with no user-facing surface beyond the restored menu. - The 2 failing `TestInjectAvoidsDuplicateTopLevelKeys` tests are pre-existing on upstream/main and fail only because this environment runs Python 3.10 (no `tomllib`); they are unrelated to this change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 22:13:21 +02:00
assert wrap_result.exit_code == 0, wrap_result.output
# Native threads are now visible under the headroom provider menu;
# third-party providers are left untouched.
assert self._count(gui_db, "headroom") == 2
assert self._count(gui_db, "openai") == 0
assert self._count(gui_db, "anthropic") == 1
assert self._count(cli_db, "headroom") == 1
with patch("headroom.cli.wrap._stop_local_proxy_for_unwrap", return_value="stopped"):
unwrap_result = runner.invoke(main, ["unwrap", "codex", "--port", "8787"])
assert unwrap_result.exit_code == 0, unwrap_result.output
# Back to native so the unproxied Codex menu is whole again.
assert self._count(gui_db, "openai") == 2
assert self._count(gui_db, "headroom") == 0
assert self._count(gui_db, "anthropic") == 1
assert self._count(cli_db, "openai") == 1
# ---------------------------------------------------------------------------
# Subscription routing: openai_base_url intercepts ChatGPT plan traffic
# ---------------------------------------------------------------------------
class TestSubscriptionRouting:
"""Codex subscription (ChatGPT plan) bypasses OPENAI_BASE_URL and the
custom model_provider; it uses the built-in ``openai`` provider whose
base_url defaults to ``https://chatgpt.com/backend-api/codex``.
Setting ``openai_base_url`` overrides that default for all auth modes."""
def test_inject_writes_openai_base_url(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = (tmp_path / ".codex" / "config.toml").read_text(encoding="utf-8")
assert 'openai_base_url = "http://127.0.0.1:8787/v1"' in content
fix(codex): poll /wham/usage for subscription limits (handshake no longer sends x-codex-* headers) (#924) ## Description Codex's subscription usage window (primary/secondary rate-limit gauges) stopped populating for ChatGPT-OAuth sessions. This PR restores it by polling Codex's dedicated usage endpoint instead of relying on response headers that are no longer sent. ### Why the previous approach no longer works The existing code populates `CodexRateLimitState` from `x-codex-*` rate-limit headers captured on the `/v1/responses` WebSocket handshake (`update_from_headers` at WS accept). That worked when OpenAI returned `x-codex-primary-used-percent`, `x-codex-primary-window-minutes`, etc. on the handshake response. OpenAI has since stopped sending those headers on the ChatGPT WebSocket handshake. I confirmed this by faithfully replaying a real Plus-account handshake (both `prewarm` and regular `request_kind`): no `x-codex-*` headers come back on either. This matches OpenAI's own move to a dedicated usage endpoint (`GET /backend-api/codex/usage` in CodexApi mode) and reports such as openai/codex#14728. So `update_from_headers` now runs on every accept but finds nothing to parse, and the window silently stays empty. The headers aren't coming back, so there is nothing to fix in the parsing path. The data now lives behind a request we have to make ourselves. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `subscription/codex_rate_limits.py`: - `parse_codex_usage_payload()` / `update_from_usage_payload()` — map the `GET /backend-api/wham/usage` JSON (`rate_limit.primary_window` / `secondary_window` with `used_percent`, `limit_window_seconds`, `reset_at`; `credits`; `rate_limit_reached_type`) into the existing `CodexRateLimitState`. `limit_window_seconds` is converted to window-minutes with the same round-up codex-rs uses (`(secs + 59) // 60`). - `maybe_schedule_usage_poll()` — fire-and-forget, throttled to one request per 60s, scoped to ChatGPT sessions (requires both a Bearer token and `ChatGPT-Account-Id`; API-key traffic is skipped). Uses an in-flight guard so concurrent accepts don't stack polls. Endpoint is overridable via `HEADROOM_CODEX_USAGE_URL`. - `proxy/handlers/openai.py`: - At the Codex WS accept site, after the now-usually-empty `update_from_headers` block, schedule the usage poll. Wrapped in `contextlib.suppress` and fully non-blocking so it can never delay or fail the WebSocket accept. The old header-capture path is intentionally left in place as a no-cost fallback in case OpenAI restores the headers. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed (live `/wham/usage` replay against a Plus account returned HTTP 200 with the expected schema; payload fixture in tests mirrors that real shape) ## Test Output ``` $ uv run pytest tests/test_codex_rate_limits.py -q ........................................ [100%] 41 passed in 0.16s $ uv run ruff check headroom/subscription/codex_rate_limits.py headroom/proxy/handlers/openai.py tests/test_codex_rate_limits.py All checks passed! $ uv run mypy headroom/subscription/codex_rate_limits.py Success: no issues found in 1 source file ``` ## Additional Notes - New tests cover: full-payload mapping, window-minutes round-up, credits balance kept only when `has_credits`, promo object vs string, empty payload returns `None`, missing `used_percent` skipped, header-gating (requires Bearer + account-id), poll throttling, and no-event-loop safety. - Scoping to `ChatGPT-Account-Id` keeps the poll off API-key traffic, and the 60s throttle plus in-flight guard bound it to at most one lightweight GET per minute per running proxy. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 00:03:14 +02:00
def test_inject_emits_requires_openai_auth_for_chatgpt(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
(config_dir / "auth.json").write_text('{"auth_mode": "chatgpt"}', encoding="utf-8")
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert "requires_openai_auth = true" in (config_dir / "config.toml").read_text(
encoding="utf-8"
)
fix(codex): poll /wham/usage for subscription limits (handshake no longer sends x-codex-* headers) (#924) ## Description Codex's subscription usage window (primary/secondary rate-limit gauges) stopped populating for ChatGPT-OAuth sessions. This PR restores it by polling Codex's dedicated usage endpoint instead of relying on response headers that are no longer sent. ### Why the previous approach no longer works The existing code populates `CodexRateLimitState` from `x-codex-*` rate-limit headers captured on the `/v1/responses` WebSocket handshake (`update_from_headers` at WS accept). That worked when OpenAI returned `x-codex-primary-used-percent`, `x-codex-primary-window-minutes`, etc. on the handshake response. OpenAI has since stopped sending those headers on the ChatGPT WebSocket handshake. I confirmed this by faithfully replaying a real Plus-account handshake (both `prewarm` and regular `request_kind`): no `x-codex-*` headers come back on either. This matches OpenAI's own move to a dedicated usage endpoint (`GET /backend-api/codex/usage` in CodexApi mode) and reports such as openai/codex#14728. So `update_from_headers` now runs on every accept but finds nothing to parse, and the window silently stays empty. The headers aren't coming back, so there is nothing to fix in the parsing path. The data now lives behind a request we have to make ourselves. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `subscription/codex_rate_limits.py`: - `parse_codex_usage_payload()` / `update_from_usage_payload()` — map the `GET /backend-api/wham/usage` JSON (`rate_limit.primary_window` / `secondary_window` with `used_percent`, `limit_window_seconds`, `reset_at`; `credits`; `rate_limit_reached_type`) into the existing `CodexRateLimitState`. `limit_window_seconds` is converted to window-minutes with the same round-up codex-rs uses (`(secs + 59) // 60`). - `maybe_schedule_usage_poll()` — fire-and-forget, throttled to one request per 60s, scoped to ChatGPT sessions (requires both a Bearer token and `ChatGPT-Account-Id`; API-key traffic is skipped). Uses an in-flight guard so concurrent accepts don't stack polls. Endpoint is overridable via `HEADROOM_CODEX_USAGE_URL`. - `proxy/handlers/openai.py`: - At the Codex WS accept site, after the now-usually-empty `update_from_headers` block, schedule the usage poll. Wrapped in `contextlib.suppress` and fully non-blocking so it can never delay or fail the WebSocket accept. The old header-capture path is intentionally left in place as a no-cost fallback in case OpenAI restores the headers. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed (live `/wham/usage` replay against a Plus account returned HTTP 200 with the expected schema; payload fixture in tests mirrors that real shape) ## Test Output ``` $ uv run pytest tests/test_codex_rate_limits.py -q ........................................ [100%] 41 passed in 0.16s $ uv run ruff check headroom/subscription/codex_rate_limits.py headroom/proxy/handlers/openai.py tests/test_codex_rate_limits.py All checks passed! $ uv run mypy headroom/subscription/codex_rate_limits.py Success: no issues found in 1 source file ``` ## Additional Notes - New tests cover: full-payload mapping, window-minutes round-up, credits balance kept only when `has_credits`, promo object vs string, empty payload returns `None`, missing `used_percent` skipped, header-gating (requires Bearer + account-id), poll throttling, and no-event-loop safety. - Scoping to `ChatGPT-Account-Id` keeps the poll off API-key traffic, and the 60s throttle plus in-flight guard bound it to at most one lightweight GET per minute per running proxy. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 00:03:14 +02:00
def test_inject_omits_requires_openai_auth_for_api_key(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
(config_dir / "auth.json").write_text('{"auth_mode": "apikey"}', encoding="utf-8")
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert "requires_openai_auth" not in (config_dir / "config.toml").read_text(
encoding="utf-8"
)
fix(codex): poll /wham/usage for subscription limits (handshake no longer sends x-codex-* headers) (#924) ## Description Codex's subscription usage window (primary/secondary rate-limit gauges) stopped populating for ChatGPT-OAuth sessions. This PR restores it by polling Codex's dedicated usage endpoint instead of relying on response headers that are no longer sent. ### Why the previous approach no longer works The existing code populates `CodexRateLimitState` from `x-codex-*` rate-limit headers captured on the `/v1/responses` WebSocket handshake (`update_from_headers` at WS accept). That worked when OpenAI returned `x-codex-primary-used-percent`, `x-codex-primary-window-minutes`, etc. on the handshake response. OpenAI has since stopped sending those headers on the ChatGPT WebSocket handshake. I confirmed this by faithfully replaying a real Plus-account handshake (both `prewarm` and regular `request_kind`): no `x-codex-*` headers come back on either. This matches OpenAI's own move to a dedicated usage endpoint (`GET /backend-api/codex/usage` in CodexApi mode) and reports such as openai/codex#14728. So `update_from_headers` now runs on every accept but finds nothing to parse, and the window silently stays empty. The headers aren't coming back, so there is nothing to fix in the parsing path. The data now lives behind a request we have to make ourselves. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `subscription/codex_rate_limits.py`: - `parse_codex_usage_payload()` / `update_from_usage_payload()` — map the `GET /backend-api/wham/usage` JSON (`rate_limit.primary_window` / `secondary_window` with `used_percent`, `limit_window_seconds`, `reset_at`; `credits`; `rate_limit_reached_type`) into the existing `CodexRateLimitState`. `limit_window_seconds` is converted to window-minutes with the same round-up codex-rs uses (`(secs + 59) // 60`). - `maybe_schedule_usage_poll()` — fire-and-forget, throttled to one request per 60s, scoped to ChatGPT sessions (requires both a Bearer token and `ChatGPT-Account-Id`; API-key traffic is skipped). Uses an in-flight guard so concurrent accepts don't stack polls. Endpoint is overridable via `HEADROOM_CODEX_USAGE_URL`. - `proxy/handlers/openai.py`: - At the Codex WS accept site, after the now-usually-empty `update_from_headers` block, schedule the usage poll. Wrapped in `contextlib.suppress` and fully non-blocking so it can never delay or fail the WebSocket accept. The old header-capture path is intentionally left in place as a no-cost fallback in case OpenAI restores the headers. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed (live `/wham/usage` replay against a Plus account returned HTTP 200 with the expected schema; payload fixture in tests mirrors that real shape) ## Test Output ``` $ uv run pytest tests/test_codex_rate_limits.py -q ........................................ [100%] 41 passed in 0.16s $ uv run ruff check headroom/subscription/codex_rate_limits.py headroom/proxy/handlers/openai.py tests/test_codex_rate_limits.py All checks passed! $ uv run mypy headroom/subscription/codex_rate_limits.py Success: no issues found in 1 source file ``` ## Additional Notes - New tests cover: full-payload mapping, window-minutes round-up, credits balance kept only when `has_credits`, promo object vs string, empty payload returns `None`, missing `used_percent` skipped, header-gating (requires Bearer + account-id), poll throttling, and no-event-loop safety. - Scoping to `ChatGPT-Account-Id` keeps the poll off API-key traffic, and the 60s throttle plus in-flight guard bound it to at most one lightweight GET per minute per running proxy. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 00:03:14 +02:00
def test_openai_base_url_port_updates_on_rewrap(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
wrap_mod._inject_codex_provider_config(8787)
wrap_mod._inject_codex_provider_config(9999)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = (tmp_path / ".codex" / "config.toml").read_text(encoding="utf-8")
assert 'openai_base_url = "http://127.0.0.1:9999/v1"' in content
assert 'openai_base_url = "http://127.0.0.1:8787/v1"' not in content
def test_openai_base_url_removed_on_unwrap(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
original = '[profiles.default]\nmodel = "gpt-4o"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text(original, encoding="utf-8")
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert 'openai_base_url = "http://127.0.0.1:8787/v1"' in config_file.read_text(
encoding="utf-8"
)
wrap_mod._restore_codex_provider_config()
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert config_file.read_text(encoding="utf-8") == original
def test_strip_cleans_orphaned_openai_base_url(self) -> None:
"""Safety net: orphaned openai_base_url lines are cleaned up."""
content = (
'[profiles.default]\nmodel = "gpt-4o"\nopenai_base_url = "http://127.0.0.1:8787/v1"\n'
)
cleaned = wrap_mod._strip_codex_headroom_blocks(content)
assert "openai_base_url" not in cleaned
assert 'model = "gpt-4o"' in cleaned
def test_no_env_key_in_injected_provider(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""env_key must be absent so Codex doesn't require OPENAI_API_KEY.
Codex treats env_key as a hard requirement if the env var is missing
it throws "Missing environment variable" at startup. Subscription
(ChatGPT Plus) users don't have OPENAI_API_KEY set, so injecting
env_key breaks them (issue #393).
"""
_set_test_home(monkeypatch, tmp_path)
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = (tmp_path / ".codex" / "config.toml").read_text(encoding="utf-8")
assert "env_key" not in content
fix(wrap): preserve custom Codex provider base_url during proxy injection (#1894) ## Description Refs #1614 (Bug 2 only; Bug 1's config-mutation ordering is covered by a separate PR). `headroom wrap codex` unconditionally pointed the proxy's upstream OpenAI route at `api.openai.com`, even when the user's Codex config already declared a custom OpenAI-compatible provider such as `freemodel.dev`, LiteLLM, or vLLM under `[model_providers.<name>]`. The proxy then silently rerouted traffic to OpenAI, which rejected the user's gateway API key, and Codex interpreted the resulting auth failures as an invalid session. ## Type of Change - [x] Bug fix ## Changes Made - `_detect_custom_codex_upstream_base_url` and `_codex_custom_provider_base_urls` in `headroom/cli/wrap.py` scan the existing `config.toml` for a user-declared custom `[model_providers.*]` table, excluding Codex built-ins and Headroom's own table, and return its `base_url` when the selection is unambiguous: either the top-level `model_provider` names it directly, or a prior wrap left the original provider in the `# was: <original>` comment from `_redirect_existing_top_level_keys`. - The detector falls back to the sole custom provider when exactly one candidate exists and no matching top-level selection is present, which covers the issue repro where the custom table exists without a static top-level provider pin. - `_inject_codex_provider_config` now detects that custom upstream before building the injected provider block. When found, it adds `X-Headroom-Base-Url` to `env_http_headers`, mapped to `HEADROOM_CODEX_UPSTREAM_BASE_URL`, matching Codex's env-var-based header contract. - `codex()` exports the detected value into `HEADROOM_CODEX_UPSTREAM_BASE_URL` for the launched Codex process unless the user already set it. The proxy's OpenAI HTTP handlers already honor `X-Headroom-Base-Url`, so HTTP `/v1/chat/completions` and `/v1/responses` requests forward to the preserved gateway instead of the default OpenAI upstream. This is scoped to the HTTP request path. Codex's WebSocket transport for `/v1/responses` resolves its upstream from a separate header-independent path and keeps the existing behavior. ## Testing - [x] Focused Codex wrap tests passed locally before PR review: `pytest tests/test_cli/test_wrap_codex.py -q` - [x] Broader Codex CLI test selection passed locally before PR review: `pytest tests/test_cli/ -k codex -q` - [x] CI lint, format, and type checks passed on PR head `fb769349`. - [x] CI build, test shards, native wrapper, Docker init, Docker wrap, security, merge-conflict, and governance jobs passed on PR head `fb769349`. - [ ] Live `headroom wrap codex` against a real `freemodel.dev` account was not run for this PR. ## Test Output Previously reported local focused test output: ```text pytest tests/test_cli/test_wrap_codex.py -q 83 passed pytest tests/test_cli/ -k codex -q 113 passed, 426 deselected ``` Previously reported local static checks: ```text ruff check ruff format --check mypy ``` CI evidence on PR head `fb769349`: GitHub Actions run `28987811482` completed successfully for CI, including `lint` with `ruff check`, `ruff format --check`, and `mypy`; `build`; `build-wheel`; four test shards; `test-agno`; `test-extras`; `test-dashboard-ui`; `windows-native-wrapper`; `macos-native-wrapper`; and `docker-native-e2e`. PR Governance run `28987811437` completed successfully. ## Real Behavior Proof - Environment: Unit-level Codex config injection using pytest tmp home, PR head `fb769349`, and GitHub Actions CI run `28987811482`. - Exact command / steps: With a Codex config containing `[model_providers.freemodel]`, `base_url = "https://api.freemodel.dev"`, and `wire_api = "responses"`, call `_inject_codex_provider_config(8787)`. - Observed result: The injector returns `https://api.freemodel.dev`; the injected `[model_providers.headroom]` table contains `env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT", "X-Headroom-Base-Url" = "HEADROOM_CODEX_UPSTREAM_BASE_URL" }`; the user's `[model_providers.freemodel]` table remains unchanged; and re-running `_inject_codex_provider_config(9999)` preserves the same upstream while updating the proxy port. - Not tested: Live external traffic through a real `freemodel.dev` key, WebSocket custom-upstream routing, and Bug 1's dependency-check-before-config-mutation path. ## Review Readiness - [x] Scope is limited to #1614 Bug 2, custom provider `base_url` preservation for Codex wrap. - [x] Bug 1 remains out of scope and is called out separately. - [x] The changed code uses the existing proxy `X-Headroom-Base-Url` contract instead of adding a new proxy route. - [x] Ambiguous multiple custom providers keep prior fallback behavior instead of guessing. - [x] A collaborator reviewed and approved the current head after running the focused Codex wrap tests locally. - [x] This PR is ready for human review. ## Checklist - [x] I have performed a self-review. - [x] Focused tests added and passing. - [x] Lint, format, and type-check clean on CI. - [x] No unrelated files changed. - [x] This PR is ready for human review.
2026-07-09 17:55:00 -04:00
# ---------------------------------------------------------------------------
# Custom upstream preservation (#1614): wrap must not silently reroute a
# pre-existing custom [model_providers.*] base_url to api.openai.com.
# ---------------------------------------------------------------------------
class TestDetectCustomCodexUpstreamBaseUrl:
"""Unit tests for the detection helper used by ``_inject_codex_provider_config``."""
def test_no_config_returns_none(self) -> None:
assert wrap_mod._detect_custom_codex_upstream_base_url("") is None
def test_no_custom_provider_returns_none(self) -> None:
content = (
'model_provider = "openai"\n\n'
"[model_providers.openai]\n"
'base_url = "https://api.openai.com/v1"\n'
)
assert wrap_mod._detect_custom_codex_upstream_base_url(content) is None
def test_sole_candidate_used_without_explicit_selection(self) -> None:
"""Matches the #1614 repro: a custom table with no static top-level pin."""
content = (
"[model_providers.freemodel]\n"
'base_url = "https://api.freemodel.dev"\n'
'wire_api = "responses"\n'
)
assert (
wrap_mod._detect_custom_codex_upstream_base_url(content) == "https://api.freemodel.dev"
)
def test_explicit_top_level_selection_wins(self) -> None:
content = (
'model_provider = "freemodel"\n\n'
"[model_providers.freemodel]\n"
'base_url = "https://api.freemodel.dev"\n\n'
"[model_providers.other]\n"
'base_url = "https://api.other.example"\n'
)
assert (
wrap_mod._detect_custom_codex_upstream_base_url(content) == "https://api.freemodel.dev"
)
def test_was_comment_recovers_selection_on_rewrap(self) -> None:
"""After a prior wrap, model_provider reads 'headroom # was: freemodel'."""
content = (
'model_provider = "headroom" # was: freemodel\n\n'
"[model_providers.freemodel]\n"
'base_url = "https://api.freemodel.dev"\n'
)
assert (
wrap_mod._detect_custom_codex_upstream_base_url(content) == "https://api.freemodel.dev"
)
def test_ambiguous_multiple_candidates_returns_none(self) -> None:
content = (
"[model_providers.freemodel]\n"
'base_url = "https://api.freemodel.dev"\n\n'
"[model_providers.other]\n"
'base_url = "https://api.other.example"\n'
)
assert wrap_mod._detect_custom_codex_upstream_base_url(content) is None
def test_builtin_provider_tables_excluded(self) -> None:
content = (
'model_provider = "openai"\n\n'
"[model_providers.openai]\n"
'base_url = "https://api.openai.com/v1"\n\n'
"[model_providers.anthropic]\n"
'base_url = "https://api.anthropic.com/v1"\n'
)
assert wrap_mod._detect_custom_codex_upstream_base_url(content) is None
def test_own_headroom_table_excluded(self) -> None:
content = (
'model_provider = "headroom"\n\n'
"[model_providers.headroom]\n"
'base_url = "http://127.0.0.1:8787/v1"\n'
)
assert wrap_mod._detect_custom_codex_upstream_base_url(content) is None
class TestInjectPreservesCustomUpstreamBaseUrl:
"""``_inject_codex_provider_config`` must preserve a pre-existing custom
provider's ``base_url`` instead of silently rerouting to api.openai.com."""
def test_inject_returns_and_carries_custom_base_url(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
config_file.write_text(
"[model_providers.freemodel]\n"
'base_url = "https://api.freemodel.dev"\n'
'wire_api = "responses"\n',
encoding="utf-8",
)
result = wrap_mod._inject_codex_provider_config(8787)
assert result == "https://api.freemodel.dev"
content = config_file.read_text(encoding="utf-8")
parsed = tomllib.loads(content)
headers = parsed["model_providers"]["headroom"]["env_http_headers"]
assert (
headers[wrap_mod._UPSTREAM_BASE_URL_HEADER_NAME] == wrap_mod._UPSTREAM_BASE_URL_ENV_VAR
)
# The user's own table is left untouched — only headroom's own is managed.
assert parsed["model_providers"]["freemodel"]["base_url"] == "https://api.freemodel.dev"
def test_inject_without_custom_provider_returns_none(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
result = wrap_mod._inject_codex_provider_config(8787)
assert result is None
content = (tmp_path / ".codex" / "config.toml").read_text(encoding="utf-8")
assert wrap_mod._UPSTREAM_BASE_URL_HEADER_NAME not in content
def test_preserved_upstream_survives_rewrap_and_port_change(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
config_file.write_text(
'model_provider = "freemodel"\n\n'
"[model_providers.freemodel]\n"
'base_url = "https://api.freemodel.dev"\n',
encoding="utf-8",
)
first = wrap_mod._inject_codex_provider_config(8787)
second = wrap_mod._inject_codex_provider_config(9999) # port change / re-wrap
assert first == "https://api.freemodel.dev"
assert second == "https://api.freemodel.dev"
content = config_file.read_text(encoding="utf-8")
parsed = tomllib.loads(content)
assert parsed["model_providers"]["headroom"]["base_url"] == "http://127.0.0.1:9999/v1"
headers = parsed["model_providers"]["headroom"]["env_http_headers"]
assert (
headers[wrap_mod._UPSTREAM_BASE_URL_HEADER_NAME] == wrap_mod._UPSTREAM_BASE_URL_ENV_VAR
)
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
class TestInjectAvoidsDuplicateTopLevelKeys:
"""Wrap must not produce a TOML-validity-breaking duplicate-key error.
Codex's ``config.toml`` is parsed strictly: two top-level
``model_provider = `` (or two ``openai_base_url = ``) declarations
cause ``codex`` to refuse to start with
``Error loading config.toml: : :1: duplicate key``. The injector
used to unconditionally prepend a top-level block, breaking any user
who had already configured their own provider (e.g. ``ccswitch``).
"""
def test_inject_does_not_create_duplicate_model_provider(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
config_file.write_text(
'model_provider = "ccswitch"\n'
'openai_base_url = "http://llm-gateway-proxy/v1"\n'
'model = "azure-gpt-5_5"\n'
"\n"
"[model_providers.ccswitch]\n"
'name = "OpenAI"\n'
'base_url = "http://llm-gateway-proxy/v1"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
'wire_api = "responses"\n',
encoding="utf-8",
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
)
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = config_file.read_text(encoding="utf-8")
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
# The wrapped file must be TOML-parseable — duplicate keys were
# the failure mode the user reported.
tomllib.loads(content)
# No duplicate top-level key for either redirectable key.
assert content.count("model_provider =") == 1
assert content.count("openai_base_url =") == 1
# And the rewritten values are the headroom ones.
assert 'model_provider = "headroom"' in content
assert 'openai_base_url = "http://127.0.0.1:8787/v1"' in content
@pytest.mark.parametrize("blank", ["", " ", "\n\t\n"])
def test_redirect_existing_top_level_keys_noop_on_blank(self, blank: str) -> None:
# No redirectable keys to rewrite in blank/whitespace content — the
# helper returns it unchanged so the caller falls back to prepending
# the marker-delimited top-level block.
assert wrap_mod._redirect_existing_top_level_keys(blank, 8787) == blank
def test_inject_preserves_user_value_in_trailing_comment(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
config_file.write_text(
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
'model_provider = "ccswitch"\nopenai_base_url = "http://llm-gateway-proxy/v1"\n',
encoding="utf-8",
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
)
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = config_file.read_text(encoding="utf-8")
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
# Original value kept in a comment so the user can recover it.
# The comment intentionally drops the surrounding quotes — the
# value is a single TOML string and the comment is human-facing.
assert "was: ccswitch" in content
assert "was: http://llm-gateway-proxy/v1" in content
def test_inject_rewrap_updates_existing_redirected_keys(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Idempotent re-wrap on a config that already has top-level keys."""
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text('model_provider = "ccswitch"\n', encoding="utf-8")
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
wrap_mod._inject_codex_provider_config(8787)
wrap_mod._inject_codex_provider_config(9999) # port change
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = config_file.read_text(encoding="utf-8")
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
tomllib.loads(content)
assert content.count("model_provider =") == 1
assert 'model_provider = "headroom"' in content
# Port updated in the openai_base_url we injected.
assert 'openai_base_url = "http://127.0.0.1:9999/v1"' in content
assert 'openai_base_url = "http://127.0.0.1:8787/v1"' not in content
def test_inject_empty_file_still_uses_marker_block(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""No existing top-level keys → fall back to the marker-delimited block."""
_set_test_home(monkeypatch, tmp_path)
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = (tmp_path / ".codex" / "config.toml").read_text(encoding="utf-8")
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
assert wrap_mod._CODEX_TOP_LEVEL_MARKER in content
assert 'model_provider = "headroom"' in content
assert 'openai_base_url = "http://127.0.0.1:8787/v1"' in content
assert "[model_providers.headroom]" in content
fix(codex): avoid duplicate headroom provider config (#1431) ## Description Fixes #1425. `headroom wrap codex` could leave `~/.codex/config.toml` invalid when the user already had a `[model_providers.headroom]` table. The previous duplicate-key handling covered top-level `model_provider` and `openai_base_url`, but the provider table was still appended as a static block. That could produce duplicate `env_http_headers` or duplicate provider-table TOML errors before Codex started. ## Type of Change - [x] Bug fix (non-breaking change fixes an issue) - [ ] New feature (non-breaking change adds functionality) - [ ] Breaking change (fix or feature would cause existing functionality change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a Codex config cleanup helper that removes any pre-existing `[model_providers.headroom]` table from the working copy before `wrap codex` appends the managed Headroom provider block. - Kept unwrap behavior backed by the existing pre-wrap snapshot, so a custom prior `headroom` provider table is restored byte-for-byte on `headroom unwrap codex`. - Added regression tests for TOML validity, a single `env_http_headers` mapping, one managed `[model_providers.headroom]` table, and unwrap restoration. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added to cover the fix ### Test Output ```text Docker: python:3.12-slim Command: uv run --frozen --with pytest pytest tests/test_cli/test_wrap_codex.py Result: 68 passed, 1 warning ``` ## Real Behavior Proof - Environment: disposable Docker container, `python:3.12-slim`, Linux, Python 3.12.13. - Exact command / steps: mounted the worktree into `/workspace`, installed build tools inside the container, then ran `uv run --frozen --with pytest pytest tests/test_cli/test_wrap_codex.py`. - Observed result: all Codex wrap tests passed, including the new regression where an existing `[model_providers.headroom]` table contains `env_http_headers` before wrapping. - Not tested: live interactive `headroom wrap codex` launch against a real user Codex session. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review
2026-06-30 20:42:48 +02:00
def test_inject_replaces_existing_headroom_provider_table(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Existing headroom provider table must not create duplicate TOML keys."""
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
config_file.write_text(
"[model_providers.headroom]\n"
'name = "Existing custom headroom"\n'
'base_url = "http://example.invalid/v1"\n'
"supports_websockets = true\n"
'env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }\n'
"\n"
"[profiles.default]\n"
'model = "gpt-5"\n'
)
wrap_mod._inject_codex_provider_config(8787)
content = config_file.read_text()
tomllib.loads(content)
assert content.count("[model_providers.headroom]") == 1
assert content.count("env_http_headers") == 1
assert 'base_url = "http://127.0.0.1:8787/v1"' in content
assert 'env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }' in content
assert "[profiles.default]" in content
assert 'model = "gpt-5"' in content
def test_unwrap_restores_prior_headroom_provider_table(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Pre-wrap headroom provider table is restored from snapshot."""
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
original = (
"[model_providers.headroom]\n"
'name = "Existing custom headroom"\n'
'base_url = "http://example.invalid/v1"\n'
"supports_websockets = true\n"
'env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }\n'
)
config_file.write_text(original)
wrap_mod._inject_codex_provider_config(8787)
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "restored"
assert config_file.read_text() == original
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
def test_unwrap_restores_prior_model_provider_after_rewrite(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""The snapshot mechanism must still restore the pre-wrap state byte-for-byte."""
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
original = 'model_provider = "ccswitch"\nopenai_base_url = "http://llm-gateway-proxy/v1"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text(original, encoding="utf-8")
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
wrap_mod._inject_codex_provider_config(8787)
status, _ = wrap_mod._restore_codex_provider_config()
assert status == "restored"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert config_file.read_text(encoding="utf-8") == original
fix(wrap): avoid duplicate top-level keys when injecting codex provider (#884) ## Description `_inject_codex_provider_config` in `headroom/cli/wrap.py` unconditionally prepended a top-level block to `~/.codex/config.toml`: ```toml # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- ``` If the user already had a top-level `model_provider` (or `openai_base_url`), the result was two top-level keys with the same name. That violates the TOML spec, and Codex refuses to start with `duplicate key`. This change makes the injector rewrite any pre-existing top-level `model_provider` / `openai_base_url` in place to the headroom values (keeping the user's original value in a `# was: …` trailing comment) and only emit the marker-delimited top-level block for keys the user has not declared. The pre-wrap snapshot mechanism is unchanged, so `headroom unwrap codex` still restores the file byte-for-byte. Closes #883 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py` - New helper `_redirect_existing_top_level_keys(content, port)`: rewrites existing top-level `model_provider` / `openai_base_url` lines to the headroom values and preserves the previous value in a trailing `# was: …` comment. - New helper `_has_redirectable_top_level_key(content, key)`: cheap predicate for the two redirectable keys. - New helper `_build_top_level_block(user_content)`: emits a marker-delimited block containing only the redirectable keys the user has **not** already declared (declared ones are rewritten in place instead, avoiding the TOML duplicate-key error). - `_inject_codex_provider_config` now rewrites declared keys in place and only prepends the marker block for the remaining keys; `requires_openai_auth` handling (#406) is preserved. - `tests/test_cli/test_wrap_codex.py` - New class `TestInjectAvoidsDuplicateTopLevelKeys` (TOML-validity after wrap on a config already declaring a provider, original-value preservation in a `# was:` comment, idempotent re-wrap with a port change, marker-block fallback on an empty file, snapshot-based unwrap restoration). The TOML-validity test parses the wrapped file with `tomllib.loads`, which fails before the fix and passes after. - `CHANGELOG.md` - Added entry under `## Unreleased` → `### Bug Fixes`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py -q ======================== 52 passed, 1 warning in 5.28s ========================= $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 358 source files ``` ## Real Behavior Proof - Environment: macOS 24.6.0, Python 3.13.3, branch `fix/wrap-codex-no-duplicate-keys` rebased onto upstream `main`, Codex CLI config at `~/.codex/config.toml`. - Exact command / steps: Seed a user config matching the bug report (`model_provider = "ccswitch"` + `openai_base_url = "…"` + `[model_providers.ccswitch]`), run the same path `headroom wrap codex` takes (`_inject_codex_provider_config(8787)`), then parse the result with `tomllib.loads(...)` and run `headroom unwrap codex`. - Observed result: On patched code the wrapped `config.toml` parses cleanly — exactly one `model_provider` and one `openai_base_url` remain (the user's prior value preserved in a `# was: …` comment) and the `[model_providers.headroom]` table is present; `unwrap` restores the file byte-for-byte. On the unpatched code the same file raises `tomllib.TOMLDecodeError` (duplicate key). Full suite: 52 passed, ruff lint + format clean (see Test Output). - Not tested: End-to-end launch of the Codex CLI against a live proxy (no Codex e2e harness in this sandbox); Windows / `$CODEX_HOME` override paths (covered only by existing tests). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/config change, no UI. ## Additional Notes `ruff check .`, `ruff format --check .`, and `mypy headroom --ignore-missing-imports` all pass on the rebased branch. The diff stays narrow: `headroom/cli/wrap.py` + its tests + a CHANGELOG entry. Co-authored-by: wangxiangyu7 <wangxiangyu7@lixiang.com>
2026-06-16 00:04:29 +08:00
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
# ---------------------------------------------------------------------------
# Integration tests: full `headroom wrap codex` / `headroom unwrap codex`
# ---------------------------------------------------------------------------
def test_wrap_codex_prepare_only_creates_backup_and_config(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_file = tmp_path / ".codex" / "config.toml"
config_file.parent.mkdir(parents=True)
original = 'model_provider = "openai"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text(original, encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
result = runner.invoke(main, ["wrap", "codex", "--prepare-only", "--port", "8787"])
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert result.exit_code == 0, result.output
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert 'model_provider = "headroom"' in config_file.read_text(encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
backup = tmp_path / ".codex" / "config.toml.headroom-backup"
assert backup.exists()
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert backup.read_text(encoding="utf-8") == original
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
def test_wrap_codex_registers_mcp_when_codex_home_does_not_exist_yet(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""MCP must register on a machine where Codex was installed but never launched.
``CodexRegistrar.detect()`` is just ``~/.codex`` being a directory, and that
directory used to be created as a side effect of writing the rtk guidance
into ``$CODEX_HOME/AGENTS.md``. Once the CLI context tools were removed,
nothing created it, so detect() said "Codex not detected" and Headroom
silently skipped MCP registration leaving every compression marker the
proxy emits unresolvable, with no error shown.
Every other codex test pre-creates ``~/.codex``, which is exactly why none of
them caught it; this one deliberately does not.
"""
_set_test_home(monkeypatch, tmp_path)
codex_dir = tmp_path / ".codex"
assert not codex_dir.exists() # the whole point
result = runner.invoke(main, ["wrap", "codex", "--prepare-only", "--port", "8787"])
assert result.exit_code == 0, result.output
config = codex_dir / "config.toml"
assert config.exists(), "wrap codex did not persist config in the durable Codex home"
assert "[mcp_servers.headroom]" in config.read_text(encoding="utf-8")
assert "not detected" not in result.output
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
def test_wrap_codex_prepare_only_respects_codex_home(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
codex_home = tmp_path / "custom-codex-home"
codex_home.mkdir()
monkeypatch.setenv("CODEX_HOME", str(codex_home))
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
result = runner.invoke(
main,
["wrap", "codex", "--prepare-only", "--no-serena", "--port", "8787"],
)
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
assert result.exit_code == 0, result.output
config_file = codex_home / "config.toml"
assert config_file.exists()
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = config_file.read_text(encoding="utf-8")
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
assert 'model_provider = "headroom"' in content
assert "[mcp_servers.headroom]" in content
assert not (tmp_path / ".codex" / "config.toml").exists()
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
def test_wrap_codex_launch_uses_durable_codex_home(
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
codex_home = tmp_path / "custom-codex-home"
codex_home.mkdir()
monkeypatch.setenv("CODEX_HOME", str(codex_home))
config_file = codex_home / "config.toml"
auth_file = codex_home / "auth.json"
original_config = '[profiles.default]\nmodel = "gpt-4o"\n'
original_auth = '{"auth_mode": "apikey"}'
config_file.write_text(original_config, encoding="utf-8")
auth_file.write_text(original_auth, encoding="utf-8")
launch_env: dict[str, str] = {}
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
rollout = codex_home / "sessions" / "2026" / "07" / "14" / "rollout-thread.jsonl"
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
def fake_launch(
*,
binary: str,
args: tuple,
env: dict[str, str],
port: int,
no_proxy: bool,
tool_label: str,
env_vars_display: list[str],
**kwargs: object,
) -> None:
del args, port, no_proxy, tool_label, env_vars_display, kwargs
assert binary == "/fake/codex"
launch_env.update(env)
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
assert Path(env["CODEX_HOME"]) == codex_home
rollout.parent.mkdir(parents=True)
rollout.write_text('{"type":"session_meta"}\n', encoding="utf-8")
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
with patch(
"headroom.cli.wrap.shutil.which",
side_effect=lambda cmd: "/fake/codex" if cmd == "codex" else None,
):
with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch):
result = runner.invoke(
main,
[
"wrap",
"codex",
"--port",
"8787",
"--no-tokensave",
"--no-serena",
],
)
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
assert result.exit_code == 0, result.output
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
assert launch_env["CODEX_HOME"] == str(codex_home)
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
assert launch_env["OPENAI_BASE_URL"] == "http://127.0.0.1:8787/v1"
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
persisted_config = config_file.read_text(encoding="utf-8")
assert original_config in persisted_config
assert "[mcp_servers.headroom]" in persisted_config
assert 'model_provider = "headroom"' not in persisted_config
assert "[model_providers.headroom]" not in persisted_config
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
assert auth_file.read_text(encoding="utf-8") == original_auth
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
assert rollout.read_text(encoding="utf-8") == '{"type":"session_meta"}\n'
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
def test_codex_session_launch_settings_keep_routing_process_local(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
) -> None:
_set_test_home(monkeypatch, tmp_path)
codex_home = tmp_path / "custom-codex-home"
codex_home.mkdir()
monkeypatch.setenv("CODEX_HOME", str(codex_home))
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
monkeypatch.setattr(wrap_mod, "_project_name_from_cwd", lambda: None)
config_file = codex_home / "config.toml"
original_config = 'model = "gpt-5"\n'
config_file.write_text(original_config, encoding="utf-8")
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
args, env, display = wrap_mod._codex_session_launch_settings(
port=9898,
codex_args=("exec", "hello"),
environ={"CODEX_HOME": str(codex_home)},
)
assert args == (
"--config",
'openai_base_url="http://127.0.0.1:9898/v1"',
"exec",
"hello",
)
assert env["CODEX_HOME"] == str(codex_home)
assert env["OPENAI_BASE_URL"] == "http://127.0.0.1:9898/v1"
assert display == ["OPENAI_BASE_URL=http://127.0.0.1:9898/v1"]
assert config_file.read_text(encoding="utf-8") == original_config
def test_codex_session_launch_settings_preserve_custom_provider_identity(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
codex_home = tmp_path / "custom-codex-home"
codex_home.mkdir()
monkeypatch.setenv("CODEX_HOME", str(codex_home))
monkeypatch.setattr(wrap_mod, "_project_name_from_cwd", lambda: None)
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
config_file = codex_home / "config.toml"
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
original_config = (
'[profiles.work]\nmodel_provider = "company"\n\n'
'[model_providers.company]\nbase_url = "https://api.example.test/v1"\n'
)
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
config_file.write_text(original_config, encoding="utf-8")
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
args, env, _ = wrap_mod._codex_session_launch_settings(
port=9898,
codex_args=("--profile", "work"),
environ={"CODEX_HOME": str(codex_home)},
)
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
assert "model_provider=headroom" not in " ".join(args)
fix(wrap): emit bare dotted keys for Codex --config overrides (#2383) ## Description `headroom wrap codex` with a custom provider emits `--config` overrides whose dotted key quotes **every** segment (`"model_providers"."litellm_prod"."base_url"=…`). Codex's override parser matches dotted segments literally and silently ignores quoted ones, so the overrides are dropped, the session keeps the provider's real `base_url`, and traffic **bypasses Headroom entirely** — the exact silent-bypass reported in #2358 on Codex 0.144.5. I reproduced the parser behavior differentially on a local Codex **0.144.1** (see Real Behavior Proof): a bare key is parsed (Codex errors on the injected value), the same key quoted produces no error at all — the override is silently discarded. Fix: `_codex_dotted_key()` now emits segments **bare** whenever they are valid bare keys (`[A-Za-z0-9_-]+`, which covers `model_providers`, every normal provider id, and hyphenated header names like `X-Headroom-Base-Url`), and quotes only segments where bare emission would corrupt the dotted path (e.g. a provider id containing a dot). Fixes #2358. ## Type of Change - [x] Bug fix (silent proxy bypass for custom-provider Codex wraps) ## Changes Made - `headroom/cli/wrap.py`: `_codex_dotted_key()` quotes only non-bare-key segments; docstring explains the observed Codex parser behavior. The default `openai` path (`openai_base_url=…`, already bare) is unchanged. - `tests/test_cli/test_wrap_codex.py`: the custom-provider launch test now pins the bare form for all three overrides (`base_url`, `supports_websockets`, `env_http_headers.X-Headroom-Base-Url`), plus 2 direct unit tests (bare-when-safe incl. hyphens, quote-only-unsafe). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff` + `mypy`, CI-pinned settings) - [x] Reproduced the parser behavior on a real Codex CLI first, then fixed ### Test Output ```text $ .venv/bin/python -m pytest tests/test_cli/test_wrap_codex.py -q 93 passed # Before the fix the updated assertions fail (generated args still fully quoted): # FAILED ...::test_codex_session_launch_settings_preserve_custom_provider_identity # FAILED ...::test_codex_dotted_key_emits_bare_segments_when_safe # FAILED ...::test_codex_dotted_key_quotes_only_unsafe_segments $ ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py # All checks passed! $ ruff format --check <both> # formatted $ mypy headroom/cli/wrap.py --ignore-missing-imports # Success: no issues ``` ## Real Behavior Proof - Environment: macOS (Darwin), Codex CLI **0.144.1** (`/opt/homebrew/bin/codex`), empty temp `CODEX_HOME` (no auth, no network side effects), branch `fix/codex-config-bare-keys` off `main` (`56c7d4a5`). - Exact command / steps: differential probe of the override parser — `codex exec --skip-git-repo-check -c 'profile="__nope__"' 'hi'` (bare key) vs `codex exec --skip-git-repo-check -c '"profile"="__nope__"' 'hi'` (quoted key), each run once against a fresh empty `CODEX_HOME`. - Observed result: bare key → Codex **parsed the override** and failed fast on it (`Error: legacy profile = "__nope__" config is no longer supported…`); quoted key → **no error referencing the override at all**, Codex proceeded to start a session (banner printed) — the quoted override was silently discarded. That is precisely the #2358 bypass mechanism: every generated custom-provider override was quoted, hence dropped, hence traffic went straight to the real upstream. - Not tested: an end-to-end wrapped session against a live LiteLLM upstream on Codex 0.144.5 (the reporter's exact version); the parser behavior above is version-adjacent (0.144.1) and the argv shape is pinned by unit tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (docstring added) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md — N/A ## Additional Notes - Segments that genuinely need quoting (a provider id with a dot) keep their quotes: on parsers that ignore quoted segments those overrides still won't apply, but bare emission would corrupt a *different* key path, which is strictly worse. Such ids are rare; the common LiteLLM/custom-provider case is fully bare after this fix.
2026-07-19 00:51:39 +08:00
# Bare dotted keys — Codex (0.144.x) silently ignores quoted segments (#2358).
assert 'model_providers.company.base_url="http://127.0.0.1:9898/v1"' in args
assert "model_providers.company.supports_websockets=true" in args
assert (
"model_providers.company.env_http_headers.X-Headroom-Base-Url"
'="HEADROOM_CODEX_UPSTREAM_BASE_URL"'
) in args
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
assert env[wrap_mod._UPSTREAM_BASE_URL_ENV_VAR] == "https://api.example.test/v1"
assert config_file.read_text(encoding="utf-8") == original_config
fix(wrap): emit bare dotted keys for Codex --config overrides (#2383) ## Description `headroom wrap codex` with a custom provider emits `--config` overrides whose dotted key quotes **every** segment (`"model_providers"."litellm_prod"."base_url"=…`). Codex's override parser matches dotted segments literally and silently ignores quoted ones, so the overrides are dropped, the session keeps the provider's real `base_url`, and traffic **bypasses Headroom entirely** — the exact silent-bypass reported in #2358 on Codex 0.144.5. I reproduced the parser behavior differentially on a local Codex **0.144.1** (see Real Behavior Proof): a bare key is parsed (Codex errors on the injected value), the same key quoted produces no error at all — the override is silently discarded. Fix: `_codex_dotted_key()` now emits segments **bare** whenever they are valid bare keys (`[A-Za-z0-9_-]+`, which covers `model_providers`, every normal provider id, and hyphenated header names like `X-Headroom-Base-Url`), and quotes only segments where bare emission would corrupt the dotted path (e.g. a provider id containing a dot). Fixes #2358. ## Type of Change - [x] Bug fix (silent proxy bypass for custom-provider Codex wraps) ## Changes Made - `headroom/cli/wrap.py`: `_codex_dotted_key()` quotes only non-bare-key segments; docstring explains the observed Codex parser behavior. The default `openai` path (`openai_base_url=…`, already bare) is unchanged. - `tests/test_cli/test_wrap_codex.py`: the custom-provider launch test now pins the bare form for all three overrides (`base_url`, `supports_websockets`, `env_http_headers.X-Headroom-Base-Url`), plus 2 direct unit tests (bare-when-safe incl. hyphens, quote-only-unsafe). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff` + `mypy`, CI-pinned settings) - [x] Reproduced the parser behavior on a real Codex CLI first, then fixed ### Test Output ```text $ .venv/bin/python -m pytest tests/test_cli/test_wrap_codex.py -q 93 passed # Before the fix the updated assertions fail (generated args still fully quoted): # FAILED ...::test_codex_session_launch_settings_preserve_custom_provider_identity # FAILED ...::test_codex_dotted_key_emits_bare_segments_when_safe # FAILED ...::test_codex_dotted_key_quotes_only_unsafe_segments $ ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py # All checks passed! $ ruff format --check <both> # formatted $ mypy headroom/cli/wrap.py --ignore-missing-imports # Success: no issues ``` ## Real Behavior Proof - Environment: macOS (Darwin), Codex CLI **0.144.1** (`/opt/homebrew/bin/codex`), empty temp `CODEX_HOME` (no auth, no network side effects), branch `fix/codex-config-bare-keys` off `main` (`56c7d4a5`). - Exact command / steps: differential probe of the override parser — `codex exec --skip-git-repo-check -c 'profile="__nope__"' 'hi'` (bare key) vs `codex exec --skip-git-repo-check -c '"profile"="__nope__"' 'hi'` (quoted key), each run once against a fresh empty `CODEX_HOME`. - Observed result: bare key → Codex **parsed the override** and failed fast on it (`Error: legacy profile = "__nope__" config is no longer supported…`); quoted key → **no error referencing the override at all**, Codex proceeded to start a session (banner printed) — the quoted override was silently discarded. That is precisely the #2358 bypass mechanism: every generated custom-provider override was quoted, hence dropped, hence traffic went straight to the real upstream. - Not tested: an end-to-end wrapped session against a live LiteLLM upstream on Codex 0.144.5 (the reporter's exact version); the parser behavior above is version-adjacent (0.144.1) and the argv shape is pinned by unit tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (docstring added) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md — N/A ## Additional Notes - Segments that genuinely need quoting (a provider id with a dot) keep their quotes: on parsers that ignore quoted segments those overrides still won't apply, but bare emission would corrupt a *different* key path, which is strictly worse. Such ids are rare; the common LiteLLM/custom-provider case is fully bare after this fix.
2026-07-19 00:51:39 +08:00
def test_codex_dotted_key_emits_bare_segments_when_safe() -> None:
"""#2358: quoted segments are silently ignored by Codex's --config parser."""
assert (
wrap_mod._codex_dotted_key("model_providers", "litellm_prod", "base_url")
== "model_providers.litellm_prod.base_url"
)
# Hyphens are valid in bare keys (header names under env_http_headers).
assert (
wrap_mod._codex_dotted_key("env_http_headers", "X-Headroom-Base-Url")
== "env_http_headers.X-Headroom-Base-Url"
)
def test_codex_dotted_key_quotes_only_unsafe_segments() -> None:
# A provider name that would corrupt the dotted path if emitted bare keeps
# its quotes; every safe neighbor stays bare.
assert (
wrap_mod._codex_dotted_key("model_providers", "my.provider", "base_url")
== 'model_providers."my.provider".base_url'
)
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
def test_wrap_codex_rejects_custom_provider_without_upstream_base_url(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
codex_home = tmp_path / "custom-codex-home"
codex_home.mkdir()
monkeypatch.setenv("CODEX_HOME", str(codex_home))
(codex_home / "config.toml").write_text(
'model_provider = "company"\n[model_providers.company]\nname = "Company"\n',
encoding="utf-8",
)
def fake_launch(**kwargs: object) -> None:
configure_launch = kwargs["configure_launch"]
assert callable(configure_launch)
configure_launch(
8787,
kwargs["args"],
kwargs["env"],
kwargs["env_vars_display"],
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
)
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
with patch(
"headroom.cli.wrap.shutil.which",
side_effect=lambda cmd: "/fake/codex" if cmd == "codex" else None,
):
with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch):
result = runner.invoke(
main,
[
"wrap",
"codex",
"--port",
"8787",
"--no-mcp",
"--no-tokensave",
"--no-serena",
],
)
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
assert result.exit_code != 0
assert "custom provider 'company' has no upstream base_url" in result.output
def test_wrap_codex_routes_model_provider_selected_by_config_argument(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
codex_home = tmp_path / "custom-codex-home"
codex_home.mkdir()
monkeypatch.setenv("CODEX_HOME", str(codex_home))
monkeypatch.setattr(wrap_mod, "_project_name_from_cwd", lambda: None)
(codex_home / "config.toml").write_text(
'[model_providers.company]\nbase_url = "https://api.example.test/v1"\n',
encoding="utf-8",
)
configured_env: dict[str, str] = {}
def fake_launch(**kwargs: object) -> None:
configure_launch = kwargs["configure_launch"]
assert callable(configure_launch)
_, env, _ = configure_launch(
8787,
kwargs["args"],
kwargs["env"],
kwargs["env_vars_display"],
)
configured_env.update(env)
fix(wrap): keep Codex RTK guidance global (#1240) ## Description Stops `headroom wrap codex` from writing RTK instructions into the shared project `AGENTS.md`. RTK guidance remains installed in the global Codex `AGENTS.md`, where it applies only to the user who configured Headroom. Closes #1235 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Remove project-level RTK guidance injection from `headroom wrap codex`. - Preserve global Codex RTK guidance injection. - Add a regression test proving an existing project `AGENTS.md` remains byte-for-byte unchanged. - Document the fix in the Unreleased changelog. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --extra dev pytest tests/test_cli/test_wrap_codex.py -q 57 passed in 9.54s $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ uv run mypy headroom/cli/wrap.py Success: no issues found in 1 source file $ npx --yes --package=@commitlint/cli --package=@commitlint/config-conventional commitlint --from HEAD~1 --to HEAD --config .commitlintrc.json exited 0 ``` ## Real Behavior Proof - Environment: Windows, Python 3.12.12, locally built Headroom CLI, isolated project directory, isolated `CODEX_HOME`, and isolated `HEADROOM_WORKSPACE_DIR`. - Exact command / steps: created a project `AGENTS.md`, recorded its SHA-256, then ran `.venv\Scripts\headroom.exe wrap codex --prepare-only --no-mcp --no-serena` with isolated environment directories and compared the project hash before and after. - Observed result: command exited 0; RTK downloaded successfully; the project `AGENTS.md` hash remained `2CFF2F420178BFEB9BB863C743805410F2CA30F3F7F70121A8538314CBD0F8B5`; the global Codex `AGENTS.md` was created and contained the `headroom:rtk-instructions` marker. - Not tested: launching an interactive Codex session after preparation; non-Codex wrapper targets, which are unchanged. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Not applicable. ## Additional Notes The repository-wide pre-commit mypy hook reports existing Windows-only `fcntl` attribute errors in `headroom/subscription/tracker.py` and `headroom/install/runtime.py`; targeted mypy for the changed module passes. The plugin-version hook was also verified directly with the project interpreter and correctly skipped this feature branch. This pull request includes code written with the assistance of AI. The changes have not yet been reviewed by a human.
2026-06-21 22:41:06 +05:30
with patch(
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
"headroom.cli.wrap.shutil.which",
side_effect=lambda cmd: "/fake/codex" if cmd == "codex" else None,
fix(wrap): keep Codex RTK guidance global (#1240) ## Description Stops `headroom wrap codex` from writing RTK instructions into the shared project `AGENTS.md`. RTK guidance remains installed in the global Codex `AGENTS.md`, where it applies only to the user who configured Headroom. Closes #1235 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Remove project-level RTK guidance injection from `headroom wrap codex`. - Preserve global Codex RTK guidance injection. - Add a regression test proving an existing project `AGENTS.md` remains byte-for-byte unchanged. - Document the fix in the Unreleased changelog. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --extra dev pytest tests/test_cli/test_wrap_codex.py -q 57 passed in 9.54s $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py 2 files already formatted $ uv run mypy headroom/cli/wrap.py Success: no issues found in 1 source file $ npx --yes --package=@commitlint/cli --package=@commitlint/config-conventional commitlint --from HEAD~1 --to HEAD --config .commitlintrc.json exited 0 ``` ## Real Behavior Proof - Environment: Windows, Python 3.12.12, locally built Headroom CLI, isolated project directory, isolated `CODEX_HOME`, and isolated `HEADROOM_WORKSPACE_DIR`. - Exact command / steps: created a project `AGENTS.md`, recorded its SHA-256, then ran `.venv\Scripts\headroom.exe wrap codex --prepare-only --no-mcp --no-serena` with isolated environment directories and compared the project hash before and after. - Observed result: command exited 0; RTK downloaded successfully; the project `AGENTS.md` hash remained `2CFF2F420178BFEB9BB863C743805410F2CA30F3F7F70121A8538314CBD0F8B5`; the global Codex `AGENTS.md` was created and contained the `headroom:rtk-instructions` marker. - Not tested: launching an interactive Codex session after preparation; non-Codex wrapper targets, which are unchanged. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Not applicable. ## Additional Notes The repository-wide pre-commit mypy hook reports existing Windows-only `fcntl` attribute errors in `headroom/subscription/tracker.py` and `headroom/install/runtime.py`; targeted mypy for the changed module passes. The plugin-version hook was also verified directly with the project interpreter and correctly skipped this feature branch. This pull request includes code written with the assistance of AI. The changes have not yet been reviewed by a human.
2026-06-21 22:41:06 +05:30
):
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch):
result = runner.invoke(
main,
[
"wrap",
"codex",
"--no-mcp",
"--no-tokensave",
"--no-serena",
"--",
"--config",
'model_provider="company"',
],
)
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
assert result.exit_code == 0, result.output
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
assert configured_env[wrap_mod._UPSTREAM_BASE_URL_ENV_VAR] == ("https://api.example.test/v1")
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
def test_unwrap_codex_without_codex_home_warns_on_ambiguous_noop(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
codex_home = tmp_path / "custom-codex-home"
codex_home.mkdir()
monkeypatch.setenv("CODEX_HOME", str(codex_home))
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
wrap_result = runner.invoke(
main,
[
"wrap",
"codex",
"--prepare-only",
"--no-mcp",
"--no-serena",
"--port",
"8787",
],
)
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
assert wrap_result.exit_code == 0, wrap_result.output
config_file = codex_home / "config.toml"
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert 'openai_base_url = "http://127.0.0.1:8787/v1"' in config_file.read_text(encoding="utf-8")
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
monkeypatch.delenv("CODEX_HOME", raising=False)
unwrap_result = runner.invoke(main, ["unwrap", "codex", "--no-stop-proxy"])
assert unwrap_result.exit_code == 0, unwrap_result.output
assert "Warning: found no Headroom wrap markers in the default Codex config" in (
unwrap_result.output
)
assert "If you wrapped Codex with CODEX_HOME" in unwrap_result.output
assert "CODEX_HOME=/path/to/codex-home headroom unwrap codex" in unwrap_result.output
assert "Nothing to undo" in unwrap_result.output
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert 'openai_base_url = "http://127.0.0.1:8787/v1"' in config_file.read_text(encoding="utf-8")
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
def test_start_proxy_uses_separate_session_for_signal_isolation(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Proxy child should not receive Ctrl-C intended for the wrapped CLI."""
popen_kwargs: dict[str, object] = {}
class FakeProc:
returncode = None
def poll(self) -> None:
return None
def fake_popen(*args: object, **kwargs: object) -> FakeProc:
popen_kwargs.update(kwargs)
return FakeProc()
monkeypatch.setattr(wrap_mod, "_get_log_path", lambda: tmp_path / "proxy.log")
monkeypatch.setattr(wrap_mod, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_mod.subprocess, "Popen", fake_popen)
proc = wrap_mod._start_proxy(8787, agent_type="codex")
assert isinstance(proc, FakeProc)
assert popen_kwargs["start_new_session"] == (wrap_mod.os.name == "posix")
feat: add dashboard agent usage stats (#814) ## Description Add a clear dashboard view for per-agent token usage so end users can see Cursor, Claude, Codex, and other detected clients with before/after token counts, tokens saved, and savings percentages. The stats API now exposes a stable `agent_usage` object that the dashboard renders near the top of the session view. Fixes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made ### New Files **Tests:** - `tests/test_dashboard_agent_usage.py` — Covers agent classification, exact per-request aggregation, and aggregate fallback behavior. ### Modified Files - `headroom/proxy/server.py` — Adds per-agent usage aggregation to `/stats` with before tokens, after tokens, output tokens, saved tokens, savings percentage, source, providers, and models. - `headroom/dashboard/templates/dashboard.html` — Adds a prominent Agent Usage panel with totals, coverage status, per-agent token-flow bars, request counts, before/after tokens, saved tokens, and share of savings. ## Testing - [x] Unit tests pass: `.venv312/bin/pytest tests/test_dashboard_agent_usage.py` - [x] Linting passes: `.venv312/bin/ruff check headroom/proxy/server.py tests/test_dashboard_agent_usage.py` - [x] Diff whitespace check passes: `git diff --check origin/main...HEAD` - [x] Dashboard smoke render: local proxy on `127.0.0.1:8790`, captured Chrome headless screenshot of `/dashboard` - [x] New tests added for new functionality ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing relevant unit tests pass locally with my changes - [ ] I have made corresponding changes to the documentation - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The agent usage panel uses exact request-log data when available. If detailed request logs are empty, it falls back to aggregate provider/model request counts and labels the coverage as aggregate fallback so users are not misled.
2026-06-12 22:12:22 +03:00
@pytest.mark.parametrize("agent_type", ["claude", "codex", "cursor"])
fix(wrap): keep agent savings opt-in (#1294) ## Description Fixes a regression from #830 where `headroom wrap codex` / `claude` / `cursor` treated `agent-90` as required even when the user started a normal proxy without `HEADROOM_SAVINGS_PROFILE`. A plain `headroom proxy` on port 8787 followed by `headroom wrap codex` currently reports `Proxy on port 8787 is missing: --savings-profile` and tries to restart the already-running proxy. The `agent-90` profile was documented as opt-in, so wrap should only require or inject it when `HEADROOM_SAVINGS_PROFILE` is explicitly set. Closes #1293 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Stop defaulting agent wrappers to `agent-90` when `HEADROOM_SAVINGS_PROFILE` is unset. - Stop reporting agent-savings config mismatches unless an agent savings profile was explicitly requested. - Add regression tests for default wrap startup, explicit profile forwarding, and reuse/restart behavior around existing proxies. - Fix repo-wide pre-commit issues found during amend: Windows-safe `fcntl` typing, an optional env typing issue, OpenCode JSON parser return typing, and ruff import/format drift. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text > maturin develop -m crates/headroom-py/Cargo.toml Finished `dev` profile [unoptimized + debuginfo] target(s) in 27.41s Built wheel for abi3 Python >= 3.10 Installed headroom-ai-0.27.0 > C:\git\headroom\.venv\Scripts\python.exe -c "from headroom._core import hello; print(hello())" headroom-core > ruff check . All checks passed! > ruff format --check . 913 files already formatted > C:\git\headroom\.venv\Scripts\python.exe -m mypy headroom Success: no issues found in 388 source files > C:\git\headroom\.venv\Scripts\python.exe -m pytest tests/test_agent_savings.py tests/test_cli/test_wrap_persistent.py tests/test_proxy_healthchecks.py tests/test_transforms/test_content_router.py -q collected 112 items 112 passed, 1 warning in 16.04s > git diff --check # no output > git commit --amend --no-edit Sync plugin versions.....................................................Passed ruff.....................................................................Passed ruff-format..............................................................Passed mypy.....................................................................Passed ``` ## Real Behavior Proof - Environment: Windows dev checkout, Python 3.13.3 via `C:\git\headroom\.venv\Scripts\python.exe`, Rust extension built with `maturin develop -m crates/headroom-py/Cargo.toml`. - Exact command / steps: reproduced the code path from #830 by exercising `_ensure_proxy(8787, False, agent_type="codex")` with a running proxy health payload that has no `savings_profile`, and by exercising `_start_proxy(8787, agent_type="codex")` with `HEADROOM_SAVINGS_PROFILE` unset and set. - Observed result: without `HEADROOM_SAVINGS_PROFILE`, wrap reuses the running proxy and `_start_proxy` does not inject `HEADROOM_SAVINGS_PROFILE` or `HEADROOM_TARGET_RATIO`; with `HEADROOM_SAVINGS_PROFILE=agent-90`, wrap still requires/forwards the profile and restarts an incompatible proxy. - Not tested: full end-to-end CLI launch against a live Codex binary. The focused proxy/wrap tests cover the failing restart/config decision directly. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes The pytest run emits an existing Windows `cp1252` background-thread warning while reading subprocess output; the tests still pass. No documentation or changelog update is included because this restores the already-documented opt-in behavior for `agent-90`.
2026-06-22 19:06:04 -05:00
def test_start_proxy_does_not_apply_agent_90_defaults(
feat: add dashboard agent usage stats (#814) ## Description Add a clear dashboard view for per-agent token usage so end users can see Cursor, Claude, Codex, and other detected clients with before/after token counts, tokens saved, and savings percentages. The stats API now exposes a stable `agent_usage` object that the dashboard renders near the top of the session view. Fixes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made ### New Files **Tests:** - `tests/test_dashboard_agent_usage.py` — Covers agent classification, exact per-request aggregation, and aggregate fallback behavior. ### Modified Files - `headroom/proxy/server.py` — Adds per-agent usage aggregation to `/stats` with before tokens, after tokens, output tokens, saved tokens, savings percentage, source, providers, and models. - `headroom/dashboard/templates/dashboard.html` — Adds a prominent Agent Usage panel with totals, coverage status, per-agent token-flow bars, request counts, before/after tokens, saved tokens, and share of savings. ## Testing - [x] Unit tests pass: `.venv312/bin/pytest tests/test_dashboard_agent_usage.py` - [x] Linting passes: `.venv312/bin/ruff check headroom/proxy/server.py tests/test_dashboard_agent_usage.py` - [x] Diff whitespace check passes: `git diff --check origin/main...HEAD` - [x] Dashboard smoke render: local proxy on `127.0.0.1:8790`, captured Chrome headless screenshot of `/dashboard` - [x] New tests added for new functionality ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing relevant unit tests pass locally with my changes - [ ] I have made corresponding changes to the documentation - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The agent usage panel uses exact request-log data when available. If detailed request logs are empty, it falls back to aggregate provider/model request counts and labels the coverage as aggregate fallback so users are not misled.
2026-06-12 22:12:22 +03:00
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, agent_type: str
) -> None:
fix(wrap): keep agent savings opt-in (#1294) ## Description Fixes a regression from #830 where `headroom wrap codex` / `claude` / `cursor` treated `agent-90` as required even when the user started a normal proxy without `HEADROOM_SAVINGS_PROFILE`. A plain `headroom proxy` on port 8787 followed by `headroom wrap codex` currently reports `Proxy on port 8787 is missing: --savings-profile` and tries to restart the already-running proxy. The `agent-90` profile was documented as opt-in, so wrap should only require or inject it when `HEADROOM_SAVINGS_PROFILE` is explicitly set. Closes #1293 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Stop defaulting agent wrappers to `agent-90` when `HEADROOM_SAVINGS_PROFILE` is unset. - Stop reporting agent-savings config mismatches unless an agent savings profile was explicitly requested. - Add regression tests for default wrap startup, explicit profile forwarding, and reuse/restart behavior around existing proxies. - Fix repo-wide pre-commit issues found during amend: Windows-safe `fcntl` typing, an optional env typing issue, OpenCode JSON parser return typing, and ruff import/format drift. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text > maturin develop -m crates/headroom-py/Cargo.toml Finished `dev` profile [unoptimized + debuginfo] target(s) in 27.41s Built wheel for abi3 Python >= 3.10 Installed headroom-ai-0.27.0 > C:\git\headroom\.venv\Scripts\python.exe -c "from headroom._core import hello; print(hello())" headroom-core > ruff check . All checks passed! > ruff format --check . 913 files already formatted > C:\git\headroom\.venv\Scripts\python.exe -m mypy headroom Success: no issues found in 388 source files > C:\git\headroom\.venv\Scripts\python.exe -m pytest tests/test_agent_savings.py tests/test_cli/test_wrap_persistent.py tests/test_proxy_healthchecks.py tests/test_transforms/test_content_router.py -q collected 112 items 112 passed, 1 warning in 16.04s > git diff --check # no output > git commit --amend --no-edit Sync plugin versions.....................................................Passed ruff.....................................................................Passed ruff-format..............................................................Passed mypy.....................................................................Passed ``` ## Real Behavior Proof - Environment: Windows dev checkout, Python 3.13.3 via `C:\git\headroom\.venv\Scripts\python.exe`, Rust extension built with `maturin develop -m crates/headroom-py/Cargo.toml`. - Exact command / steps: reproduced the code path from #830 by exercising `_ensure_proxy(8787, False, agent_type="codex")` with a running proxy health payload that has no `savings_profile`, and by exercising `_start_proxy(8787, agent_type="codex")` with `HEADROOM_SAVINGS_PROFILE` unset and set. - Observed result: without `HEADROOM_SAVINGS_PROFILE`, wrap reuses the running proxy and `_start_proxy` does not inject `HEADROOM_SAVINGS_PROFILE` or `HEADROOM_TARGET_RATIO`; with `HEADROOM_SAVINGS_PROFILE=agent-90`, wrap still requires/forwards the profile and restarts an incompatible proxy. - Not tested: full end-to-end CLI launch against a live Codex binary. The focused proxy/wrap tests cover the failing restart/config decision directly. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes The pytest run emits an existing Windows `cp1252` background-thread warning while reading subprocess output; the tests still pass. No documentation or changelog update is included because this restores the already-documented opt-in behavior for `agent-90`.
2026-06-22 19:06:04 -05:00
"""Wrapped coding agents keep agent-savings opt-in by default."""
feat: ship the coding profile as Headroom's out-of-box default posture (#1893) Make a bare `headroom proxy` (and the uvicorn factory / argparse main) default to the cache-mode coding posture instead of requiring users to set a dozen env vars. Profile (agent_savings.py): * "coding" is now the DEFAULT profile (DEFAULT_PROFILE), and it is rewritten for cache mode: proxy_mode="cache" and compress_user_messages=True (cache mode compresses the newest OBSERVATION delta — a user/tool turn — so compress_user must be on or there is nothing to compress; prefix stability is preserved by the delta engine, not by refusing to touch user turns). * AgentSavingsProfile carries the standalone router/handler toggles too (tool_search, cross_turn_dedup, lossless_then_lossy, protect_reads, code_aware, effort_router, lossless, min_chars_for_block); proxy_env() emits them. Defaults preserve current behavior for the other profiles. * coding sets: tool_search=1, dedupe=1, lossless_then_lossy=1, protect_reads=1, code_aware=1, effort_router=0, lossless=0, min_chars_for_block=25. CCR stays ON (no HEADROOM_NO_CCR) so any lossy loss is recoverable. * apply_agent_savings_env_defaults() now honors an explicit HEADROOM_SAVINGS_PROFILE already in the env before falling back to the default. Delivery (pollution-free by construction): * MODE and savings_profile default via INLINE defaults in the config builders (cache / coding) — no global env mutation, so unit tests that build config directly keep clean defaults. * The request-time toggles are seeded into os.environ (setdefault) via seed_proxy_env_defaults() ONLY at the executable/deployment entries — run_server() (before serving) and create_app_from_env() (uvicorn factory) — NOT in the CLI command or any library builder, so CliRunner tests never leak coding defaults into os.environ across tests. * CLI code_aware now defaults ON, matching the argparse server path (degrades to a no-op without tree-sitter). All explicit user env vars / CLI flags still win (setdefault + `or` fallbacks). HEADROOM_LOSSLESS was already 0 by default; unchanged. Tests: coding-profile + CLI-proxy-env tests updated to the new defaults; 1047 passed across the touched areas (only pre-existing memory/env failures remain). ## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 10:49:54 -04:00
# Clean baseline: the proxy's out-of-box coding profile seeds these into the
# process env at startup (``seed_proxy_env_defaults``), which another test in
# the shard can leave behind in ``os.environ``. This test is about what the
# WRAPPER adds, so start from an unset env rather than inheriting pollution.
for _var in (
"HEADROOM_SAVINGS_PROFILE",
"HEADROOM_TARGET_RATIO",
"HEADROOM_MAX_ITEMS",
"HEADROOM_SMART_CRUSHER_COMPACTION",
):
monkeypatch.delenv(_var, raising=False)
feat: add dashboard agent usage stats (#814) ## Description Add a clear dashboard view for per-agent token usage so end users can see Cursor, Claude, Codex, and other detected clients with before/after token counts, tokens saved, and savings percentages. The stats API now exposes a stable `agent_usage` object that the dashboard renders near the top of the session view. Fixes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made ### New Files **Tests:** - `tests/test_dashboard_agent_usage.py` — Covers agent classification, exact per-request aggregation, and aggregate fallback behavior. ### Modified Files - `headroom/proxy/server.py` — Adds per-agent usage aggregation to `/stats` with before tokens, after tokens, output tokens, saved tokens, savings percentage, source, providers, and models. - `headroom/dashboard/templates/dashboard.html` — Adds a prominent Agent Usage panel with totals, coverage status, per-agent token-flow bars, request counts, before/after tokens, saved tokens, and share of savings. ## Testing - [x] Unit tests pass: `.venv312/bin/pytest tests/test_dashboard_agent_usage.py` - [x] Linting passes: `.venv312/bin/ruff check headroom/proxy/server.py tests/test_dashboard_agent_usage.py` - [x] Diff whitespace check passes: `git diff --check origin/main...HEAD` - [x] Dashboard smoke render: local proxy on `127.0.0.1:8790`, captured Chrome headless screenshot of `/dashboard` - [x] New tests added for new functionality ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing relevant unit tests pass locally with my changes - [ ] I have made corresponding changes to the documentation - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The agent usage panel uses exact request-log data when available. If detailed request logs are empty, it falls back to aggregate provider/model request counts and labels the coverage as aggregate fallback so users are not misled.
2026-06-12 22:12:22 +03:00
popen_kwargs: dict[str, object] = {}
class FakeProc:
returncode = None
def poll(self) -> None:
return None
def fake_popen(*args: object, **kwargs: object) -> FakeProc:
popen_kwargs.update(kwargs)
return FakeProc()
monkeypatch.setattr(wrap_mod, "_get_log_path", lambda: tmp_path / "proxy.log")
monkeypatch.setattr(wrap_mod, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_mod.subprocess, "Popen", fake_popen)
wrap_mod._start_proxy(8787, agent_type=agent_type)
env = popen_kwargs["env"]
assert isinstance(env, dict)
fix(wrap): keep agent savings opt-in (#1294) ## Description Fixes a regression from #830 where `headroom wrap codex` / `claude` / `cursor` treated `agent-90` as required even when the user started a normal proxy without `HEADROOM_SAVINGS_PROFILE`. A plain `headroom proxy` on port 8787 followed by `headroom wrap codex` currently reports `Proxy on port 8787 is missing: --savings-profile` and tries to restart the already-running proxy. The `agent-90` profile was documented as opt-in, so wrap should only require or inject it when `HEADROOM_SAVINGS_PROFILE` is explicitly set. Closes #1293 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Stop defaulting agent wrappers to `agent-90` when `HEADROOM_SAVINGS_PROFILE` is unset. - Stop reporting agent-savings config mismatches unless an agent savings profile was explicitly requested. - Add regression tests for default wrap startup, explicit profile forwarding, and reuse/restart behavior around existing proxies. - Fix repo-wide pre-commit issues found during amend: Windows-safe `fcntl` typing, an optional env typing issue, OpenCode JSON parser return typing, and ruff import/format drift. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text > maturin develop -m crates/headroom-py/Cargo.toml Finished `dev` profile [unoptimized + debuginfo] target(s) in 27.41s Built wheel for abi3 Python >= 3.10 Installed headroom-ai-0.27.0 > C:\git\headroom\.venv\Scripts\python.exe -c "from headroom._core import hello; print(hello())" headroom-core > ruff check . All checks passed! > ruff format --check . 913 files already formatted > C:\git\headroom\.venv\Scripts\python.exe -m mypy headroom Success: no issues found in 388 source files > C:\git\headroom\.venv\Scripts\python.exe -m pytest tests/test_agent_savings.py tests/test_cli/test_wrap_persistent.py tests/test_proxy_healthchecks.py tests/test_transforms/test_content_router.py -q collected 112 items 112 passed, 1 warning in 16.04s > git diff --check # no output > git commit --amend --no-edit Sync plugin versions.....................................................Passed ruff.....................................................................Passed ruff-format..............................................................Passed mypy.....................................................................Passed ``` ## Real Behavior Proof - Environment: Windows dev checkout, Python 3.13.3 via `C:\git\headroom\.venv\Scripts\python.exe`, Rust extension built with `maturin develop -m crates/headroom-py/Cargo.toml`. - Exact command / steps: reproduced the code path from #830 by exercising `_ensure_proxy(8787, False, agent_type="codex")` with a running proxy health payload that has no `savings_profile`, and by exercising `_start_proxy(8787, agent_type="codex")` with `HEADROOM_SAVINGS_PROFILE` unset and set. - Observed result: without `HEADROOM_SAVINGS_PROFILE`, wrap reuses the running proxy and `_start_proxy` does not inject `HEADROOM_SAVINGS_PROFILE` or `HEADROOM_TARGET_RATIO`; with `HEADROOM_SAVINGS_PROFILE=agent-90`, wrap still requires/forwards the profile and restarts an incompatible proxy. - Not tested: full end-to-end CLI launch against a live Codex binary. The focused proxy/wrap tests cover the failing restart/config decision directly. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes The pytest run emits an existing Windows `cp1252` background-thread warning while reading subprocess output; the tests still pass. No documentation or changelog update is included because this restores the already-documented opt-in behavior for `agent-90`.
2026-06-22 19:06:04 -05:00
assert "HEADROOM_SAVINGS_PROFILE" not in env
assert "HEADROOM_TARGET_RATIO" not in env
assert "HEADROOM_MAX_ITEMS" not in env
assert "HEADROOM_SMART_CRUSHER_COMPACTION" not in env
feat: add dashboard agent usage stats (#814) ## Description Add a clear dashboard view for per-agent token usage so end users can see Cursor, Claude, Codex, and other detected clients with before/after token counts, tokens saved, and savings percentages. The stats API now exposes a stable `agent_usage` object that the dashboard renders near the top of the session view. Fixes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made ### New Files **Tests:** - `tests/test_dashboard_agent_usage.py` — Covers agent classification, exact per-request aggregation, and aggregate fallback behavior. ### Modified Files - `headroom/proxy/server.py` — Adds per-agent usage aggregation to `/stats` with before tokens, after tokens, output tokens, saved tokens, savings percentage, source, providers, and models. - `headroom/dashboard/templates/dashboard.html` — Adds a prominent Agent Usage panel with totals, coverage status, per-agent token-flow bars, request counts, before/after tokens, saved tokens, and share of savings. ## Testing - [x] Unit tests pass: `.venv312/bin/pytest tests/test_dashboard_agent_usage.py` - [x] Linting passes: `.venv312/bin/ruff check headroom/proxy/server.py tests/test_dashboard_agent_usage.py` - [x] Diff whitespace check passes: `git diff --check origin/main...HEAD` - [x] Dashboard smoke render: local proxy on `127.0.0.1:8790`, captured Chrome headless screenshot of `/dashboard` - [x] New tests added for new functionality ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing relevant unit tests pass locally with my changes - [ ] I have made corresponding changes to the documentation - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The agent usage panel uses exact request-log data when available. If detailed request logs are empty, it falls back to aggregate provider/model request counts and labels the coverage as aggregate fallback so users are not misled.
2026-06-12 22:12:22 +03:00
def test_start_proxy_preserves_explicit_savings_overrides(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""User-provided savings env vars should override wrapper defaults."""
popen_kwargs: dict[str, object] = {}
class FakeProc:
returncode = None
def poll(self) -> None:
return None
def fake_popen(*args: object, **kwargs: object) -> FakeProc:
popen_kwargs.update(kwargs)
return FakeProc()
monkeypatch.setenv("HEADROOM_TARGET_RATIO", "0.20")
monkeypatch.setenv("HEADROOM_MAX_ITEMS", "12")
monkeypatch.setattr(wrap_mod, "_get_log_path", lambda: tmp_path / "proxy.log")
monkeypatch.setattr(wrap_mod, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_mod.subprocess, "Popen", fake_popen)
wrap_mod._start_proxy(8787, agent_type="codex")
env = popen_kwargs["env"]
assert isinstance(env, dict)
assert env["HEADROOM_TARGET_RATIO"] == "0.20"
assert env["HEADROOM_MAX_ITEMS"] == "12"
def test_launch_tool_ignores_sigint_in_wrapper(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Ctrl-C should be handled by the child CLI, not kill the proxy from wrapper."""
signal_handlers: dict[object, object] = {}
class FakeCompleted:
returncode = 0
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
monkeypatch.setattr(wrap_mod, "_ensure_proxy", lambda *args, **kwargs: (None, 8787))
monkeypatch.setattr(
wrap_mod.signal, "signal", lambda sig, fn: signal_handlers.setdefault(sig, fn)
)
monkeypatch.setattr(wrap_mod.subprocess, "run", lambda *args, **kwargs: FakeCompleted())
with pytest.raises(SystemExit) as exc:
wrap_mod._launch_tool(
binary="codex",
args=(),
env={},
port=8787,
no_proxy=True,
tool_label="CODEX",
env_vars_display=[],
)
assert exc.value.code == 0
assert signal_handlers[wrap_mod.signal.SIGINT] is wrap_mod._ignore_child_sigint
def test_wrap_codex_prepare_only_updates_stale_mcp_proxy_url(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_file = tmp_path / ".codex" / "config.toml"
config_file.parent.mkdir(parents=True)
config_file.write_text(
"# --- Headroom MCP server ---\n"
"[mcp_servers.headroom]\n"
'command = "headroom"\n'
'args = ["mcp", "serve"]\n'
"\n"
"[mcp_servers.headroom.env]\n"
'HEADROOM_PROXY_URL = "http://127.0.0.1:9000"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
"# --- end Headroom MCP server ---\n",
encoding="utf-8",
)
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
result = runner.invoke(main, ["wrap", "codex", "--prepare-only", "--port", "8787"])
assert result.exit_code == 0, result.output
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = config_file.read_text(encoding="utf-8")
fix(mcp): register managed installs with a resolvable headroom command (#1386) ## Description Managed Headroom installs can register the MCP server with a bare `headroom mcp serve` command even when the active runtime lives in a venv outside `PATH`. That leaves Claude and Codex with a registration they cannot re-launch reliably, and Claude eventually fails with `Failed to reconnect to headroom: ENOENT`. This PR reuses the existing runtime command resolver when building the shared Headroom MCP spec, so the generated registration follows the active install instead of assuming `headroom` is globally discoverable. It also updates the shared-builder and registrar tests so the proof rows now flow through `build_headroom_spec()` and prove the same resolved command contract on both the Claude CLI path and the Codex TOML path. A follow-up CI fix keeps the Docker init E2E expectation aligned with that same resolver-backed contract. Closes #487 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/mcp_registry/install.py`: build the Headroom MCP server spec from the canonical runtime command resolver instead of hardcoding `headroom mcp serve` - `tests/test_mcp_registry/test_install.py`: cover the shared builder's direct-binary and module-fallback command shapes - `tests/test_mcp_registry/test_claude_registrar.py`: prove the Claude CLI registration forwards the resolved command vector end to end - `tests/test_mcp_registry/test_codex_registrar.py`: prove the Codex registrar writes the same resolved command vector into TOML - `e2e/init/run.py`: derive the Docker init E2E's expected Claude MCP registration argv from `resolve_headroom_command()` so the CI harness follows the same runtime contract - `CHANGELOG.md`: note the managed-install MCP registration fix ## Testing - [x] Unit tests pass (`uv run pytest tests/test_mcp_registry/test_install.py -v`, `uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v`, `uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v`) - [x] Linting passes (`uv run ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) or explain N/A truthfully - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_mcp_registry/test_install.py -v ============================= 12 passed in 0.13s ============================== $ uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v ============================= 24 passed in 0.18s ============================== $ uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v ============================= 25 passed in 0.20s ============================== $ uv run python -c "from e2e.init.run import _expected_headroom_mcp_call; print(_expected_headroom_mcp_call('http://127.0.0.1:9011'))" ['mcp', 'add', 'headroom', '-s', 'user', '-e', 'HEADROOM_PROXY_URL=http://127.0.0.1:9011', '--', '.../headroom', 'mcp', 'serve'] $ uv run ruff check e2e/init/run.py All checks passed! $ uv run ruff format e2e/init/run.py --check 1 file already formatted $ uv run ruff check . All checks passed! $ uv run ruff format . --check 987 files already formatted ``` `uv run mypy headroom` was not run locally; this repo's focused local gate for the touched Python registry path is the targeted pytest set plus Ruff. ## Real Behavior Proof - Environment: managed-install-safe MCP registration path, Python 3.11+, no provider required - Exact command / steps: run the focused MCP registry pytest files, inspect the captured Claude CLI argv and rendered Codex TOML block, and verify the Docker init E2E expectation derives its Claude MCP argv from the same runtime helper - Observed result: the persisted MCP registration uses a resolvable command tied to the active Headroom runtime instead of bare `headroom`, while `HEADROOM_PROXY_URL` handling stays unchanged - Not tested: full live Claude reconnect against a real managed venv, unless that is run during implementation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Scoped to the MCP registration slice in `#487`. The RTK hook rewriting thread from the same issue is intentionally out of scope here. - `@erikpr1994` isolated the managed-install `ENOENT` failure mode in the issue thread and narrowed it to the bare-command MCP registration path. - If existing owned registrations with the old bare-command contract need an in-place upgrade path, that should be handled explicitly in the final diff rather than left implicit.
2026-06-27 00:39:00 -04:00
parsed = tomllib.loads(content)
expected = build_headroom_spec()
headroom_mcp = parsed["mcp_servers"]["headroom"]
assert "[mcp_servers.headroom]" in content
fix(mcp): register managed installs with a resolvable headroom command (#1386) ## Description Managed Headroom installs can register the MCP server with a bare `headroom mcp serve` command even when the active runtime lives in a venv outside `PATH`. That leaves Claude and Codex with a registration they cannot re-launch reliably, and Claude eventually fails with `Failed to reconnect to headroom: ENOENT`. This PR reuses the existing runtime command resolver when building the shared Headroom MCP spec, so the generated registration follows the active install instead of assuming `headroom` is globally discoverable. It also updates the shared-builder and registrar tests so the proof rows now flow through `build_headroom_spec()` and prove the same resolved command contract on both the Claude CLI path and the Codex TOML path. A follow-up CI fix keeps the Docker init E2E expectation aligned with that same resolver-backed contract. Closes #487 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/mcp_registry/install.py`: build the Headroom MCP server spec from the canonical runtime command resolver instead of hardcoding `headroom mcp serve` - `tests/test_mcp_registry/test_install.py`: cover the shared builder's direct-binary and module-fallback command shapes - `tests/test_mcp_registry/test_claude_registrar.py`: prove the Claude CLI registration forwards the resolved command vector end to end - `tests/test_mcp_registry/test_codex_registrar.py`: prove the Codex registrar writes the same resolved command vector into TOML - `e2e/init/run.py`: derive the Docker init E2E's expected Claude MCP registration argv from `resolve_headroom_command()` so the CI harness follows the same runtime contract - `CHANGELOG.md`: note the managed-install MCP registration fix ## Testing - [x] Unit tests pass (`uv run pytest tests/test_mcp_registry/test_install.py -v`, `uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v`, `uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v`) - [x] Linting passes (`uv run ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) or explain N/A truthfully - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_mcp_registry/test_install.py -v ============================= 12 passed in 0.13s ============================== $ uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v ============================= 24 passed in 0.18s ============================== $ uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v ============================= 25 passed in 0.20s ============================== $ uv run python -c "from e2e.init.run import _expected_headroom_mcp_call; print(_expected_headroom_mcp_call('http://127.0.0.1:9011'))" ['mcp', 'add', 'headroom', '-s', 'user', '-e', 'HEADROOM_PROXY_URL=http://127.0.0.1:9011', '--', '.../headroom', 'mcp', 'serve'] $ uv run ruff check e2e/init/run.py All checks passed! $ uv run ruff format e2e/init/run.py --check 1 file already formatted $ uv run ruff check . All checks passed! $ uv run ruff format . --check 987 files already formatted ``` `uv run mypy headroom` was not run locally; this repo's focused local gate for the touched Python registry path is the targeted pytest set plus Ruff. ## Real Behavior Proof - Environment: managed-install-safe MCP registration path, Python 3.11+, no provider required - Exact command / steps: run the focused MCP registry pytest files, inspect the captured Claude CLI argv and rendered Codex TOML block, and verify the Docker init E2E expectation derives its Claude MCP argv from the same runtime helper - Observed result: the persisted MCP registration uses a resolvable command tied to the active Headroom runtime instead of bare `headroom`, while `HEADROOM_PROXY_URL` handling stays unchanged - Not tested: full live Claude reconnect against a real managed venv, unless that is run during implementation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Scoped to the MCP registration slice in `#487`. The RTK hook rewriting thread from the same issue is intentionally out of scope here. - `@erikpr1994` isolated the managed-install `ENOENT` failure mode in the issue thread and narrowed it to the bare-command MCP registration path. - If existing owned registrations with the old bare-command contract need an in-place upgrade path, that should be handled explicitly in the final diff rather than left implicit.
2026-06-27 00:39:00 -04:00
assert headroom_mcp["command"] == expected.command
assert headroom_mcp["args"] == list(expected.args)
assert "env" not in headroom_mcp or "HEADROOM_PROXY_URL" not in headroom_mcp["env"]
assert "http://127.0.0.1:9000" not in content
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
def test_wrap_codex_memory_prepare_only_uses_local_db_without_persisting_it(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
monkeypatch.setenv("USER", "codex-user")
project_dir = tmp_path / "project-a"
project_dir.mkdir()
monkeypatch.chdir(project_dir)
backend_paths: list[str] = []
imported_users: list[str] = []
class FakeBackend:
async def _ensure_initialized(self) -> None:
return None
async def close(self) -> None:
return None
class FakeClaudeCodeAdapter:
def __init__(self, memory_dir: Path) -> None:
self.memory_dir = memory_dir
def fake_build_sync_backend(db_path: str) -> FakeBackend:
backend_paths.append(db_path)
return FakeBackend()
async def fake_sync_import(
backend: FakeBackend, adapter: FakeClaudeCodeAdapter, user_id: str
) -> int:
imported_users.append(user_id)
return 0
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
with patch("headroom.memory.sync._build_sync_backend", side_effect=fake_build_sync_backend):
with patch("headroom.memory.sync.sync_import", side_effect=fake_sync_import):
with patch(
"headroom.memory.sync_adapters.claude_code.ClaudeCodeAdapter",
FakeClaudeCodeAdapter,
):
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
with patch(
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
"headroom.memory.sync_adapters.claude_code.get_claude_memory_dir",
return_value=tmp_path / "claude-memory",
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
):
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
result = runner.invoke(
main,
[
"wrap",
"codex",
"--memory",
"--prepare-only",
"--no-mcp",
"--no-serena",
],
)
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
assert result.exit_code == 0, result.output
assert backend_paths == [str(project_dir / ".headroom" / "memory.db")]
assert imported_users == ["codex-user"]
content = (tmp_path / ".codex" / "config.toml").read_text()
assert "[mcp_servers.headroom_memory]" in content
assert '"--user", "codex-user"' in content
assert "--db" not in content
2026-05-09 22:57:35 -07:00
def test_wrap_codex_prepare_only_registers_serena_when_uvx_exists(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_file = tmp_path / ".codex" / "config.toml"
config_file.parent.mkdir(parents=True)
def fake_which(cmd: str) -> str | None:
if cmd == "uvx":
return "/usr/local/bin/uvx"
return None
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
with patch("headroom.cli.wrap.shutil.which", side_effect=fake_which):
# Serena is the code-memory MCP; assert it lands in the codex config.
result = runner.invoke(main, ["wrap", "codex", "--prepare-only"])
2026-05-09 22:57:35 -07:00
assert result.exit_code == 0, result.output
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = config_file.read_text(encoding="utf-8")
2026-05-09 22:57:35 -07:00
assert "[mcp_servers.serena]" in content
assert 'command = "uvx"' in content
assert '"--context", "codex"' in content
def test_wrap_codex_prepare_only_no_serena_skips_serena(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_file = tmp_path / ".codex" / "config.toml"
config_file.parent.mkdir(parents=True)
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
result = runner.invoke(main, ["wrap", "codex", "--prepare-only", "--no-serena"])
2026-05-09 22:57:35 -07:00
assert result.exit_code == 0, result.output
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert "[mcp_servers.serena]" not in config_file.read_text(encoding="utf-8")
2026-05-09 22:57:35 -07:00
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
def test_unwrap_codex_restores_prior_config_end_to_end(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""The bug report, reproduced: wrap → unwrap must round-trip cleanly."""
_set_test_home(monkeypatch, tmp_path)
config_file = tmp_path / ".codex" / "config.toml"
config_file.parent.mkdir(parents=True)
original = (
"[profiles.default]\n"
'model = "gpt-4o"\n'
"\n"
"[model_providers.openai]\n"
'base_url = "https://api.openai.com/v1"\n'
)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text(original, encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
wrap_result = runner.invoke(main, ["wrap", "codex", "--prepare-only", "--port", "8787"])
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert wrap_result.exit_code == 0, wrap_result.output
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert 'model_provider = "headroom"' in config_file.read_text(encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
stopped: list[int] = []
with patch(
"headroom.cli.wrap._stop_local_proxy_for_unwrap",
side_effect=lambda port: stopped.append(port) or "stopped",
):
unwrap_result = runner.invoke(main, ["unwrap", "codex", "--port", "9999"])
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert unwrap_result.exit_code == 0, unwrap_result.output
# Config must be byte-for-byte what the user had before wrap, and the
# injected block must be gone — no more "Missing OPENAI_API_KEY" when the
# proxy is stopped.
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
assert config_file.read_text(encoding="utf-8") == original
assert 'model_provider = "headroom"' not in config_file.read_text(encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert not (tmp_path / ".codex" / "config.toml.headroom-backup").exists()
assert stopped == [9999]
assert "Stopped local Headroom proxy on port 9999" in unwrap_result.output
def test_unwrap_codex_no_stop_proxy_leaves_proxy_alone(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
monkeypatch.setenv("CODEX_HOME", str(tmp_path / "explicit-codex-home"))
with patch("headroom.cli.wrap._stop_local_proxy_for_unwrap") as stop_proxy:
result = runner.invoke(main, ["unwrap", "codex", "--no-stop-proxy"])
assert result.exit_code == 0, result.output
stop_proxy.assert_not_called()
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
def test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
monkeypatch.setenv("USER", "codex-user")
project_dir = tmp_path / "project-a"
project_dir.mkdir()
monkeypatch.chdir(project_dir)
class FakeBackend:
async def _ensure_initialized(self) -> None:
return None
async def close(self) -> None:
return None
async def fake_sync_import(backend: FakeBackend, adapter: object, user_id: str) -> int:
return 0
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
with patch("headroom.memory.sync._build_sync_backend", return_value=FakeBackend()):
with patch("headroom.memory.sync.sync_import", side_effect=fake_sync_import):
with patch(
"headroom.memory.sync_adapters.claude_code.ClaudeCodeAdapter",
autospec=True,
):
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
with patch(
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
"headroom.memory.sync_adapters.claude_code.get_claude_memory_dir",
return_value=tmp_path / "claude-memory",
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
):
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
wrap_result = runner.invoke(
main,
[
"wrap",
"codex",
"--memory",
"--prepare-only",
"--no-mcp",
"--no-serena",
],
)
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
assert wrap_result.exit_code == 0, wrap_result.output
config_file = tmp_path / ".codex" / "config.toml"
content = config_file.read_text()
assert "[mcp_servers.headroom_memory]" in content
assert '"--user", "codex-user"' in content
with patch("headroom.cli.wrap._stop_local_proxy_for_unwrap") as stop_proxy:
unwrap_result = runner.invoke(main, ["unwrap", "codex", "--no-stop-proxy"])
assert unwrap_result.exit_code == 0, unwrap_result.output
assert not config_file.exists()
assert not (tmp_path / ".codex" / "config.toml.headroom-backup").exists()
stop_proxy.assert_not_called()
def test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
monkeypatch.setenv("USER", "codex-user")
project_dir = tmp_path / "project-a"
project_dir.mkdir()
monkeypatch.chdir(project_dir)
class FakeBackend:
async def _ensure_initialized(self) -> None:
return None
async def close(self) -> None:
return None
async def fake_sync_import(backend: FakeBackend, adapter: object, user_id: str) -> int:
return 0
def fake_which(cmd: str) -> str | None:
return None if cmd == "codex" else shutil.which(cmd)
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
with patch("headroom.cli.wrap.shutil.which", side_effect=fake_which):
with patch("headroom.memory.sync._build_sync_backend", return_value=FakeBackend()):
with patch("headroom.memory.sync.sync_import", side_effect=fake_sync_import):
with patch(
"headroom.memory.sync_adapters.claude_code.ClaudeCodeAdapter",
autospec=True,
):
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
with patch(
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
"headroom.memory.sync_adapters.claude_code.get_claude_memory_dir",
return_value=tmp_path / "claude-memory",
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
):
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
wrap_result = runner.invoke(
main,
["wrap", "codex", "--memory", "--no-mcp", "--no-serena"],
)
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
assert wrap_result.exit_code == 1
config_file = tmp_path / ".codex" / "config.toml"
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
assert not config_file.exists()
assert not (tmp_path / ".codex" / "config.toml.headroom-backup").exists()
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
with patch("headroom.cli.wrap._stop_local_proxy_for_unwrap") as stop_proxy:
unwrap_result = runner.invoke(main, ["unwrap", "codex", "--no-stop-proxy"])
assert unwrap_result.exit_code == 0, unwrap_result.output
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
assert "Nothing to undo" in unwrap_result.output
fix(codex): stop pinning Codex memory MCP to one project db (#1269) ## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 08:49:07 -04:00
stop_proxy.assert_not_called()
def test_stop_local_proxy_for_unwrap_kills_identified_headroom_proxy(
monkeypatch: pytest.MonkeyPatch,
) -> None:
killed: list[tuple[int, int]] = []
monkeypatch.setattr(wrap_mod, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_mod, "_query_proxy_config", lambda port: {"pid": "12345"})
monkeypatch.setattr(
wrap_mod,
"_kill_proxy_by_pid",
lambda pid, port: killed.append((pid, port)) or True,
)
assert wrap_mod._stop_local_proxy_for_unwrap(8787) == "stopped"
assert killed == [(12345, 8787)]
def test_stop_local_proxy_for_unwrap_refuses_unidentified_listener(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(wrap_mod, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_mod, "_query_proxy_config", lambda port: None)
with patch("headroom.cli.wrap._kill_proxy_by_pid") as kill_proxy:
assert wrap_mod._stop_local_proxy_for_unwrap(8787) == "unidentified"
kill_proxy.assert_not_called()
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
def test_unwrap_codex_is_safe_noop_with_explicit_codex_home(
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
monkeypatch.setenv("CODEX_HOME", str(tmp_path / "explicit-codex-home"))
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
result = runner.invoke(main, ["unwrap", "codex"])
assert result.exit_code == 0, result.output
assert "Nothing to undo" in result.output
fix(codex): respect CODEX_HOME for wrap config (#731) > ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`) Fixes #730. ## Summary - centralize Codex config path resolution in `headroom wrap codex` so it honors `CODEX_HOME` when set - make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of always writing to `~/.codex/config.toml` - route optional memory MCP and global Codex `AGENTS.md` injection through the same Codex home helper - make `headroom unwrap codex` print a warning, but still succeed, when `CODEX_HOME` is unset and the default Codex config has no Headroom markers - add regression coverage for provider injection, prepare-only wrapping, MCP registration under a custom Codex home, and the ambiguous unwrap warning - update `CHANGELOG.md` under `Unreleased > Bug Fixes` ## Real behavior proof Setup tested on: - Linux `7.0.10-2-cachyos` - Python 3.14.3 via `uv` - local fork branch `fix/codex-home` - custom Codex home created outside `~/.codex` Exact command run after the patch: ```bash tmp_home=$(mktemp -d) mkdir -p "$tmp_home/codex_custom" HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787 find "$tmp_home" -maxdepth 3 -type f -print | sort sed -n '1,140p' "$tmp_home/codex_custom/config.toml" test -e "$tmp_home/.codex/config.toml" && echo yes || echo no HOME="$tmp_home" USERPROFILE="$tmp_home" \ UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \ uv run --with fastapi --with uvicorn --with httpx --with websockets \ headroom unwrap codex --no-stop-proxy ``` After-fix evidence + observed result: Interactive check: - A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena` launch was tested locally after the patch and worked as expected. - The prepare-only proof below shows the same config path behavior without requiring an interactive Codex session in CI/reviewer environments. ```text MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running) Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml --- files --- /tmp/tmp.UPUvloGNYE/codex_custom/config.toml /tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup --- custom config --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- model_provider = "headroom" openai_base_url = "http://127.0.0.1:8787/v1" # --- end Headroom --- # --- Headroom MCP server --- [mcp_servers.headroom] command = "headroom" args = ["mcp", "serve"] # --- end Headroom MCP server --- # --- Headroom proxy (auto-injected by headroom wrap codex) --- [model_providers.headroom] name = "OpenAI via Headroom proxy" base_url = "http://127.0.0.1:8787/v1" supports_websockets = true # --- end Headroom --- --- default config exists? --- no Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex. Nothing to undo: .../.codex/config.toml has no Headroom wrap markers. ``` What I did not test: - Windows/macOS path behavior - full repository test suite, because this local Python 3.14 environment hits optional dependency/build constraints outside this patch ## Testing ```bash UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py ``` Results: ```text 63 passed, 1 warning in 1.48s All checks passed! 4 files already formatted ``` Notes: - Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the editable build. - `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing `uv.lock` has a GitPython wheel filename/version mismatch.
2026-06-10 20:21:29 -03:00
assert "Warning:" not in result.output
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert not (tmp_path / ".codex" / "config.toml").exists()
def test_unwrap_codex_removes_headroom_only_config_file(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
wrap_result = runner.invoke(main, ["wrap", "codex", "--prepare-only", "--port", "8787"])
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert wrap_result.exit_code == 0, wrap_result.output
config_file = tmp_path / ".codex" / "config.toml"
assert config_file.exists()
unwrap_result = runner.invoke(main, ["unwrap", "codex"])
assert unwrap_result.exit_code == 0, unwrap_result.output
assert not config_file.exists()
def test_unwrap_codex_preserves_unrelated_sections(
runner: CliRunner, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
config_file = tmp_path / ".codex" / "config.toml"
config_file.parent.mkdir(parents=True)
# A config with an MCP server the user configured by hand.
original = '[mcp_servers.local_thing]\ncommand = "/usr/local/bin/thing"\nargs = ["--serve"]\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text(original, encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
runner.invoke(main, ["wrap", "codex", "--prepare-only", "--port", "8787"])
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
result = runner.invoke(main, ["unwrap", "codex"])
assert result.exit_code == 0, result.output
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
restored = config_file.read_text(encoding="utf-8")
fix(wrap): unwrap codex restores prior config.toml `headroom wrap codex` injects a `model_provider = "headroom"` block plus a `[model_providers.headroom]` table into `~/.codex/config.toml` so Codex routes both HTTP and WebSocket traffic through the proxy. The matching `unwrap codex` subcommand did not exist, so the injected block stayed in `config.toml` forever — the moment the proxy stopped, Codex (CLI and macOS app) started erroring with `Missing environment variable: OPENAI_API_KEY`, and users had to hand- edit the file to recover. Fix: * `_inject_codex_provider_config` now snapshots the pre-wrap file to `~/.codex/config.toml.headroom-backup` before the first modification and leaves that snapshot untouched on subsequent wrap runs. The injection is also rewritten to use two self-contained marker- delimited blocks (top-level key and provider table) so stripping them never consumes user content that sits between them. * `_inject_memory_mcp_config` takes the same snapshot, so `wrap codex --memory` without a full provider injection is still fully reversible. * New `_restore_codex_provider_config` helper and `unwrap codex` click command: * backup present → restore byte-for-byte and delete the backup; * backup absent but Headroom block present → strip the block and keep surrounding user content; * config contained only Headroom content → remove the file so Codex falls back to defaults; * nothing to undo → safe no-op. Codex is the only wrap target that modifies a persistent user config file: claude/aider/cursor/copilot all go through env vars or project- scoped files only, so this bug was unique to Codex. Tests: * `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the strip/snapshot helpers directly, round-trip idempotency of wrap → wrap → unwrap, handling of malformed prior configs, and end-to-end CliRunner invocations of `headroom wrap codex --prepare-only` / `headroom unwrap codex` against a temp `$HOME`. * All 153 existing `tests/test_cli/` tests continue to pass. Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2) by the `sync-plugin-versions` pre-commit hook; the previous values (0.10.3) had drifted. Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on current `main` (0.11.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:13:54 -05:00
assert restored == original
feat(proxy): per-project savings breakdown on the dashboard (claude, codex, aider, copilot, cursor) (#803) ## Summary Adds a **per-project savings breakdown** to the proxy dashboard, covering **all wrap-supported agents: Claude Code, Codex, aider, Copilot, and Cursor**. Spec and rationale in #802 (feature-request issue — happy to adjust scope per maintainer feedback). How it works — two attribution channels, by client capability: **Header channel** (clients that can send custom headers): - `headroom wrap claude` appends `X-Headroom-Project: <basename(cwd)>` to `ANTHROPIC_CUSTOM_HEADERS` (a user-supplied x-headroom-project header always wins; other user headers preserved). - `headroom wrap codex` extends the injected `[model_providers.headroom]` block with `env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }` and sets `HEADROOM_PROJECT` per launch — Codex only sends the header when the env var is set, so the static config stays inert outside wrap. **Base-URL prefix channel** (clients that cannot send custom headers): - The proxy accepts `/p/<url-encoded-name>/...`; middleware strips the prefix before routing (before Starlette caches the URL) and binds the project. The explicit header wins over the prefix. - `headroom wrap aider` points `OPENAI_API_BASE` / `ANTHROPIC_BASE_URL` at the prefixed URL. - `headroom wrap copilot` points `COPILOT_PROVIDER_BASE_URL` at the prefixed URL (BYOK anthropic/openai provider types and the GitHub-subscription path). - `headroom wrap cursor` prints the prefixed Override Base URL in its setup instructions, with a note explaining the attribution. **Shared plumbing:** - New `headroom/proxy/project_context.py`: header classification, `/p/` prefix split/strip, URL-prefix builder, and a contextvar bound per request (HTTP middleware + WS accept for the Codex responses bridge); the outcome funnel resolves it (explicit `RequestOutcome.project` wins), stamps `RequestLog.tags["project"]`, and forwards it through `PrometheusMetrics.record_request` into the `SavingsTracker`. - `SavingsTracker`: persisted state gains a `projects` map (requests, tokens saved, savings USD, input tokens/cost, last activity). Schema v2 → v3 with transparent forward migration (v2 files load cleanly, `projects` starts empty). Names sanitized (printable-only, 128-char cap); map capped at 50 projects, evicting the smallest bucket. - `/stats` exposes `savings.per_project` (and `projects`/`projects_limit` inside `persistent_savings`); `/stats-history` exposes `projects`. - Dashboard gains a "Per-Project Savings" table mirroring the per-model table (Alpine `x-text` only — names are user-supplied, no HTML injection). **Behavior changes:** none for unattributed traffic — no header and no prefix means no bucket, aggregate totals exactly as before (regression-tested: legacy `/stats` and `/stats-history` shapes are pinned by tests). ## Real behavior proof **Setup tested on:** macOS (Darwin 25.5.0), Python 3.11.9, `pip install -e ".[dev]"`, proxy on spare ports with isolated `HEADROOM_SAVINGS_PATH`; real Claude Code CLI (subscription auth) and real Codex CLI (ChatGPT auth) as clients. **Header channel — exact steps:** ``` HEADROOM_SAVINGS_PATH=/tmp/headroom-proof-savings.json .venv/bin/python -m headroom.cli proxy --port 9123 # background cd /tmp/proof-alpha && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap codex --port 9123 --no-rtk --no-mcp --no-serena -- exec --skip-git-repo-check "Reply with exactly: OK" ``` **Observed** (copied live `/stats` output; all runs replied `OK` through the proxy): ```json { "proof-beta": { "requests": 3, "tokens_saved": 140, "compression_savings_usd": 0.0007, "total_input_tokens": 38581, "total_input_cost_usd": 0.360433, "last_activity_at": "2026-06-10T09:19:17Z", "savings_percent": 0.36 }, "proof-alpha": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 9050, "total_input_cost_usd": 0.32245, "last_activity_at": "2026-06-10T09:18:09Z", "savings_percent": 0.0 } } ``` `proof-beta` aggregates one Claude Code turn + one Codex `exec` run (Codex attribution flows through `env_http_headers`). **Prefix channel — exact steps** (the same mechanism the aider/copilot/cursor wraps emit, driven by a real client): ``` .venv/bin/python -m headroom.cli proxy --port 9124 # background, isolated savings path ANTHROPIC_BASE_URL="http://127.0.0.1:9124/p/aider-style-project" claude -p "Reply with exactly: OK" ``` **Observed:** ```json { "aider-style-project": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 8571, "total_input_cost_usd": 1.018575, "last_activity_at": "2026-06-10T10:10:13Z", "savings_percent": 0.0 } } ``` `/stats-history` returned `schema_version: 3` with the same `projects` keys; `GET /dashboard` HTML contains the new "Per-Project Savings" table; persisted state survived a tracker reload; a hand-written v2 savings file loaded cleanly with an empty `projects` map. **What I did NOT test live:** the actual aider/Copilot/Cursor binaries end-to-end (their wraps emit exactly the prefixed URLs exercised above — unit tests pin the emitted env/URLs); Codex subscription-mode routing via the built-in `openai` provider (no provider headers there — such traffic simply stays unattributed); multi-worker uvicorn; Windows. ## Tests - `tests/test_proxy_project_savings.py` (17 tests): sanitization, header classification, `/p/` prefix split + URL-builder round-trip, tracker aggregation/persistence/migration/cardinality-cap/state-sanitization, funnel→`/stats` end-to-end, middleware binding for header + prefix + precedence, plus regression tests pinning legacy `/stats`/`/stats-history` shape and unattributed-traffic totals. - Wrap/provider tests extended: `test_cli/test_wrap_helpers.py`, `test_cli/test_wrap_codex.py`, `test_provider_aider.py`, `test_provider_cursor.py`, `test_provider_copilot_wrap.py` (prefixed env/URLs, user-override wins, no duplicate header, TOML block contents, block strip, setup-line note). - Full `pytest` suite run locally; `ruff check` + `ruff format` clean on all touched files. - `CHANGELOG.md` updated. ## Dependencies None added or bumped. Closes #802 (pending maintainer 👍 per CONTRIBUTING — raised there first with the full spec). --------- Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
2026-06-11 04:04:45 +02:00
# ---------------------------------------------------------------------------
# Per-project savings: env_http_headers in the injected provider block
# ---------------------------------------------------------------------------
class TestCodexProjectHeaderConfig:
"""The injected provider maps X-Headroom-Project to HEADROOM_PROJECT.
Codex's ``env_http_headers`` sends a header only when the mapped env var
is set at Codex runtime, so `headroom wrap codex` exports
``HEADROOM_PROJECT`` and the proxy attributes savings per project.
"""
def test_inject_writes_env_http_headers_mapping(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_set_test_home(monkeypatch, tmp_path)
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = (tmp_path / ".codex" / "config.toml").read_text(encoding="utf-8")
feat(proxy): per-project savings breakdown on the dashboard (claude, codex, aider, copilot, cursor) (#803) ## Summary Adds a **per-project savings breakdown** to the proxy dashboard, covering **all wrap-supported agents: Claude Code, Codex, aider, Copilot, and Cursor**. Spec and rationale in #802 (feature-request issue — happy to adjust scope per maintainer feedback). How it works — two attribution channels, by client capability: **Header channel** (clients that can send custom headers): - `headroom wrap claude` appends `X-Headroom-Project: <basename(cwd)>` to `ANTHROPIC_CUSTOM_HEADERS` (a user-supplied x-headroom-project header always wins; other user headers preserved). - `headroom wrap codex` extends the injected `[model_providers.headroom]` block with `env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }` and sets `HEADROOM_PROJECT` per launch — Codex only sends the header when the env var is set, so the static config stays inert outside wrap. **Base-URL prefix channel** (clients that cannot send custom headers): - The proxy accepts `/p/<url-encoded-name>/...`; middleware strips the prefix before routing (before Starlette caches the URL) and binds the project. The explicit header wins over the prefix. - `headroom wrap aider` points `OPENAI_API_BASE` / `ANTHROPIC_BASE_URL` at the prefixed URL. - `headroom wrap copilot` points `COPILOT_PROVIDER_BASE_URL` at the prefixed URL (BYOK anthropic/openai provider types and the GitHub-subscription path). - `headroom wrap cursor` prints the prefixed Override Base URL in its setup instructions, with a note explaining the attribution. **Shared plumbing:** - New `headroom/proxy/project_context.py`: header classification, `/p/` prefix split/strip, URL-prefix builder, and a contextvar bound per request (HTTP middleware + WS accept for the Codex responses bridge); the outcome funnel resolves it (explicit `RequestOutcome.project` wins), stamps `RequestLog.tags["project"]`, and forwards it through `PrometheusMetrics.record_request` into the `SavingsTracker`. - `SavingsTracker`: persisted state gains a `projects` map (requests, tokens saved, savings USD, input tokens/cost, last activity). Schema v2 → v3 with transparent forward migration (v2 files load cleanly, `projects` starts empty). Names sanitized (printable-only, 128-char cap); map capped at 50 projects, evicting the smallest bucket. - `/stats` exposes `savings.per_project` (and `projects`/`projects_limit` inside `persistent_savings`); `/stats-history` exposes `projects`. - Dashboard gains a "Per-Project Savings" table mirroring the per-model table (Alpine `x-text` only — names are user-supplied, no HTML injection). **Behavior changes:** none for unattributed traffic — no header and no prefix means no bucket, aggregate totals exactly as before (regression-tested: legacy `/stats` and `/stats-history` shapes are pinned by tests). ## Real behavior proof **Setup tested on:** macOS (Darwin 25.5.0), Python 3.11.9, `pip install -e ".[dev]"`, proxy on spare ports with isolated `HEADROOM_SAVINGS_PATH`; real Claude Code CLI (subscription auth) and real Codex CLI (ChatGPT auth) as clients. **Header channel — exact steps:** ``` HEADROOM_SAVINGS_PATH=/tmp/headroom-proof-savings.json .venv/bin/python -m headroom.cli proxy --port 9123 # background cd /tmp/proof-alpha && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap codex --port 9123 --no-rtk --no-mcp --no-serena -- exec --skip-git-repo-check "Reply with exactly: OK" ``` **Observed** (copied live `/stats` output; all runs replied `OK` through the proxy): ```json { "proof-beta": { "requests": 3, "tokens_saved": 140, "compression_savings_usd": 0.0007, "total_input_tokens": 38581, "total_input_cost_usd": 0.360433, "last_activity_at": "2026-06-10T09:19:17Z", "savings_percent": 0.36 }, "proof-alpha": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 9050, "total_input_cost_usd": 0.32245, "last_activity_at": "2026-06-10T09:18:09Z", "savings_percent": 0.0 } } ``` `proof-beta` aggregates one Claude Code turn + one Codex `exec` run (Codex attribution flows through `env_http_headers`). **Prefix channel — exact steps** (the same mechanism the aider/copilot/cursor wraps emit, driven by a real client): ``` .venv/bin/python -m headroom.cli proxy --port 9124 # background, isolated savings path ANTHROPIC_BASE_URL="http://127.0.0.1:9124/p/aider-style-project" claude -p "Reply with exactly: OK" ``` **Observed:** ```json { "aider-style-project": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 8571, "total_input_cost_usd": 1.018575, "last_activity_at": "2026-06-10T10:10:13Z", "savings_percent": 0.0 } } ``` `/stats-history` returned `schema_version: 3` with the same `projects` keys; `GET /dashboard` HTML contains the new "Per-Project Savings" table; persisted state survived a tracker reload; a hand-written v2 savings file loaded cleanly with an empty `projects` map. **What I did NOT test live:** the actual aider/Copilot/Cursor binaries end-to-end (their wraps emit exactly the prefixed URLs exercised above — unit tests pin the emitted env/URLs); Codex subscription-mode routing via the built-in `openai` provider (no provider headers there — such traffic simply stays unattributed); multi-worker uvicorn; Windows. ## Tests - `tests/test_proxy_project_savings.py` (17 tests): sanitization, header classification, `/p/` prefix split + URL-builder round-trip, tracker aggregation/persistence/migration/cardinality-cap/state-sanitization, funnel→`/stats` end-to-end, middleware binding for header + prefix + precedence, plus regression tests pinning legacy `/stats`/`/stats-history` shape and unattributed-traffic totals. - Wrap/provider tests extended: `test_cli/test_wrap_helpers.py`, `test_cli/test_wrap_codex.py`, `test_provider_aider.py`, `test_provider_cursor.py`, `test_provider_copilot_wrap.py` (prefixed env/URLs, user-override wins, no duplicate header, TOML block contents, block strip, setup-line note). - Full `pytest` suite run locally; `ruff check` + `ruff format` clean on all touched files. - `CHANGELOG.md` updated. ## Dependencies None added or bumped. Closes #802 (pending maintainer 👍 per CONTRIBUTING — raised there first with the full spec). --------- Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
2026-06-11 04:04:45 +02:00
assert 'env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }' in content
def test_env_http_headers_inside_provider_section(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""The mapping must live inside [model_providers.headroom], before
the closing marker, so it applies to the Headroom provider."""
_set_test_home(monkeypatch, tmp_path)
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
content = (tmp_path / ".codex" / "config.toml").read_text(encoding="utf-8")
feat(proxy): per-project savings breakdown on the dashboard (claude, codex, aider, copilot, cursor) (#803) ## Summary Adds a **per-project savings breakdown** to the proxy dashboard, covering **all wrap-supported agents: Claude Code, Codex, aider, Copilot, and Cursor**. Spec and rationale in #802 (feature-request issue — happy to adjust scope per maintainer feedback). How it works — two attribution channels, by client capability: **Header channel** (clients that can send custom headers): - `headroom wrap claude` appends `X-Headroom-Project: <basename(cwd)>` to `ANTHROPIC_CUSTOM_HEADERS` (a user-supplied x-headroom-project header always wins; other user headers preserved). - `headroom wrap codex` extends the injected `[model_providers.headroom]` block with `env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }` and sets `HEADROOM_PROJECT` per launch — Codex only sends the header when the env var is set, so the static config stays inert outside wrap. **Base-URL prefix channel** (clients that cannot send custom headers): - The proxy accepts `/p/<url-encoded-name>/...`; middleware strips the prefix before routing (before Starlette caches the URL) and binds the project. The explicit header wins over the prefix. - `headroom wrap aider` points `OPENAI_API_BASE` / `ANTHROPIC_BASE_URL` at the prefixed URL. - `headroom wrap copilot` points `COPILOT_PROVIDER_BASE_URL` at the prefixed URL (BYOK anthropic/openai provider types and the GitHub-subscription path). - `headroom wrap cursor` prints the prefixed Override Base URL in its setup instructions, with a note explaining the attribution. **Shared plumbing:** - New `headroom/proxy/project_context.py`: header classification, `/p/` prefix split/strip, URL-prefix builder, and a contextvar bound per request (HTTP middleware + WS accept for the Codex responses bridge); the outcome funnel resolves it (explicit `RequestOutcome.project` wins), stamps `RequestLog.tags["project"]`, and forwards it through `PrometheusMetrics.record_request` into the `SavingsTracker`. - `SavingsTracker`: persisted state gains a `projects` map (requests, tokens saved, savings USD, input tokens/cost, last activity). Schema v2 → v3 with transparent forward migration (v2 files load cleanly, `projects` starts empty). Names sanitized (printable-only, 128-char cap); map capped at 50 projects, evicting the smallest bucket. - `/stats` exposes `savings.per_project` (and `projects`/`projects_limit` inside `persistent_savings`); `/stats-history` exposes `projects`. - Dashboard gains a "Per-Project Savings" table mirroring the per-model table (Alpine `x-text` only — names are user-supplied, no HTML injection). **Behavior changes:** none for unattributed traffic — no header and no prefix means no bucket, aggregate totals exactly as before (regression-tested: legacy `/stats` and `/stats-history` shapes are pinned by tests). ## Real behavior proof **Setup tested on:** macOS (Darwin 25.5.0), Python 3.11.9, `pip install -e ".[dev]"`, proxy on spare ports with isolated `HEADROOM_SAVINGS_PATH`; real Claude Code CLI (subscription auth) and real Codex CLI (ChatGPT auth) as clients. **Header channel — exact steps:** ``` HEADROOM_SAVINGS_PATH=/tmp/headroom-proof-savings.json .venv/bin/python -m headroom.cli proxy --port 9123 # background cd /tmp/proof-alpha && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap codex --port 9123 --no-rtk --no-mcp --no-serena -- exec --skip-git-repo-check "Reply with exactly: OK" ``` **Observed** (copied live `/stats` output; all runs replied `OK` through the proxy): ```json { "proof-beta": { "requests": 3, "tokens_saved": 140, "compression_savings_usd": 0.0007, "total_input_tokens": 38581, "total_input_cost_usd": 0.360433, "last_activity_at": "2026-06-10T09:19:17Z", "savings_percent": 0.36 }, "proof-alpha": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 9050, "total_input_cost_usd": 0.32245, "last_activity_at": "2026-06-10T09:18:09Z", "savings_percent": 0.0 } } ``` `proof-beta` aggregates one Claude Code turn + one Codex `exec` run (Codex attribution flows through `env_http_headers`). **Prefix channel — exact steps** (the same mechanism the aider/copilot/cursor wraps emit, driven by a real client): ``` .venv/bin/python -m headroom.cli proxy --port 9124 # background, isolated savings path ANTHROPIC_BASE_URL="http://127.0.0.1:9124/p/aider-style-project" claude -p "Reply with exactly: OK" ``` **Observed:** ```json { "aider-style-project": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 8571, "total_input_cost_usd": 1.018575, "last_activity_at": "2026-06-10T10:10:13Z", "savings_percent": 0.0 } } ``` `/stats-history` returned `schema_version: 3` with the same `projects` keys; `GET /dashboard` HTML contains the new "Per-Project Savings" table; persisted state survived a tracker reload; a hand-written v2 savings file loaded cleanly with an empty `projects` map. **What I did NOT test live:** the actual aider/Copilot/Cursor binaries end-to-end (their wraps emit exactly the prefixed URLs exercised above — unit tests pin the emitted env/URLs); Codex subscription-mode routing via the built-in `openai` provider (no provider headers there — such traffic simply stays unattributed); multi-worker uvicorn; Windows. ## Tests - `tests/test_proxy_project_savings.py` (17 tests): sanitization, header classification, `/p/` prefix split + URL-builder round-trip, tracker aggregation/persistence/migration/cardinality-cap/state-sanitization, funnel→`/stats` end-to-end, middleware binding for header + prefix + precedence, plus regression tests pinning legacy `/stats`/`/stats-history` shape and unattributed-traffic totals. - Wrap/provider tests extended: `test_cli/test_wrap_helpers.py`, `test_cli/test_wrap_codex.py`, `test_provider_aider.py`, `test_provider_cursor.py`, `test_provider_copilot_wrap.py` (prefixed env/URLs, user-override wins, no duplicate header, TOML block contents, block strip, setup-line note). - Full `pytest` suite run locally; `ruff check` + `ruff format` clean on all touched files. - `CHANGELOG.md` updated. ## Dependencies None added or bumped. Closes #802 (pending maintainer 👍 per CONTRIBUTING — raised there first with the full spec). --------- Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
2026-06-11 04:04:45 +02:00
section_start = content.index("[model_providers.headroom]")
mapping_pos = content.index("env_http_headers")
end_marker_pos = content.index(wrap_mod._CODEX_END_MARKER, section_start)
assert section_start < mapping_pos < end_marker_pos
def test_strip_removes_block_with_env_http_headers(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""_strip_codex_headroom_blocks removes the whole injected block,
including the new env_http_headers line, leaving user content."""
_set_test_home(monkeypatch, tmp_path)
config_dir = tmp_path / ".codex"
config_dir.mkdir()
config_file = config_dir / "config.toml"
original = '[profiles.default]\nmodel = "gpt-4o"\n'
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
config_file.write_text(original, encoding="utf-8")
feat(proxy): per-project savings breakdown on the dashboard (claude, codex, aider, copilot, cursor) (#803) ## Summary Adds a **per-project savings breakdown** to the proxy dashboard, covering **all wrap-supported agents: Claude Code, Codex, aider, Copilot, and Cursor**. Spec and rationale in #802 (feature-request issue — happy to adjust scope per maintainer feedback). How it works — two attribution channels, by client capability: **Header channel** (clients that can send custom headers): - `headroom wrap claude` appends `X-Headroom-Project: <basename(cwd)>` to `ANTHROPIC_CUSTOM_HEADERS` (a user-supplied x-headroom-project header always wins; other user headers preserved). - `headroom wrap codex` extends the injected `[model_providers.headroom]` block with `env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }` and sets `HEADROOM_PROJECT` per launch — Codex only sends the header when the env var is set, so the static config stays inert outside wrap. **Base-URL prefix channel** (clients that cannot send custom headers): - The proxy accepts `/p/<url-encoded-name>/...`; middleware strips the prefix before routing (before Starlette caches the URL) and binds the project. The explicit header wins over the prefix. - `headroom wrap aider` points `OPENAI_API_BASE` / `ANTHROPIC_BASE_URL` at the prefixed URL. - `headroom wrap copilot` points `COPILOT_PROVIDER_BASE_URL` at the prefixed URL (BYOK anthropic/openai provider types and the GitHub-subscription path). - `headroom wrap cursor` prints the prefixed Override Base URL in its setup instructions, with a note explaining the attribution. **Shared plumbing:** - New `headroom/proxy/project_context.py`: header classification, `/p/` prefix split/strip, URL-prefix builder, and a contextvar bound per request (HTTP middleware + WS accept for the Codex responses bridge); the outcome funnel resolves it (explicit `RequestOutcome.project` wins), stamps `RequestLog.tags["project"]`, and forwards it through `PrometheusMetrics.record_request` into the `SavingsTracker`. - `SavingsTracker`: persisted state gains a `projects` map (requests, tokens saved, savings USD, input tokens/cost, last activity). Schema v2 → v3 with transparent forward migration (v2 files load cleanly, `projects` starts empty). Names sanitized (printable-only, 128-char cap); map capped at 50 projects, evicting the smallest bucket. - `/stats` exposes `savings.per_project` (and `projects`/`projects_limit` inside `persistent_savings`); `/stats-history` exposes `projects`. - Dashboard gains a "Per-Project Savings" table mirroring the per-model table (Alpine `x-text` only — names are user-supplied, no HTML injection). **Behavior changes:** none for unattributed traffic — no header and no prefix means no bucket, aggregate totals exactly as before (regression-tested: legacy `/stats` and `/stats-history` shapes are pinned by tests). ## Real behavior proof **Setup tested on:** macOS (Darwin 25.5.0), Python 3.11.9, `pip install -e ".[dev]"`, proxy on spare ports with isolated `HEADROOM_SAVINGS_PATH`; real Claude Code CLI (subscription auth) and real Codex CLI (ChatGPT auth) as clients. **Header channel — exact steps:** ``` HEADROOM_SAVINGS_PATH=/tmp/headroom-proof-savings.json .venv/bin/python -m headroom.cli proxy --port 9123 # background cd /tmp/proof-alpha && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap codex --port 9123 --no-rtk --no-mcp --no-serena -- exec --skip-git-repo-check "Reply with exactly: OK" ``` **Observed** (copied live `/stats` output; all runs replied `OK` through the proxy): ```json { "proof-beta": { "requests": 3, "tokens_saved": 140, "compression_savings_usd": 0.0007, "total_input_tokens": 38581, "total_input_cost_usd": 0.360433, "last_activity_at": "2026-06-10T09:19:17Z", "savings_percent": 0.36 }, "proof-alpha": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 9050, "total_input_cost_usd": 0.32245, "last_activity_at": "2026-06-10T09:18:09Z", "savings_percent": 0.0 } } ``` `proof-beta` aggregates one Claude Code turn + one Codex `exec` run (Codex attribution flows through `env_http_headers`). **Prefix channel — exact steps** (the same mechanism the aider/copilot/cursor wraps emit, driven by a real client): ``` .venv/bin/python -m headroom.cli proxy --port 9124 # background, isolated savings path ANTHROPIC_BASE_URL="http://127.0.0.1:9124/p/aider-style-project" claude -p "Reply with exactly: OK" ``` **Observed:** ```json { "aider-style-project": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 8571, "total_input_cost_usd": 1.018575, "last_activity_at": "2026-06-10T10:10:13Z", "savings_percent": 0.0 } } ``` `/stats-history` returned `schema_version: 3` with the same `projects` keys; `GET /dashboard` HTML contains the new "Per-Project Savings" table; persisted state survived a tracker reload; a hand-written v2 savings file loaded cleanly with an empty `projects` map. **What I did NOT test live:** the actual aider/Copilot/Cursor binaries end-to-end (their wraps emit exactly the prefixed URLs exercised above — unit tests pin the emitted env/URLs); Codex subscription-mode routing via the built-in `openai` provider (no provider headers there — such traffic simply stays unattributed); multi-worker uvicorn; Windows. ## Tests - `tests/test_proxy_project_savings.py` (17 tests): sanitization, header classification, `/p/` prefix split + URL-builder round-trip, tracker aggregation/persistence/migration/cardinality-cap/state-sanitization, funnel→`/stats` end-to-end, middleware binding for header + prefix + precedence, plus regression tests pinning legacy `/stats`/`/stats-history` shape and unattributed-traffic totals. - Wrap/provider tests extended: `test_cli/test_wrap_helpers.py`, `test_cli/test_wrap_codex.py`, `test_provider_aider.py`, `test_provider_cursor.py`, `test_provider_copilot_wrap.py` (prefixed env/URLs, user-override wins, no duplicate header, TOML block contents, block strip, setup-line note). - Full `pytest` suite run locally; `ruff check` + `ruff format` clean on all touched files. - `CHANGELOG.md` updated. ## Dependencies None added or bumped. Closes #802 (pending maintainer 👍 per CONTRIBUTING — raised there first with the full spec). --------- Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
2026-06-11 04:04:45 +02:00
wrap_mod._inject_codex_provider_config(8787)
fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164) ## Description On Windows, `Path.read_text()` and `open()` default to the system locale encoding (cp1252, GBK, etc.) instead of UTF-8. This causes `UnicodeDecodeError` when reading or writing instruction files that contain multi-byte UTF-8 characters such as smart quotes or em dashes. The RTK instructions block itself contains an em dash (U+2014, `—`), so `_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or similar hint files. Closes #1126 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and `open()` calls in `headroom/cli/wrap.py` that handle instruction or config files (18 call sites) - Update test assertions in `test_wrap_hintfile_agents.py`, `test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with `encoding="utf-8"` - Add `test_inject_rtk_handles_utf8_content` verifying that existing hint files with smart quotes and em dashes survive RTK injection without crashing ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v 47 passed in 1.28s ``` ## Real Behavior Proof - Environment: Windows 11 China (GBK locale), Python 3.11, headroom main (f03e77b) - Exact command / steps: python -m pytest tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows) - Observed result: Before fix, test_prepare_only_injects_rtk_into_hintfile fails with UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block). After fix, all 12 hintfile tests pass including new UTF-8 round-trip test. - Not tested: no manual `headroom wrap copilot` run against a real Copilot installation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This is the same class of bug reported in #733 (GBK config.toml corruption). This PR fixes the `wrap.py` call sites; other modules (`learn/analyzer.py`, `install/providers.py`) have the same pattern and could benefit from the same treatment in a follow-up. --------- Signed-off-by: Yiming Zeng <yzeng424@gmail.com> Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-27 01:07:03 +08:00
wrapped = config_file.read_text(encoding="utf-8")
feat(proxy): per-project savings breakdown on the dashboard (claude, codex, aider, copilot, cursor) (#803) ## Summary Adds a **per-project savings breakdown** to the proxy dashboard, covering **all wrap-supported agents: Claude Code, Codex, aider, Copilot, and Cursor**. Spec and rationale in #802 (feature-request issue — happy to adjust scope per maintainer feedback). How it works — two attribution channels, by client capability: **Header channel** (clients that can send custom headers): - `headroom wrap claude` appends `X-Headroom-Project: <basename(cwd)>` to `ANTHROPIC_CUSTOM_HEADERS` (a user-supplied x-headroom-project header always wins; other user headers preserved). - `headroom wrap codex` extends the injected `[model_providers.headroom]` block with `env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }` and sets `HEADROOM_PROJECT` per launch — Codex only sends the header when the env var is set, so the static config stays inert outside wrap. **Base-URL prefix channel** (clients that cannot send custom headers): - The proxy accepts `/p/<url-encoded-name>/...`; middleware strips the prefix before routing (before Starlette caches the URL) and binds the project. The explicit header wins over the prefix. - `headroom wrap aider` points `OPENAI_API_BASE` / `ANTHROPIC_BASE_URL` at the prefixed URL. - `headroom wrap copilot` points `COPILOT_PROVIDER_BASE_URL` at the prefixed URL (BYOK anthropic/openai provider types and the GitHub-subscription path). - `headroom wrap cursor` prints the prefixed Override Base URL in its setup instructions, with a note explaining the attribution. **Shared plumbing:** - New `headroom/proxy/project_context.py`: header classification, `/p/` prefix split/strip, URL-prefix builder, and a contextvar bound per request (HTTP middleware + WS accept for the Codex responses bridge); the outcome funnel resolves it (explicit `RequestOutcome.project` wins), stamps `RequestLog.tags["project"]`, and forwards it through `PrometheusMetrics.record_request` into the `SavingsTracker`. - `SavingsTracker`: persisted state gains a `projects` map (requests, tokens saved, savings USD, input tokens/cost, last activity). Schema v2 → v3 with transparent forward migration (v2 files load cleanly, `projects` starts empty). Names sanitized (printable-only, 128-char cap); map capped at 50 projects, evicting the smallest bucket. - `/stats` exposes `savings.per_project` (and `projects`/`projects_limit` inside `persistent_savings`); `/stats-history` exposes `projects`. - Dashboard gains a "Per-Project Savings" table mirroring the per-model table (Alpine `x-text` only — names are user-supplied, no HTML injection). **Behavior changes:** none for unattributed traffic — no header and no prefix means no bucket, aggregate totals exactly as before (regression-tested: legacy `/stats` and `/stats-history` shapes are pinned by tests). ## Real behavior proof **Setup tested on:** macOS (Darwin 25.5.0), Python 3.11.9, `pip install -e ".[dev]"`, proxy on spare ports with isolated `HEADROOM_SAVINGS_PATH`; real Claude Code CLI (subscription auth) and real Codex CLI (ChatGPT auth) as clients. **Header channel — exact steps:** ``` HEADROOM_SAVINGS_PATH=/tmp/headroom-proof-savings.json .venv/bin/python -m headroom.cli proxy --port 9123 # background cd /tmp/proof-alpha && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap codex --port 9123 --no-rtk --no-mcp --no-serena -- exec --skip-git-repo-check "Reply with exactly: OK" ``` **Observed** (copied live `/stats` output; all runs replied `OK` through the proxy): ```json { "proof-beta": { "requests": 3, "tokens_saved": 140, "compression_savings_usd": 0.0007, "total_input_tokens": 38581, "total_input_cost_usd": 0.360433, "last_activity_at": "2026-06-10T09:19:17Z", "savings_percent": 0.36 }, "proof-alpha": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 9050, "total_input_cost_usd": 0.32245, "last_activity_at": "2026-06-10T09:18:09Z", "savings_percent": 0.0 } } ``` `proof-beta` aggregates one Claude Code turn + one Codex `exec` run (Codex attribution flows through `env_http_headers`). **Prefix channel — exact steps** (the same mechanism the aider/copilot/cursor wraps emit, driven by a real client): ``` .venv/bin/python -m headroom.cli proxy --port 9124 # background, isolated savings path ANTHROPIC_BASE_URL="http://127.0.0.1:9124/p/aider-style-project" claude -p "Reply with exactly: OK" ``` **Observed:** ```json { "aider-style-project": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 8571, "total_input_cost_usd": 1.018575, "last_activity_at": "2026-06-10T10:10:13Z", "savings_percent": 0.0 } } ``` `/stats-history` returned `schema_version: 3` with the same `projects` keys; `GET /dashboard` HTML contains the new "Per-Project Savings" table; persisted state survived a tracker reload; a hand-written v2 savings file loaded cleanly with an empty `projects` map. **What I did NOT test live:** the actual aider/Copilot/Cursor binaries end-to-end (their wraps emit exactly the prefixed URLs exercised above — unit tests pin the emitted env/URLs); Codex subscription-mode routing via the built-in `openai` provider (no provider headers there — such traffic simply stays unattributed); multi-worker uvicorn; Windows. ## Tests - `tests/test_proxy_project_savings.py` (17 tests): sanitization, header classification, `/p/` prefix split + URL-builder round-trip, tracker aggregation/persistence/migration/cardinality-cap/state-sanitization, funnel→`/stats` end-to-end, middleware binding for header + prefix + precedence, plus regression tests pinning legacy `/stats`/`/stats-history` shape and unattributed-traffic totals. - Wrap/provider tests extended: `test_cli/test_wrap_helpers.py`, `test_cli/test_wrap_codex.py`, `test_provider_aider.py`, `test_provider_cursor.py`, `test_provider_copilot_wrap.py` (prefixed env/URLs, user-override wins, no duplicate header, TOML block contents, block strip, setup-line note). - Full `pytest` suite run locally; `ruff check` + `ruff format` clean on all touched files. - `CHANGELOG.md` updated. ## Dependencies None added or bumped. Closes #802 (pending maintainer 👍 per CONTRIBUTING — raised there first with the full spec). --------- Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
2026-06-11 04:04:45 +02:00
assert "env_http_headers" in wrapped
cleaned = wrap_mod._strip_codex_headroom_blocks(wrapped)
assert "env_http_headers" not in cleaned
assert "X-Headroom-Project" not in cleaned
assert "[model_providers.headroom]" not in cleaned
assert 'model = "gpt-4o"' in cleaned
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
# ---------------------------------------------------------------------------
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
# Regression: codex preserves the requested port through the session-scoped runner
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
# ---------------------------------------------------------------------------
class TestCodexPortResolution:
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
"""codex() hands the requested port to the session-scoped wrap runner.
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
Regression for headroom#1406 round 2 review: the codex command must keep
the selected-port contract intact after the session-home refactor instead
of silently dropping or rewriting the requested port before the shared
launch path handles proxy reuse and fallback.
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
"""
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
def test_delegates_to_session_runner(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""codex() passes the requested port through to _run_codex_wrap."""
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
_set_test_home(monkeypatch, Path("/tmp/test_headroom_codex"))
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
call_kw: dict = {}
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
def mock_run_codex_wrap(**kwargs: object) -> None:
call_kw.update(kwargs)
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
monkeypatch.setattr(wrap_mod, "_run_codex_wrap", mock_run_codex_wrap)
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
runner = CliRunner()
result = runner.invoke(
main,
fix: remove rtk and lean-ctx CLI context tools (#2677) ## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt.
2026-07-30 22:59:41 -07:00
["wrap", "codex", "--port", "8787", "--no-mcp", "--no-serena"],
fix(wrap): replace stale-proxy detection with Vite-style port fallback (#1406) ## Description Replace the 180-line process-killing approach with a 15-line Vite-style socket.bind probe: if port is busy (EADDRINUSE or EACCES), try the next available port. Pure stdlib -- no /proc, no lsof, no subprocess -- works identically on Linux/macOS/Windows. ## Problem When `headroom wrap <agent>` is killed without proper cleanup (window close, SSH timeout, crash), the background proxy becomes orphaned and holds the port. The next `headroom wrap` on the same port would wait 30-45 seconds then fail with a confusing error. This PR takes a simpler, safer approach: find the next available port. No process detection, no killing. ### Related issues - **#589** (Port 8787 reserved by Windows) -- partially addressed: EACCES is now skipped together with EADDRINUSE - **#804** (Shared proxy killed by exiting session) -- already fixed upstream via `_live_proxy_clients` marker files; this PR doesn't touch that code ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: Add `_find_available_port()` -- socket.bind loop that skips EADDRINUSE (busy) and EACCES (reserved/privileged) ports, returns first available port in range. Replace `_ensure_proxy` port-bind check with auto-fallback call to `_find_available_port`. Remove `_ensure_port_free()` call from `_start_proxy()`. Remove 8 dead functions: `_find_process_on_port`, `_linux_find_process_on_port`, `_resolve_inode_to_pid`, `_is_headroom_proxy`, `_read_process_cmdline`, `_kill_process`, `_ensure_port_free`, `_format_unbindable_port_error`. - `tests/test_cli/test_wrap_helpers.py`: Remove 14 old `TestEnsurePortFree` tests (mocked /proc parsing, process killing). Add 6 new `TestFindAvailablePort` tests covering: port free, first port busy, multiple busy, EACCES skipped, unexpected error propagated, range exhausted. - `tests/test_cli/test_wrap_persistent.py`: Adapt persistence tests for `_find_available_port` mock. Rewrite unbindable-port test to use new error path. Zero new dependencies. Zero changes to core proxy server, MCP, compression, or providers. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_cli/ -v`) - [x] New tests added for new functionality ### Test Output ``` > python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v --no-header ============================= 6 passed ============================== test_port_free_returns_same PASSED test_port_busy_finds_next PASSED test_multiple_busy_ports PASSED test_propagates_unexpected_error PASSED test_propagates_eaddrinuse_with_eacces PASSED test_exhausts_range PASSED > python -m pytest tests/test_cli/ -q ============================= 445 passed in 8.40s ============================== ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.12.3 - Exact command / steps: Ran `python -m pytest tests/test_cli/test_wrap_helpers.py::TestFindAvailablePort -v` -- 6/6 pass for port fallback. Ran full test suite `python -m pytest tests/test_cli/ -q` -- 445/445 pass. - Observed result: `_find_available_port(8787)` returns 8787 when free, 8788 when 8787 is busy. EACCES skipped same as EADDRINUSE. Non-retryable errors (EADDRNOTAVAIL) propagate immediately. - Not tested: Windows EACCES fallback (no Windows CI runner). macOS port fallback (no macOS runner). Code path is identical across platforms (stdlib socket only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally
2026-07-08 01:10:52 +08:00
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
feat(codex): keep wrap routing session-scoped (#1507) ## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts.
2026-07-11 12:03:57 -04:00
assert call_kw.get("port") == 8787
assert call_kw.get("no_proxy") is False
assert call_kw.get("prepare_only") is False
fix(wrap/codex): export the detected custom upstream base URL (#2125) ## Description `headroom wrap codex` detects a user's custom upstream gateway but never tells Codex to use it, so the user's gateway key is sent to `api.openai.com`. `_inject_codex_provider_config` handles a Codex user who has an OpenAI-compatible gateway declared in `~/.codex/config.toml`, e.g. ```toml model_provider = "freemodel" [model_providers.freemodel] base_url = "https://api.freemodel.dev" ``` It injects the Headroom provider with `env_http_headers = { ... "X-Headroom-Base-Url" = "HEADROOM_CODEX_UPSTREAM_BASE_URL" }` and **returns the preserved upstream URL** so the caller can export it. Its docstring even says: *"Callers that go on to launch Codex should export this value into `HEADROOM_CODEX_UPSTREAM_BASE_URL`."* But `_prepare_codex_wrap_state` called it as a bare statement and discarded the return, and `_run_codex_wrap` / `_build_codex_launch_env` only ever set `OPENAI_BASE_URL`. A repo-wide grep confirms `HEADROOM_CODEX_UPSTREAM_BASE_URL` (`_UPSTREAM_BASE_URL_ENV_VAR`) is never assigned into any process env — it appears only at its definition and in that docstring. Since Codex only emits the `X-Headroom-Base-Url` header when the env var exists, the header is omitted, the proxy's OpenAI handler falls back to its hardcoded `https://api.openai.com`, and the user's `freemodel.dev` key is sent to OpenAI, which rejects it. This is a regression: the wiring existed in the original `#1614` fix (`_codex_custom_upstream = _inject_codex_provider_config(...)` then `env[_UPSTREAM_BASE_URL_ENV_VAR] = ...`) and was dropped by a later refactor that extracted `_prepare_codex_wrap_state`. ## Fix Restore the wiring: `_prepare_codex_wrap_state` now captures and returns `_inject_codex_provider_config`'s value, and `_run_codex_wrap` exports it into the launch env (`env[_UPSTREAM_BASE_URL_ENV_VAR] = custom_upstream`) when it is non-None and not already set, so a user-provided value still wins. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: `_prepare_codex_wrap_state` returns the detected custom upstream URL; `_run_codex_wrap` exports it into the launch env (and its display list) when set. - `tests/test_cli/test_wrap_codex.py`: add `TestCodexLaunchExportsCustomUpstream` — drives `_run_codex_wrap` with mocked prepare/launch and asserts the launch env carries `HEADROOM_CODEX_UPSTREAM_BASE_URL` when a custom upstream is detected, and does not when there isn't one. - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [x] Unit tests pass (`uv run --extra dev pytest tests/test_cli/test_wrap_codex.py::TestCodexLaunchExportsCustomUpstream -q`) - [x] Linting passes (`uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py headroom/memory/factory.py`) - [x] Type checking passes (`uvx mypy==1.20.2 headroom/memory/factory.py`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ python -m py_compile headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py OK ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing `headroom` pulls in the torch/transformers stack and running the CLI test locally OOM-kills this box, so I verified the wiring with a dependency-free script that models prepare -> run -> the proxy's upstream fallback, and left the full pytest (including the new CLI test) to CI. - Exact command / steps: modelled the old flow (inject return discarded) and the new flow (return exported into the launch env), then applied the proxy's rule that a missing `HEADROOM_CODEX_UPSTREAM_BASE_URL` falls back to `api.openai.com`. - Observed result: old effective upstream is `https://api.openai.com` (the gateway key is misrouted); new effective upstream is `https://api.freemodel.dev` (the user's gateway). The new CLI test asserts the launch env carries the var when a custom upstream is present and omits it otherwise. - Not tested: a live Codex process reading the env and emitting the header; full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes Merged current `main` to pick up the repository-wide mypy cache-key annotation fix, then verified the focused regression locally. the change threads one return value through two functions and exports it, verified by the wiring proof and a new CLI test that drives `_run_codex_wrap` with the heavy prepare/launch steps mocked so only the env-export logic is exercised. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-14 21:32:24 +05:30
class TestCodexLaunchExportsCustomUpstream:
"""`_run_codex_wrap` must export the detected custom upstream base URL into
the launch env so Codex emits the ``X-Headroom-Base-Url`` header. Otherwise
the proxy falls back to api.openai.com and the user's gateway key is sent to
the wrong host (regression of #1614)."""
def _launch_env(self, monkeypatch, tmp_path, *, custom_upstream):
captured: dict = {}
monkeypatch.setattr(wrap_mod.shutil, "which", lambda name: "/usr/bin/codex")
monkeypatch.setattr(wrap_mod, "_codex_home_dir", lambda: tmp_path)
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
monkeypatch.setattr(wrap_mod, "_offer_dangling_codex_recovery", lambda active_home: None)
monkeypatch.setattr(wrap_mod, "_prepare_codex_wrap_state", lambda **kwargs: None)
if custom_upstream:
(tmp_path / "config.toml").write_text(
"\n".join(
(
'model_provider = "gateway"',
"[model_providers.gateway]",
f'base_url = "{custom_upstream}"',
)
)
+ "\n",
encoding="utf-8",
)
fix(wrap/codex): export the detected custom upstream base URL (#2125) ## Description `headroom wrap codex` detects a user's custom upstream gateway but never tells Codex to use it, so the user's gateway key is sent to `api.openai.com`. `_inject_codex_provider_config` handles a Codex user who has an OpenAI-compatible gateway declared in `~/.codex/config.toml`, e.g. ```toml model_provider = "freemodel" [model_providers.freemodel] base_url = "https://api.freemodel.dev" ``` It injects the Headroom provider with `env_http_headers = { ... "X-Headroom-Base-Url" = "HEADROOM_CODEX_UPSTREAM_BASE_URL" }` and **returns the preserved upstream URL** so the caller can export it. Its docstring even says: *"Callers that go on to launch Codex should export this value into `HEADROOM_CODEX_UPSTREAM_BASE_URL`."* But `_prepare_codex_wrap_state` called it as a bare statement and discarded the return, and `_run_codex_wrap` / `_build_codex_launch_env` only ever set `OPENAI_BASE_URL`. A repo-wide grep confirms `HEADROOM_CODEX_UPSTREAM_BASE_URL` (`_UPSTREAM_BASE_URL_ENV_VAR`) is never assigned into any process env — it appears only at its definition and in that docstring. Since Codex only emits the `X-Headroom-Base-Url` header when the env var exists, the header is omitted, the proxy's OpenAI handler falls back to its hardcoded `https://api.openai.com`, and the user's `freemodel.dev` key is sent to OpenAI, which rejects it. This is a regression: the wiring existed in the original `#1614` fix (`_codex_custom_upstream = _inject_codex_provider_config(...)` then `env[_UPSTREAM_BASE_URL_ENV_VAR] = ...`) and was dropped by a later refactor that extracted `_prepare_codex_wrap_state`. ## Fix Restore the wiring: `_prepare_codex_wrap_state` now captures and returns `_inject_codex_provider_config`'s value, and `_run_codex_wrap` exports it into the launch env (`env[_UPSTREAM_BASE_URL_ENV_VAR] = custom_upstream`) when it is non-None and not already set, so a user-provided value still wins. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: `_prepare_codex_wrap_state` returns the detected custom upstream URL; `_run_codex_wrap` exports it into the launch env (and its display list) when set. - `tests/test_cli/test_wrap_codex.py`: add `TestCodexLaunchExportsCustomUpstream` — drives `_run_codex_wrap` with mocked prepare/launch and asserts the launch env carries `HEADROOM_CODEX_UPSTREAM_BASE_URL` when a custom upstream is detected, and does not when there isn't one. - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [x] Unit tests pass (`uv run --extra dev pytest tests/test_cli/test_wrap_codex.py::TestCodexLaunchExportsCustomUpstream -q`) - [x] Linting passes (`uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py headroom/memory/factory.py`) - [x] Type checking passes (`uvx mypy==1.20.2 headroom/memory/factory.py`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ python -m py_compile headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py OK ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing `headroom` pulls in the torch/transformers stack and running the CLI test locally OOM-kills this box, so I verified the wiring with a dependency-free script that models prepare -> run -> the proxy's upstream fallback, and left the full pytest (including the new CLI test) to CI. - Exact command / steps: modelled the old flow (inject return discarded) and the new flow (return exported into the launch env), then applied the proxy's rule that a missing `HEADROOM_CODEX_UPSTREAM_BASE_URL` falls back to `api.openai.com`. - Observed result: old effective upstream is `https://api.openai.com` (the gateway key is misrouted); new effective upstream is `https://api.freemodel.dev` (the user's gateway). The new CLI test asserts the launch env carries the var when a custom upstream is present and omits it otherwise. - Not tested: a live Codex process reading the env and emitting the header; full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes Merged current `main` to pick up the repository-wide mypy cache-key annotation fix, then verified the focused regression locally. the change threads one return value through two functions and exports it, verified by the wiring proof and a new CLI test that drives `_run_codex_wrap` with the heavy prepare/launch steps mocked so only the env-export logic is exercised. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-14 21:32:24 +05:30
fix(codex): preserve wrapped sessions and recover state (#2160) ## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head `2d89ecec`. - Exact command / steps: Run `pytest -q tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py`, then run `ruff check` and `ruff format --check` against `headroom/cli/wrap.py`, `headroom/cli/recover.py`, `headroom/providers/codex/recovery.py`, `tests/test_cli/test_wrap_codex.py`, and `tests/test_cli/test_recover_codex.py`. - Observed result: `122 passed in 10.08s`; Ruff reported `All checks passed!` and `5 files already formatted`. - Not tested: Launching a real Codex process or modifying a real user `CODEX_HOME`; these were intentionally excluded to protect live user state. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where the behavior is hard to understand - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass with my changes - [x] I have updated `CHANGELOG.md` if applicable ## Additional Notes The temporary-home behavior was introduced by #1507 in `ad9d086f43a664c4c2a19060b847f2e03ce4f6ad`. Related context: #730, #731, #961, #1034, #1050, #1349, #1853, #1889, #2103, and #2104. A temporary home that macOS or `TemporaryDirectory` already deleted cannot be reconstructed unless a retained `source-pinned/` copy exists. Recovery identifies genuine dangling SQLite paths, audits surviving durable history, and recovers any retained pinned source it can find. Prompt text without a rollout cannot reconstruct a full transcript. The unchecked changelog item is not applicable because this repository does not require a changelog entry for this fix. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:58:21 +02:00
def _fake_launch(*, env, port, configure_launch, args=(), env_vars_display=(), **kwargs):
if configure_launch is not None:
_args, env, _display = configure_launch(port, args, env, list(env_vars_display))
fix(wrap/codex): export the detected custom upstream base URL (#2125) ## Description `headroom wrap codex` detects a user's custom upstream gateway but never tells Codex to use it, so the user's gateway key is sent to `api.openai.com`. `_inject_codex_provider_config` handles a Codex user who has an OpenAI-compatible gateway declared in `~/.codex/config.toml`, e.g. ```toml model_provider = "freemodel" [model_providers.freemodel] base_url = "https://api.freemodel.dev" ``` It injects the Headroom provider with `env_http_headers = { ... "X-Headroom-Base-Url" = "HEADROOM_CODEX_UPSTREAM_BASE_URL" }` and **returns the preserved upstream URL** so the caller can export it. Its docstring even says: *"Callers that go on to launch Codex should export this value into `HEADROOM_CODEX_UPSTREAM_BASE_URL`."* But `_prepare_codex_wrap_state` called it as a bare statement and discarded the return, and `_run_codex_wrap` / `_build_codex_launch_env` only ever set `OPENAI_BASE_URL`. A repo-wide grep confirms `HEADROOM_CODEX_UPSTREAM_BASE_URL` (`_UPSTREAM_BASE_URL_ENV_VAR`) is never assigned into any process env — it appears only at its definition and in that docstring. Since Codex only emits the `X-Headroom-Base-Url` header when the env var exists, the header is omitted, the proxy's OpenAI handler falls back to its hardcoded `https://api.openai.com`, and the user's `freemodel.dev` key is sent to OpenAI, which rejects it. This is a regression: the wiring existed in the original `#1614` fix (`_codex_custom_upstream = _inject_codex_provider_config(...)` then `env[_UPSTREAM_BASE_URL_ENV_VAR] = ...`) and was dropped by a later refactor that extracted `_prepare_codex_wrap_state`. ## Fix Restore the wiring: `_prepare_codex_wrap_state` now captures and returns `_inject_codex_provider_config`'s value, and `_run_codex_wrap` exports it into the launch env (`env[_UPSTREAM_BASE_URL_ENV_VAR] = custom_upstream`) when it is non-None and not already set, so a user-provided value still wins. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: `_prepare_codex_wrap_state` returns the detected custom upstream URL; `_run_codex_wrap` exports it into the launch env (and its display list) when set. - `tests/test_cli/test_wrap_codex.py`: add `TestCodexLaunchExportsCustomUpstream` — drives `_run_codex_wrap` with mocked prepare/launch and asserts the launch env carries `HEADROOM_CODEX_UPSTREAM_BASE_URL` when a custom upstream is detected, and does not when there isn't one. - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [x] Unit tests pass (`uv run --extra dev pytest tests/test_cli/test_wrap_codex.py::TestCodexLaunchExportsCustomUpstream -q`) - [x] Linting passes (`uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py headroom/memory/factory.py`) - [x] Type checking passes (`uvx mypy==1.20.2 headroom/memory/factory.py`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ python -m py_compile headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py OK ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing `headroom` pulls in the torch/transformers stack and running the CLI test locally OOM-kills this box, so I verified the wiring with a dependency-free script that models prepare -> run -> the proxy's upstream fallback, and left the full pytest (including the new CLI test) to CI. - Exact command / steps: modelled the old flow (inject return discarded) and the new flow (return exported into the launch env), then applied the proxy's rule that a missing `HEADROOM_CODEX_UPSTREAM_BASE_URL` falls back to `api.openai.com`. - Observed result: old effective upstream is `https://api.openai.com` (the gateway key is misrouted); new effective upstream is `https://api.freemodel.dev` (the user's gateway). The new CLI test asserts the launch env carries the var when a custom upstream is present and omits it otherwise. - Not tested: a live Codex process reading the env and emitting the header; full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes Merged current `main` to pick up the repository-wide mypy cache-key annotation fix, then verified the focused regression locally. the change threads one return value through two functions and exports it, verified by the wiring proof and a new CLI test that drives `_run_codex_wrap` with the heavy prepare/launch steps mocked so only the env-export logic is exercised. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-14 21:32:24 +05:30
captured["env"] = env
monkeypatch.setattr(wrap_mod, "_launch_tool", _fake_launch)
wrap_mod._run_codex_wrap(
port=8787,
no_mcp=True,
no_tokensave=True,
serena=False,
no_serena=True,
code_graph=False,
no_proxy=True,
learn=False,
memory=False,
backend=None,
anyllm_provider=None,
region=None,
verbose=False,
prepare_only=False,
codex_args=(),
)
return captured["env"]
def test_custom_upstream_exported_into_launch_env(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
env = self._launch_env(monkeypatch, tmp_path, custom_upstream="https://api.freemodel.dev")
assert env[wrap_mod._UPSTREAM_BASE_URL_ENV_VAR] == "https://api.freemodel.dev"
def test_no_custom_upstream_leaves_env_var_unset(
self, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
env = self._launch_env(monkeypatch, tmp_path, custom_upstream=None)
assert wrap_mod._UPSTREAM_BASE_URL_ENV_VAR not in env