headroom/tests/test_wrap_code_memory.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

211 lines
8.7 KiB
Python
Raw Permalink Normal View History

refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) ## Description `tokensave` was a **downloaded third-party Rust binary** (`aovestdipaperino/tokensave`) that `headroom wrap` registered as a code-graph MCP server. This removes it entirely and standardises on **Serena** as the code-memory MCP — which was already the default in `wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads or executes a binary of its own for code memory. The change is a *removal + safe transition*, not a behaviour flip: Serena was already the default, so existing users move over automatically. This PR also folds in a small README repositioning (Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx are third-party tools we don't control), since it's the same tooling-stack story. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - **Removed the external tool:** `headroom/graph/tokensave_installer.py` (the download-and-execute path), `build_tokensave_spec`, and the `_ensure_tokensave_binary` / `_index_tokensave_project` / `_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`. - **`--code-memory`** now offers `serena` (default) or `none` — the `tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as its (default-on) code-memory MCP. - **Tombstone / transition (ledger-verified):** `headroom wrap` **and** `headroom unwrap` remove a previously Headroom-installed `tokensave` MCP entry so upgrading users stop launching it, and print that the leftover `~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to delete. A user-managed `tokensave` entry is left untouched. Mirrors the existing `codebase-memory-mcp` retirement. - **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and `--no-tokensave` resolve to Serena instead of erroring; `--no-serena` now means "no code memory". **No state migration needed** — both tools' indexes are regenerable caches of the source, so Serena simply re-indexes. - **Docs/README:** replaced the "tokensave binary trust model" section with a Serena note + an "Upgrading from tokensave?" callout; retired the RTK "first-class part of our stack" framing. - **Tests:** deleted the tokensave-only test files (`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`, `test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py` for the new resolver/dispatch, fixed a codex test that patched a removed symbol. Net: **+102 / −1187 lines.** ## Testing - [x] Unit tests pass (`pytest`) — targeted to the affected areas - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py All checks passed! $ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py Success: no issues found in 12 source files $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \ tests/test_cli/test_wrap_claude_vertex_proxy_env.py \ tests/test_cli/test_wrap_claude_finally_unbound.py -q ======================== 120 passed in 97.86s (0:01:37) ======================== $ pytest tests/test_cli --collect-only -q ========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal ``` ## Real Behavior Proof - **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on branch `tejas/remove-tokensave`. - **Exact command / steps:** - `python -c "from headroom.integrations.strands.bundle import HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False, enable_serena_mcp=False); print(len(b.tools))"` → confirms the module imports after `build_tokensave_spec` removal (the import that my change would otherwise break). - CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`) writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context", "codex"`) to the codex config and **no** tokensave entry. - `_resolve_code_memory` unit tests confirm: default → `serena`; `HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`; `--code-memory bogus` → `ClickException`. - **Observed result:** import OK (`tools: 0`); Serena registered, tokensave absent; resolver behaves as above; 120/120 tests pass. - **Not tested:** a live `headroom wrap` against a real agent on a machine with a *previously-installed* tokensave MCP entry — the tombstone-removal path is covered by unit tests with a fake registrar/ledger, not an end-to-end run. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes - `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden, deprecated flags (no-ops or mapped) so existing scripts don't break. - `--code-graph` is unchanged — it's the proxy's live file-watcher flag and was never the tokensave MCP; only the dead tokensave hook behind it was removed. - `CHANGELOG.md` intentionally left untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-22 20:59:24 -07:00
"""Code-memory MCP is selectable via --code-memory (default serena).
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
Covers the resolver precedence (selector > deprecated flags > default), the
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) ## Description `tokensave` was a **downloaded third-party Rust binary** (`aovestdipaperino/tokensave`) that `headroom wrap` registered as a code-graph MCP server. This removes it entirely and standardises on **Serena** as the code-memory MCP — which was already the default in `wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads or executes a binary of its own for code memory. The change is a *removal + safe transition*, not a behaviour flip: Serena was already the default, so existing users move over automatically. This PR also folds in a small README repositioning (Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx are third-party tools we don't control), since it's the same tooling-stack story. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - **Removed the external tool:** `headroom/graph/tokensave_installer.py` (the download-and-execute path), `build_tokensave_spec`, and the `_ensure_tokensave_binary` / `_index_tokensave_project` / `_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`. - **`--code-memory`** now offers `serena` (default) or `none` — the `tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as its (default-on) code-memory MCP. - **Tombstone / transition (ledger-verified):** `headroom wrap` **and** `headroom unwrap` remove a previously Headroom-installed `tokensave` MCP entry so upgrading users stop launching it, and print that the leftover `~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to delete. A user-managed `tokensave` entry is left untouched. Mirrors the existing `codebase-memory-mcp` retirement. - **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and `--no-tokensave` resolve to Serena instead of erroring; `--no-serena` now means "no code memory". **No state migration needed** — both tools' indexes are regenerable caches of the source, so Serena simply re-indexes. - **Docs/README:** replaced the "tokensave binary trust model" section with a Serena note + an "Upgrading from tokensave?" callout; retired the RTK "first-class part of our stack" framing. - **Tests:** deleted the tokensave-only test files (`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`, `test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py` for the new resolver/dispatch, fixed a codex test that patched a removed symbol. Net: **+102 / −1187 lines.** ## Testing - [x] Unit tests pass (`pytest`) — targeted to the affected areas - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py All checks passed! $ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py Success: no issues found in 12 source files $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \ tests/test_cli/test_wrap_claude_vertex_proxy_env.py \ tests/test_cli/test_wrap_claude_finally_unbound.py -q ======================== 120 passed in 97.86s (0:01:37) ======================== $ pytest tests/test_cli --collect-only -q ========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal ``` ## Real Behavior Proof - **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on branch `tejas/remove-tokensave`. - **Exact command / steps:** - `python -c "from headroom.integrations.strands.bundle import HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False, enable_serena_mcp=False); print(len(b.tools))"` → confirms the module imports after `build_tokensave_spec` removal (the import that my change would otherwise break). - CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`) writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context", "codex"`) to the codex config and **no** tokensave entry. - `_resolve_code_memory` unit tests confirm: default → `serena`; `HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`; `--code-memory bogus` → `ClickException`. - **Observed result:** import OK (`tools: 0`); Serena registered, tokensave absent; resolver behaves as above; 120/120 tests pass. - **Not tested:** a live `headroom wrap` against a real agent on a machine with a *previously-installed* tokensave MCP entry — the tombstone-removal path is covered by unit tests with a fake registrar/ledger, not an end-to-end run. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes - `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden, deprecated flags (no-ops or mapped) so existing scripts don't break. - `--code-graph` is unchanged — it's the proxy's live file-watcher flag and was never the tokensave MCP; only the dead tokensave hook behind it was removed. - `CHANGELOG.md` intentionally left untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-22 20:59:24 -07:00
graceful retirement of the removed ``tokensave`` option, the orchestrator
dispatch for each selection, and that --code-memory is exposed on the
code-memory-capable subcommands (claude/codex/grok) but not others.
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
"""
from __future__ import annotations
import os
from unittest.mock import patch
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) ## Description `tokensave` was a **downloaded third-party Rust binary** (`aovestdipaperino/tokensave`) that `headroom wrap` registered as a code-graph MCP server. This removes it entirely and standardises on **Serena** as the code-memory MCP — which was already the default in `wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads or executes a binary of its own for code memory. The change is a *removal + safe transition*, not a behaviour flip: Serena was already the default, so existing users move over automatically. This PR also folds in a small README repositioning (Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx are third-party tools we don't control), since it's the same tooling-stack story. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - **Removed the external tool:** `headroom/graph/tokensave_installer.py` (the download-and-execute path), `build_tokensave_spec`, and the `_ensure_tokensave_binary` / `_index_tokensave_project` / `_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`. - **`--code-memory`** now offers `serena` (default) or `none` — the `tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as its (default-on) code-memory MCP. - **Tombstone / transition (ledger-verified):** `headroom wrap` **and** `headroom unwrap` remove a previously Headroom-installed `tokensave` MCP entry so upgrading users stop launching it, and print that the leftover `~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to delete. A user-managed `tokensave` entry is left untouched. Mirrors the existing `codebase-memory-mcp` retirement. - **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and `--no-tokensave` resolve to Serena instead of erroring; `--no-serena` now means "no code memory". **No state migration needed** — both tools' indexes are regenerable caches of the source, so Serena simply re-indexes. - **Docs/README:** replaced the "tokensave binary trust model" section with a Serena note + an "Upgrading from tokensave?" callout; retired the RTK "first-class part of our stack" framing. - **Tests:** deleted the tokensave-only test files (`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`, `test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py` for the new resolver/dispatch, fixed a codex test that patched a removed symbol. Net: **+102 / −1187 lines.** ## Testing - [x] Unit tests pass (`pytest`) — targeted to the affected areas - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py All checks passed! $ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py Success: no issues found in 12 source files $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \ tests/test_cli/test_wrap_claude_vertex_proxy_env.py \ tests/test_cli/test_wrap_claude_finally_unbound.py -q ======================== 120 passed in 97.86s (0:01:37) ======================== $ pytest tests/test_cli --collect-only -q ========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal ``` ## Real Behavior Proof - **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on branch `tejas/remove-tokensave`. - **Exact command / steps:** - `python -c "from headroom.integrations.strands.bundle import HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False, enable_serena_mcp=False); print(len(b.tools))"` → confirms the module imports after `build_tokensave_spec` removal (the import that my change would otherwise break). - CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`) writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context", "codex"`) to the codex config and **no** tokensave entry. - `_resolve_code_memory` unit tests confirm: default → `serena`; `HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`; `--code-memory bogus` → `ClickException`. - **Observed result:** import OK (`tools: 0`); Serena registered, tokensave absent; resolver behaves as above; 120/120 tests pass. - **Not tested:** a live `headroom wrap` against a real agent on a machine with a *previously-installed* tokensave MCP entry — the tombstone-removal path is covered by unit tests with a fake registrar/ledger, not an end-to-end run. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes - `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden, deprecated flags (no-ops or mapped) so existing scripts don't break. - `--code-graph` is unchanged — it's the proxy's live file-watcher flag and was never the tokensave MCP; only the dead tokensave hook behind it was removed. - `CHANGELOG.md` intentionally left untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-22 20:59:24 -07:00
import click
fix(wrap/serena): stop creating serena_config.yml, unbricking Serena on fresh installs (#2676) ## Description `_ensure_serena_dashboard_disabled()` wrote a one-key bootstrap config (`web_dashboard_open_on_launch: false`) into `~/.serena/serena_config.yml` when the file was absent, assuming Serena fills in any key it omits. Verified against **Serena 1.6.2.dev0** (`serena/config/serena_config.py`), that holds for every field except one. Serena autogenerates its own complete config **only when the path does not exist**: ```python if not os.path.exists(config_file_path): cls._generate_config_file(config_file_path) ``` Once any file is present it validates instead. Every other field falls back to a dataclass default via `get_value_or_default`, but a missing `projects` key is fatal (~line 1064): ``` SerenaConfigError: `projects` key not found in Serena configuration. ``` So Headroom's own bootstrap file killed Serena on **every machine without a pre-existing Serena config**. The MCP server exited during handshake — surfacing as `connection closed: initialize response` on Codex and a bare `MCP error -32000: Connection closed` on OpenCode (#2674) — and `serena project index` failed identically. Headroom now leaves that file to Serena. That is immune to Serena adding required keys later; guessing the schema is what caused the outage. The popup never needed the file anyway: `build_serena_spec` passes `--open-web-dashboard False`, which Serena applies *after* loading the config (`serena/mcp.py:361` — `config.web_dashboard_open_on_launch = open_web_dashboard`), so the flag wins regardless of what is on disk. An **existing** config is still edited in place — dashboard key flipped, `projects: []` backfilled to repair machines an affected version already wrote — preserving a populated `projects` list, other keys and comments. Closes #2674 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - `_ensure_serena_dashboard_disabled()` never creates `serena_config.yml`; it only edits an existing one, and backfills `projects: []` there to repair already-broken machines. - Dropped `_scope_serena_languages` + `_detect_repo_languages` + `_EXT_TO_SERENA_LANGUAGE` (**−133 lines**). Dead weight: Serena determines languages itself in `ProjectConfig.autogenerate` (`_determine_project_language_servers`) and records them under `language_servers` — `languages`, which Headroom wrote, is a legacy name Serena migrates via `RENAMED_FIELDS`. Serena's generated file uses a block-style list, so our single-line-flow regex never matched it: on any Serena-generated `project.yml` the function was a **verified no-op**. The only case where it acted was creating the file — the same partial-config trap — which also skipped the `project.local.yml` sidecar Serena writes alongside. - **Test isolation:** the MCP install ledger defaults to `~/.headroom/mcp_installs.json`, so any test registering a server wrote into the developer's real ledger (observed adding a live `claude/serena` entry during a local run). `conftest.py` now redirects it per-test. - **Repo config:** `.serena/project.yml` carried a stale `project_name` (`"feature-opencode-wrap"`) and listed only `typescript`, so Serena's symbol index skipped 1331 Python and 194 Rust files for every contributor. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_serena_boost.py \ tests/test_cli/test_serena_migrate.py tests/test_cli/test_serena_disable.py -q 35 passed, 1 skipped in 0.84s $ SERENA_SRC=<serena checkout> pytest tests/test_wrap_code_memory.py -q 13 passed in 0.62s # the skipped test runs when a Serena source tree is available $ ruff check headroom/ tests/ --exclude headroom/dashboard/templates All checks passed! $ mypy headroom/cli/wrap.py Success: no issues found in 1 source file ``` New tests. The key one asserts the invariant rather than our own key list, so it stays correct even if Serena adds a required key — a test pinning `projects: []` would keep passing while users broke again: - `test_serena_config_is_never_created_by_headroom` — Headroom must not pre-empt Serena's bootstrap - `test_serena_dashboard_disabled_repairs_config_missing_projects` — heals a config an affected version wrote - `test_serena_dashboard_disabled_preserves_registered_projects` — never clobbers the real registry; comments kept, no duplicate key - `test_serena_dashboard_disabled_is_idempotent` - `test_serena_config_required_keys_match_serena_source` — reads Serena's real source and pins the two facts this fix rests on (bootstrap-only-when-absent, `projects` is the sole fatal omission). Skipped unless `SERENA_SRC` is set; deliberately **not** named `HEADROOM_*` because `conftest.py` scrubs that namespace, which would make it silently always-skip. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Serena 1.6.2.dev0 via `uvx --from git+https://github.com/oraios/serena`, Codex CLI 0.146.0. - **Exact command / steps:** a probe doing a real JSON-RPC `initialize` handshake against the exact command `headroom wrap` registers — i.e. what Codex/OpenCode actually do — in a throwaway `HOME` per case. (A) pre-seeded with the one-line config an affected version wrote; (B) no config; (C) real `headroom wrap codex --prepare-only`, then handshake. - **Observed result:** ```text === A. BROKEN: single-key config (Headroom 0.33.0) === MCP handshake: FAIL — no initialize response (exit=1). stderr tail: File ".../serena/config/serena_config.py", line 1064, in from_config_file raise SerenaConfigError("`projects` key not found in Serena configuration. ...") serena.config.serena_config.SerenaConfigError: `projects` key not found ... config after run: 1 lines, has 'projects': False === B. FIXED: no config, Serena bootstraps it === MCP handshake: PASS — initialize OK — serverInfo.name='Serena' config after run: 213 lines, has 'projects': True === C. FULL FLOW: real `headroom wrap codex` then handshake === Serena: no serena_config.yml yet — letting Serena generate it Serena MCP: registered (restart OpenAI Codex CLI if it was already running) Serena: project pre-indexed (symbol cache warmed) serena_config.yml: 213 lines, written by Serena (correct) MCP handshake: PASS — initialize OK — serverInfo.name='Serena' --- verdict --- A (broken config) started: False <- expected False B (fixed, no config) started: True <- expected True C (after real wrap) started: True <- expected True ``` A second `wrap` in the same HOME flips the dashboard without damage: `true` → `false`, `projects` intact, all 153 comment lines intact. The writer was isolated against a pristine 213-line Serena config: **delta 0 newlines**. - **Not tested:** Windows and Linux (macOS only); Serena versions other than 1.6.2.dev0; the JetBrains language backend. The probe needs network + `uvx` (~2 min) so it is a manual verification tool, not wired into CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - **Docs:** N/A — no user-facing docs described the `serena_config.yml` bootstrap or the language scoping. - Also fixes the OpenCode report (#2674). The Codex-side report of the same root cause quotes the `SerenaConfigError` verbatim; OpenCode only surfaces the generic `-32000`, which is why it read as two different bugs. - Users already broken by an affected version are repaired automatically on their next `headroom wrap` — no manual `serena_config.yml` edit needed. - A stacked PR removing the rtk/lean-ctx CLI context tools is based on this branch; this one is deliberately small so it can land first.
2026-07-30 20:55:47 -07:00
import pytest
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
from click.testing import CliRunner
from headroom.cli import wrap
def _clean_env() -> dict[str, str]:
env = dict(os.environ)
env.pop("HEADROOM_CODE_MEMORY", None)
return env
def test_default_is_serena() -> None:
with patch.dict(os.environ, _clean_env(), clear=True):
assert wrap._resolve_code_memory({}) == wrap._CODE_MEMORY_SERENA
def test_selector_env_wins() -> None:
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) ## Description `tokensave` was a **downloaded third-party Rust binary** (`aovestdipaperino/tokensave`) that `headroom wrap` registered as a code-graph MCP server. This removes it entirely and standardises on **Serena** as the code-memory MCP — which was already the default in `wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads or executes a binary of its own for code memory. The change is a *removal + safe transition*, not a behaviour flip: Serena was already the default, so existing users move over automatically. This PR also folds in a small README repositioning (Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx are third-party tools we don't control), since it's the same tooling-stack story. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - **Removed the external tool:** `headroom/graph/tokensave_installer.py` (the download-and-execute path), `build_tokensave_spec`, and the `_ensure_tokensave_binary` / `_index_tokensave_project` / `_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`. - **`--code-memory`** now offers `serena` (default) or `none` — the `tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as its (default-on) code-memory MCP. - **Tombstone / transition (ledger-verified):** `headroom wrap` **and** `headroom unwrap` remove a previously Headroom-installed `tokensave` MCP entry so upgrading users stop launching it, and print that the leftover `~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to delete. A user-managed `tokensave` entry is left untouched. Mirrors the existing `codebase-memory-mcp` retirement. - **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and `--no-tokensave` resolve to Serena instead of erroring; `--no-serena` now means "no code memory". **No state migration needed** — both tools' indexes are regenerable caches of the source, so Serena simply re-indexes. - **Docs/README:** replaced the "tokensave binary trust model" section with a Serena note + an "Upgrading from tokensave?" callout; retired the RTK "first-class part of our stack" framing. - **Tests:** deleted the tokensave-only test files (`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`, `test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py` for the new resolver/dispatch, fixed a codex test that patched a removed symbol. Net: **+102 / −1187 lines.** ## Testing - [x] Unit tests pass (`pytest`) — targeted to the affected areas - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py All checks passed! $ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py Success: no issues found in 12 source files $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \ tests/test_cli/test_wrap_claude_vertex_proxy_env.py \ tests/test_cli/test_wrap_claude_finally_unbound.py -q ======================== 120 passed in 97.86s (0:01:37) ======================== $ pytest tests/test_cli --collect-only -q ========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal ``` ## Real Behavior Proof - **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on branch `tejas/remove-tokensave`. - **Exact command / steps:** - `python -c "from headroom.integrations.strands.bundle import HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False, enable_serena_mcp=False); print(len(b.tools))"` → confirms the module imports after `build_tokensave_spec` removal (the import that my change would otherwise break). - CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`) writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context", "codex"`) to the codex config and **no** tokensave entry. - `_resolve_code_memory` unit tests confirm: default → `serena`; `HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`; `--code-memory bogus` → `ClickException`. - **Observed result:** import OK (`tools: 0`); Serena registered, tokensave absent; resolver behaves as above; 120/120 tests pass. - **Not tested:** a live `headroom wrap` against a real agent on a machine with a *previously-installed* tokensave MCP entry — the tombstone-removal path is covered by unit tests with a fake registrar/ledger, not an end-to-end run. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes - `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden, deprecated flags (no-ops or mapped) so existing scripts don't break. - `--code-graph` is unchanged — it's the proxy's live file-watcher flag and was never the tokensave MCP; only the dead tokensave hook behind it was removed. - `CHANGELOG.md` intentionally left untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-22 20:59:24 -07:00
for val in (wrap._CODE_MEMORY_SERENA, wrap._CODE_MEMORY_NONE):
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
with patch.dict(os.environ, {"HEADROOM_CODE_MEMORY": val}):
# selector beats any legacy flag
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) ## Description `tokensave` was a **downloaded third-party Rust binary** (`aovestdipaperino/tokensave`) that `headroom wrap` registered as a code-graph MCP server. This removes it entirely and standardises on **Serena** as the code-memory MCP — which was already the default in `wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads or executes a binary of its own for code memory. The change is a *removal + safe transition*, not a behaviour flip: Serena was already the default, so existing users move over automatically. This PR also folds in a small README repositioning (Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx are third-party tools we don't control), since it's the same tooling-stack story. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - **Removed the external tool:** `headroom/graph/tokensave_installer.py` (the download-and-execute path), `build_tokensave_spec`, and the `_ensure_tokensave_binary` / `_index_tokensave_project` / `_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`. - **`--code-memory`** now offers `serena` (default) or `none` — the `tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as its (default-on) code-memory MCP. - **Tombstone / transition (ledger-verified):** `headroom wrap` **and** `headroom unwrap` remove a previously Headroom-installed `tokensave` MCP entry so upgrading users stop launching it, and print that the leftover `~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to delete. A user-managed `tokensave` entry is left untouched. Mirrors the existing `codebase-memory-mcp` retirement. - **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and `--no-tokensave` resolve to Serena instead of erroring; `--no-serena` now means "no code memory". **No state migration needed** — both tools' indexes are regenerable caches of the source, so Serena simply re-indexes. - **Docs/README:** replaced the "tokensave binary trust model" section with a Serena note + an "Upgrading from tokensave?" callout; retired the RTK "first-class part of our stack" framing. - **Tests:** deleted the tokensave-only test files (`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`, `test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py` for the new resolver/dispatch, fixed a codex test that patched a removed symbol. Net: **+102 / −1187 lines.** ## Testing - [x] Unit tests pass (`pytest`) — targeted to the affected areas - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py All checks passed! $ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py Success: no issues found in 12 source files $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \ tests/test_cli/test_wrap_claude_vertex_proxy_env.py \ tests/test_cli/test_wrap_claude_finally_unbound.py -q ======================== 120 passed in 97.86s (0:01:37) ======================== $ pytest tests/test_cli --collect-only -q ========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal ``` ## Real Behavior Proof - **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on branch `tejas/remove-tokensave`. - **Exact command / steps:** - `python -c "from headroom.integrations.strands.bundle import HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False, enable_serena_mcp=False); print(len(b.tools))"` → confirms the module imports after `build_tokensave_spec` removal (the import that my change would otherwise break). - CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`) writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context", "codex"`) to the codex config and **no** tokensave entry. - `_resolve_code_memory` unit tests confirm: default → `serena`; `HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`; `--code-memory bogus` → `ClickException`. - **Observed result:** import OK (`tools: 0`); Serena registered, tokensave absent; resolver behaves as above; 120/120 tests pass. - **Not tested:** a live `headroom wrap` against a real agent on a machine with a *previously-installed* tokensave MCP entry — the tombstone-removal path is covered by unit tests with a fake registrar/ledger, not an end-to-end run. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes - `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden, deprecated flags (no-ops or mapped) so existing scripts don't break. - `--code-graph` is unchanged — it's the proxy's live file-watcher flag and was never the tokensave MCP; only the dead tokensave hook behind it was removed. - `CHANGELOG.md` intentionally left untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-22 20:59:24 -07:00
assert wrap._resolve_code_memory({"serena": True, "no_serena": True}) == val
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
def test_deprecated_flags_map_into_selector() -> None:
with patch.dict(os.environ, _clean_env(), clear=True):
assert wrap._resolve_code_memory({"serena": True}) == wrap._CODE_MEMORY_SERENA
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) ## Description `tokensave` was a **downloaded third-party Rust binary** (`aovestdipaperino/tokensave`) that `headroom wrap` registered as a code-graph MCP server. This removes it entirely and standardises on **Serena** as the code-memory MCP — which was already the default in `wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads or executes a binary of its own for code memory. The change is a *removal + safe transition*, not a behaviour flip: Serena was already the default, so existing users move over automatically. This PR also folds in a small README repositioning (Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx are third-party tools we don't control), since it's the same tooling-stack story. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - **Removed the external tool:** `headroom/graph/tokensave_installer.py` (the download-and-execute path), `build_tokensave_spec`, and the `_ensure_tokensave_binary` / `_index_tokensave_project` / `_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`. - **`--code-memory`** now offers `serena` (default) or `none` — the `tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as its (default-on) code-memory MCP. - **Tombstone / transition (ledger-verified):** `headroom wrap` **and** `headroom unwrap` remove a previously Headroom-installed `tokensave` MCP entry so upgrading users stop launching it, and print that the leftover `~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to delete. A user-managed `tokensave` entry is left untouched. Mirrors the existing `codebase-memory-mcp` retirement. - **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and `--no-tokensave` resolve to Serena instead of erroring; `--no-serena` now means "no code memory". **No state migration needed** — both tools' indexes are regenerable caches of the source, so Serena simply re-indexes. - **Docs/README:** replaced the "tokensave binary trust model" section with a Serena note + an "Upgrading from tokensave?" callout; retired the RTK "first-class part of our stack" framing. - **Tests:** deleted the tokensave-only test files (`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`, `test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py` for the new resolver/dispatch, fixed a codex test that patched a removed symbol. Net: **+102 / −1187 lines.** ## Testing - [x] Unit tests pass (`pytest`) — targeted to the affected areas - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py All checks passed! $ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py Success: no issues found in 12 source files $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \ tests/test_cli/test_wrap_claude_vertex_proxy_env.py \ tests/test_cli/test_wrap_claude_finally_unbound.py -q ======================== 120 passed in 97.86s (0:01:37) ======================== $ pytest tests/test_cli --collect-only -q ========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal ``` ## Real Behavior Proof - **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on branch `tejas/remove-tokensave`. - **Exact command / steps:** - `python -c "from headroom.integrations.strands.bundle import HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False, enable_serena_mcp=False); print(len(b.tools))"` → confirms the module imports after `build_tokensave_spec` removal (the import that my change would otherwise break). - CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`) writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context", "codex"`) to the codex config and **no** tokensave entry. - `_resolve_code_memory` unit tests confirm: default → `serena`; `HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`; `--code-memory bogus` → `ClickException`. - **Observed result:** import OK (`tools: 0`); Serena registered, tokensave absent; resolver behaves as above; 120/120 tests pass. - **Not tested:** a live `headroom wrap` against a real agent on a machine with a *previously-installed* tokensave MCP entry — the tombstone-removal path is covered by unit tests with a fake registrar/ledger, not an end-to-end run. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes - `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden, deprecated flags (no-ops or mapped) so existing scripts don't break. - `--code-graph` is unchanged — it's the proxy's live file-watcher flag and was never the tokensave MCP; only the dead tokensave hook behind it was removed. - `CHANGELOG.md` intentionally left untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-22 20:59:24 -07:00
# tokensave is retired: --no-tokensave is now a no-op → default serena
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
assert wrap._resolve_code_memory({"no_tokensave": True}) == wrap._CODE_MEMORY_SERENA
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) ## Description `tokensave` was a **downloaded third-party Rust binary** (`aovestdipaperino/tokensave`) that `headroom wrap` registered as a code-graph MCP server. This removes it entirely and standardises on **Serena** as the code-memory MCP — which was already the default in `wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads or executes a binary of its own for code memory. The change is a *removal + safe transition*, not a behaviour flip: Serena was already the default, so existing users move over automatically. This PR also folds in a small README repositioning (Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx are third-party tools we don't control), since it's the same tooling-stack story. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - **Removed the external tool:** `headroom/graph/tokensave_installer.py` (the download-and-execute path), `build_tokensave_spec`, and the `_ensure_tokensave_binary` / `_index_tokensave_project` / `_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`. - **`--code-memory`** now offers `serena` (default) or `none` — the `tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as its (default-on) code-memory MCP. - **Tombstone / transition (ledger-verified):** `headroom wrap` **and** `headroom unwrap` remove a previously Headroom-installed `tokensave` MCP entry so upgrading users stop launching it, and print that the leftover `~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to delete. A user-managed `tokensave` entry is left untouched. Mirrors the existing `codebase-memory-mcp` retirement. - **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and `--no-tokensave` resolve to Serena instead of erroring; `--no-serena` now means "no code memory". **No state migration needed** — both tools' indexes are regenerable caches of the source, so Serena simply re-indexes. - **Docs/README:** replaced the "tokensave binary trust model" section with a Serena note + an "Upgrading from tokensave?" callout; retired the RTK "first-class part of our stack" framing. - **Tests:** deleted the tokensave-only test files (`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`, `test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py` for the new resolver/dispatch, fixed a codex test that patched a removed symbol. Net: **+102 / −1187 lines.** ## Testing - [x] Unit tests pass (`pytest`) — targeted to the affected areas - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py All checks passed! $ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py Success: no issues found in 12 source files $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \ tests/test_cli/test_wrap_claude_vertex_proxy_env.py \ tests/test_cli/test_wrap_claude_finally_unbound.py -q ======================== 120 passed in 97.86s (0:01:37) ======================== $ pytest tests/test_cli --collect-only -q ========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal ``` ## Real Behavior Proof - **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on branch `tejas/remove-tokensave`. - **Exact command / steps:** - `python -c "from headroom.integrations.strands.bundle import HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False, enable_serena_mcp=False); print(len(b.tools))"` → confirms the module imports after `build_tokensave_spec` removal (the import that my change would otherwise break). - CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`) writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context", "codex"`) to the codex config and **no** tokensave entry. - `_resolve_code_memory` unit tests confirm: default → `serena`; `HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`; `--code-memory bogus` → `ClickException`. - **Observed result:** import OK (`tools: 0`); Serena registered, tokensave absent; resolver behaves as above; 120/120 tests pass. - **Not tested:** a live `headroom wrap` against a real agent on a machine with a *previously-installed* tokensave MCP entry — the tombstone-removal path is covered by unit tests with a fake registrar/ledger, not an end-to-end run. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes - `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden, deprecated flags (no-ops or mapped) so existing scripts don't break. - `--code-graph` is unchanged — it's the proxy's live file-watcher flag and was never the tokensave MCP; only the dead tokensave hook behind it was removed. - `CHANGELOG.md` intentionally left untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-22 20:59:24 -07:00
# --no-serena means "no code memory" now that tokensave is gone
assert wrap._resolve_code_memory({"no_serena": True}) == wrap._CODE_MEMORY_NONE
def test_retired_tokensave_selector_maps_to_serena() -> None:
# An explicit HEADROOM_CODE_MEMORY=tokensave (or --code-memory tokensave from
# an old script) degrades gracefully to Serena instead of erroring.
with patch.dict(os.environ, {"HEADROOM_CODE_MEMORY": "tokensave"}):
assert wrap._resolve_code_memory({}) == wrap._CODE_MEMORY_SERENA
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
def test_serena_dashboard_disabled_flips_existing_config(tmp_path, monkeypatch) -> None:
monkeypatch.setenv("HOME", str(tmp_path))
cfg = tmp_path / ".serena" / "serena_config.yml"
cfg.parent.mkdir(parents=True)
cfg.write_text(
"web_dashboard: true\nweb_dashboard_open_on_launch: true\ngui_log_window: false\n"
)
wrap._ensure_serena_dashboard_disabled()
text = cfg.read_text()
assert "web_dashboard_open_on_launch: false" in text
assert "web_dashboard: true" in text # other keys preserved
fix(wrap/serena): stop creating serena_config.yml, unbricking Serena on fresh installs (#2676) ## Description `_ensure_serena_dashboard_disabled()` wrote a one-key bootstrap config (`web_dashboard_open_on_launch: false`) into `~/.serena/serena_config.yml` when the file was absent, assuming Serena fills in any key it omits. Verified against **Serena 1.6.2.dev0** (`serena/config/serena_config.py`), that holds for every field except one. Serena autogenerates its own complete config **only when the path does not exist**: ```python if not os.path.exists(config_file_path): cls._generate_config_file(config_file_path) ``` Once any file is present it validates instead. Every other field falls back to a dataclass default via `get_value_or_default`, but a missing `projects` key is fatal (~line 1064): ``` SerenaConfigError: `projects` key not found in Serena configuration. ``` So Headroom's own bootstrap file killed Serena on **every machine without a pre-existing Serena config**. The MCP server exited during handshake — surfacing as `connection closed: initialize response` on Codex and a bare `MCP error -32000: Connection closed` on OpenCode (#2674) — and `serena project index` failed identically. Headroom now leaves that file to Serena. That is immune to Serena adding required keys later; guessing the schema is what caused the outage. The popup never needed the file anyway: `build_serena_spec` passes `--open-web-dashboard False`, which Serena applies *after* loading the config (`serena/mcp.py:361` — `config.web_dashboard_open_on_launch = open_web_dashboard`), so the flag wins regardless of what is on disk. An **existing** config is still edited in place — dashboard key flipped, `projects: []` backfilled to repair machines an affected version already wrote — preserving a populated `projects` list, other keys and comments. Closes #2674 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - `_ensure_serena_dashboard_disabled()` never creates `serena_config.yml`; it only edits an existing one, and backfills `projects: []` there to repair already-broken machines. - Dropped `_scope_serena_languages` + `_detect_repo_languages` + `_EXT_TO_SERENA_LANGUAGE` (**−133 lines**). Dead weight: Serena determines languages itself in `ProjectConfig.autogenerate` (`_determine_project_language_servers`) and records them under `language_servers` — `languages`, which Headroom wrote, is a legacy name Serena migrates via `RENAMED_FIELDS`. Serena's generated file uses a block-style list, so our single-line-flow regex never matched it: on any Serena-generated `project.yml` the function was a **verified no-op**. The only case where it acted was creating the file — the same partial-config trap — which also skipped the `project.local.yml` sidecar Serena writes alongside. - **Test isolation:** the MCP install ledger defaults to `~/.headroom/mcp_installs.json`, so any test registering a server wrote into the developer's real ledger (observed adding a live `claude/serena` entry during a local run). `conftest.py` now redirects it per-test. - **Repo config:** `.serena/project.yml` carried a stale `project_name` (`"feature-opencode-wrap"`) and listed only `typescript`, so Serena's symbol index skipped 1331 Python and 194 Rust files for every contributor. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_serena_boost.py \ tests/test_cli/test_serena_migrate.py tests/test_cli/test_serena_disable.py -q 35 passed, 1 skipped in 0.84s $ SERENA_SRC=<serena checkout> pytest tests/test_wrap_code_memory.py -q 13 passed in 0.62s # the skipped test runs when a Serena source tree is available $ ruff check headroom/ tests/ --exclude headroom/dashboard/templates All checks passed! $ mypy headroom/cli/wrap.py Success: no issues found in 1 source file ``` New tests. The key one asserts the invariant rather than our own key list, so it stays correct even if Serena adds a required key — a test pinning `projects: []` would keep passing while users broke again: - `test_serena_config_is_never_created_by_headroom` — Headroom must not pre-empt Serena's bootstrap - `test_serena_dashboard_disabled_repairs_config_missing_projects` — heals a config an affected version wrote - `test_serena_dashboard_disabled_preserves_registered_projects` — never clobbers the real registry; comments kept, no duplicate key - `test_serena_dashboard_disabled_is_idempotent` - `test_serena_config_required_keys_match_serena_source` — reads Serena's real source and pins the two facts this fix rests on (bootstrap-only-when-absent, `projects` is the sole fatal omission). Skipped unless `SERENA_SRC` is set; deliberately **not** named `HEADROOM_*` because `conftest.py` scrubs that namespace, which would make it silently always-skip. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Serena 1.6.2.dev0 via `uvx --from git+https://github.com/oraios/serena`, Codex CLI 0.146.0. - **Exact command / steps:** a probe doing a real JSON-RPC `initialize` handshake against the exact command `headroom wrap` registers — i.e. what Codex/OpenCode actually do — in a throwaway `HOME` per case. (A) pre-seeded with the one-line config an affected version wrote; (B) no config; (C) real `headroom wrap codex --prepare-only`, then handshake. - **Observed result:** ```text === A. BROKEN: single-key config (Headroom 0.33.0) === MCP handshake: FAIL — no initialize response (exit=1). stderr tail: File ".../serena/config/serena_config.py", line 1064, in from_config_file raise SerenaConfigError("`projects` key not found in Serena configuration. ...") serena.config.serena_config.SerenaConfigError: `projects` key not found ... config after run: 1 lines, has 'projects': False === B. FIXED: no config, Serena bootstraps it === MCP handshake: PASS — initialize OK — serverInfo.name='Serena' config after run: 213 lines, has 'projects': True === C. FULL FLOW: real `headroom wrap codex` then handshake === Serena: no serena_config.yml yet — letting Serena generate it Serena MCP: registered (restart OpenAI Codex CLI if it was already running) Serena: project pre-indexed (symbol cache warmed) serena_config.yml: 213 lines, written by Serena (correct) MCP handshake: PASS — initialize OK — serverInfo.name='Serena' --- verdict --- A (broken config) started: False <- expected False B (fixed, no config) started: True <- expected True C (after real wrap) started: True <- expected True ``` A second `wrap` in the same HOME flips the dashboard without damage: `true` → `false`, `projects` intact, all 153 comment lines intact. The writer was isolated against a pristine 213-line Serena config: **delta 0 newlines**. - **Not tested:** Windows and Linux (macOS only); Serena versions other than 1.6.2.dev0; the JetBrains language backend. The probe needs network + `uvx` (~2 min) so it is a manual verification tool, not wired into CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - **Docs:** N/A — no user-facing docs described the `serena_config.yml` bootstrap or the language scoping. - Also fixes the OpenCode report (#2674). The Codex-side report of the same root cause quotes the `SerenaConfigError` verbatim; OpenCode only surfaces the generic `-32000`, which is why it read as two different bugs. - Users already broken by an affected version are repaired automatically on their next `headroom wrap` — no manual `serena_config.yml` edit needed. - A stacked PR removing the rtk/lean-ctx CLI context tools is based on this branch; this one is deliberately small so it can land first.
2026-07-30 20:55:47 -07:00
def test_serena_config_is_never_created_by_headroom(tmp_path, monkeypatch) -> None:
"""Headroom must NOT pre-empt Serena's own config bootstrap (#2674).
This is the exact invariant, and it is the reason the outage happened.
Verified against Serena 1.6.2.dev0 ``serena/config/serena_config.py``: Serena
autogenerates a complete config only when the path does not exist; once any
file is there it validates instead, and a missing ``projects`` key is fatal
(``SerenaConfigError``). Headroom used to write a one-key bootstrap file,
which killed Serena's MCP handshake on every fresh install.
Asserting "we write nothing" is stronger than asserting which keys we write:
it stays correct even if Serena adds a new required key, whereas a test that
pins our own key list would go on passing while users broke again.
"""
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
monkeypatch.setenv("HOME", str(tmp_path))
fix(wrap/serena): stop creating serena_config.yml, unbricking Serena on fresh installs (#2676) ## Description `_ensure_serena_dashboard_disabled()` wrote a one-key bootstrap config (`web_dashboard_open_on_launch: false`) into `~/.serena/serena_config.yml` when the file was absent, assuming Serena fills in any key it omits. Verified against **Serena 1.6.2.dev0** (`serena/config/serena_config.py`), that holds for every field except one. Serena autogenerates its own complete config **only when the path does not exist**: ```python if not os.path.exists(config_file_path): cls._generate_config_file(config_file_path) ``` Once any file is present it validates instead. Every other field falls back to a dataclass default via `get_value_or_default`, but a missing `projects` key is fatal (~line 1064): ``` SerenaConfigError: `projects` key not found in Serena configuration. ``` So Headroom's own bootstrap file killed Serena on **every machine without a pre-existing Serena config**. The MCP server exited during handshake — surfacing as `connection closed: initialize response` on Codex and a bare `MCP error -32000: Connection closed` on OpenCode (#2674) — and `serena project index` failed identically. Headroom now leaves that file to Serena. That is immune to Serena adding required keys later; guessing the schema is what caused the outage. The popup never needed the file anyway: `build_serena_spec` passes `--open-web-dashboard False`, which Serena applies *after* loading the config (`serena/mcp.py:361` — `config.web_dashboard_open_on_launch = open_web_dashboard`), so the flag wins regardless of what is on disk. An **existing** config is still edited in place — dashboard key flipped, `projects: []` backfilled to repair machines an affected version already wrote — preserving a populated `projects` list, other keys and comments. Closes #2674 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - `_ensure_serena_dashboard_disabled()` never creates `serena_config.yml`; it only edits an existing one, and backfills `projects: []` there to repair already-broken machines. - Dropped `_scope_serena_languages` + `_detect_repo_languages` + `_EXT_TO_SERENA_LANGUAGE` (**−133 lines**). Dead weight: Serena determines languages itself in `ProjectConfig.autogenerate` (`_determine_project_language_servers`) and records them under `language_servers` — `languages`, which Headroom wrote, is a legacy name Serena migrates via `RENAMED_FIELDS`. Serena's generated file uses a block-style list, so our single-line-flow regex never matched it: on any Serena-generated `project.yml` the function was a **verified no-op**. The only case where it acted was creating the file — the same partial-config trap — which also skipped the `project.local.yml` sidecar Serena writes alongside. - **Test isolation:** the MCP install ledger defaults to `~/.headroom/mcp_installs.json`, so any test registering a server wrote into the developer's real ledger (observed adding a live `claude/serena` entry during a local run). `conftest.py` now redirects it per-test. - **Repo config:** `.serena/project.yml` carried a stale `project_name` (`"feature-opencode-wrap"`) and listed only `typescript`, so Serena's symbol index skipped 1331 Python and 194 Rust files for every contributor. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_serena_boost.py \ tests/test_cli/test_serena_migrate.py tests/test_cli/test_serena_disable.py -q 35 passed, 1 skipped in 0.84s $ SERENA_SRC=<serena checkout> pytest tests/test_wrap_code_memory.py -q 13 passed in 0.62s # the skipped test runs when a Serena source tree is available $ ruff check headroom/ tests/ --exclude headroom/dashboard/templates All checks passed! $ mypy headroom/cli/wrap.py Success: no issues found in 1 source file ``` New tests. The key one asserts the invariant rather than our own key list, so it stays correct even if Serena adds a required key — a test pinning `projects: []` would keep passing while users broke again: - `test_serena_config_is_never_created_by_headroom` — Headroom must not pre-empt Serena's bootstrap - `test_serena_dashboard_disabled_repairs_config_missing_projects` — heals a config an affected version wrote - `test_serena_dashboard_disabled_preserves_registered_projects` — never clobbers the real registry; comments kept, no duplicate key - `test_serena_dashboard_disabled_is_idempotent` - `test_serena_config_required_keys_match_serena_source` — reads Serena's real source and pins the two facts this fix rests on (bootstrap-only-when-absent, `projects` is the sole fatal omission). Skipped unless `SERENA_SRC` is set; deliberately **not** named `HEADROOM_*` because `conftest.py` scrubs that namespace, which would make it silently always-skip. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Serena 1.6.2.dev0 via `uvx --from git+https://github.com/oraios/serena`, Codex CLI 0.146.0. - **Exact command / steps:** a probe doing a real JSON-RPC `initialize` handshake against the exact command `headroom wrap` registers — i.e. what Codex/OpenCode actually do — in a throwaway `HOME` per case. (A) pre-seeded with the one-line config an affected version wrote; (B) no config; (C) real `headroom wrap codex --prepare-only`, then handshake. - **Observed result:** ```text === A. BROKEN: single-key config (Headroom 0.33.0) === MCP handshake: FAIL — no initialize response (exit=1). stderr tail: File ".../serena/config/serena_config.py", line 1064, in from_config_file raise SerenaConfigError("`projects` key not found in Serena configuration. ...") serena.config.serena_config.SerenaConfigError: `projects` key not found ... config after run: 1 lines, has 'projects': False === B. FIXED: no config, Serena bootstraps it === MCP handshake: PASS — initialize OK — serverInfo.name='Serena' config after run: 213 lines, has 'projects': True === C. FULL FLOW: real `headroom wrap codex` then handshake === Serena: no serena_config.yml yet — letting Serena generate it Serena MCP: registered (restart OpenAI Codex CLI if it was already running) Serena: project pre-indexed (symbol cache warmed) serena_config.yml: 213 lines, written by Serena (correct) MCP handshake: PASS — initialize OK — serverInfo.name='Serena' --- verdict --- A (broken config) started: False <- expected False B (fixed, no config) started: True <- expected True C (after real wrap) started: True <- expected True ``` A second `wrap` in the same HOME flips the dashboard without damage: `true` → `false`, `projects` intact, all 153 comment lines intact. The writer was isolated against a pristine 213-line Serena config: **delta 0 newlines**. - **Not tested:** Windows and Linux (macOS only); Serena versions other than 1.6.2.dev0; the JetBrains language backend. The probe needs network + `uvx` (~2 min) so it is a manual verification tool, not wired into CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - **Docs:** N/A — no user-facing docs described the `serena_config.yml` bootstrap or the language scoping. - Also fixes the OpenCode report (#2674). The Codex-side report of the same root cause quotes the `SerenaConfigError` verbatim; OpenCode only surfaces the generic `-32000`, which is why it read as two different bugs. - Users already broken by an affected version are repaired automatically on their next `headroom wrap` — no manual `serena_config.yml` edit needed. - A stacked PR removing the rtk/lean-ctx CLI context tools is based on this branch; this one is deliberately small so it can land first.
2026-07-30 20:55:47 -07:00
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
wrap._ensure_serena_dashboard_disabled()
fix(wrap/serena): stop creating serena_config.yml, unbricking Serena on fresh installs (#2676) ## Description `_ensure_serena_dashboard_disabled()` wrote a one-key bootstrap config (`web_dashboard_open_on_launch: false`) into `~/.serena/serena_config.yml` when the file was absent, assuming Serena fills in any key it omits. Verified against **Serena 1.6.2.dev0** (`serena/config/serena_config.py`), that holds for every field except one. Serena autogenerates its own complete config **only when the path does not exist**: ```python if not os.path.exists(config_file_path): cls._generate_config_file(config_file_path) ``` Once any file is present it validates instead. Every other field falls back to a dataclass default via `get_value_or_default`, but a missing `projects` key is fatal (~line 1064): ``` SerenaConfigError: `projects` key not found in Serena configuration. ``` So Headroom's own bootstrap file killed Serena on **every machine without a pre-existing Serena config**. The MCP server exited during handshake — surfacing as `connection closed: initialize response` on Codex and a bare `MCP error -32000: Connection closed` on OpenCode (#2674) — and `serena project index` failed identically. Headroom now leaves that file to Serena. That is immune to Serena adding required keys later; guessing the schema is what caused the outage. The popup never needed the file anyway: `build_serena_spec` passes `--open-web-dashboard False`, which Serena applies *after* loading the config (`serena/mcp.py:361` — `config.web_dashboard_open_on_launch = open_web_dashboard`), so the flag wins regardless of what is on disk. An **existing** config is still edited in place — dashboard key flipped, `projects: []` backfilled to repair machines an affected version already wrote — preserving a populated `projects` list, other keys and comments. Closes #2674 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - `_ensure_serena_dashboard_disabled()` never creates `serena_config.yml`; it only edits an existing one, and backfills `projects: []` there to repair already-broken machines. - Dropped `_scope_serena_languages` + `_detect_repo_languages` + `_EXT_TO_SERENA_LANGUAGE` (**−133 lines**). Dead weight: Serena determines languages itself in `ProjectConfig.autogenerate` (`_determine_project_language_servers`) and records them under `language_servers` — `languages`, which Headroom wrote, is a legacy name Serena migrates via `RENAMED_FIELDS`. Serena's generated file uses a block-style list, so our single-line-flow regex never matched it: on any Serena-generated `project.yml` the function was a **verified no-op**. The only case where it acted was creating the file — the same partial-config trap — which also skipped the `project.local.yml` sidecar Serena writes alongside. - **Test isolation:** the MCP install ledger defaults to `~/.headroom/mcp_installs.json`, so any test registering a server wrote into the developer's real ledger (observed adding a live `claude/serena` entry during a local run). `conftest.py` now redirects it per-test. - **Repo config:** `.serena/project.yml` carried a stale `project_name` (`"feature-opencode-wrap"`) and listed only `typescript`, so Serena's symbol index skipped 1331 Python and 194 Rust files for every contributor. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_serena_boost.py \ tests/test_cli/test_serena_migrate.py tests/test_cli/test_serena_disable.py -q 35 passed, 1 skipped in 0.84s $ SERENA_SRC=<serena checkout> pytest tests/test_wrap_code_memory.py -q 13 passed in 0.62s # the skipped test runs when a Serena source tree is available $ ruff check headroom/ tests/ --exclude headroom/dashboard/templates All checks passed! $ mypy headroom/cli/wrap.py Success: no issues found in 1 source file ``` New tests. The key one asserts the invariant rather than our own key list, so it stays correct even if Serena adds a required key — a test pinning `projects: []` would keep passing while users broke again: - `test_serena_config_is_never_created_by_headroom` — Headroom must not pre-empt Serena's bootstrap - `test_serena_dashboard_disabled_repairs_config_missing_projects` — heals a config an affected version wrote - `test_serena_dashboard_disabled_preserves_registered_projects` — never clobbers the real registry; comments kept, no duplicate key - `test_serena_dashboard_disabled_is_idempotent` - `test_serena_config_required_keys_match_serena_source` — reads Serena's real source and pins the two facts this fix rests on (bootstrap-only-when-absent, `projects` is the sole fatal omission). Skipped unless `SERENA_SRC` is set; deliberately **not** named `HEADROOM_*` because `conftest.py` scrubs that namespace, which would make it silently always-skip. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Serena 1.6.2.dev0 via `uvx --from git+https://github.com/oraios/serena`, Codex CLI 0.146.0. - **Exact command / steps:** a probe doing a real JSON-RPC `initialize` handshake against the exact command `headroom wrap` registers — i.e. what Codex/OpenCode actually do — in a throwaway `HOME` per case. (A) pre-seeded with the one-line config an affected version wrote; (B) no config; (C) real `headroom wrap codex --prepare-only`, then handshake. - **Observed result:** ```text === A. BROKEN: single-key config (Headroom 0.33.0) === MCP handshake: FAIL — no initialize response (exit=1). stderr tail: File ".../serena/config/serena_config.py", line 1064, in from_config_file raise SerenaConfigError("`projects` key not found in Serena configuration. ...") serena.config.serena_config.SerenaConfigError: `projects` key not found ... config after run: 1 lines, has 'projects': False === B. FIXED: no config, Serena bootstraps it === MCP handshake: PASS — initialize OK — serverInfo.name='Serena' config after run: 213 lines, has 'projects': True === C. FULL FLOW: real `headroom wrap codex` then handshake === Serena: no serena_config.yml yet — letting Serena generate it Serena MCP: registered (restart OpenAI Codex CLI if it was already running) Serena: project pre-indexed (symbol cache warmed) serena_config.yml: 213 lines, written by Serena (correct) MCP handshake: PASS — initialize OK — serverInfo.name='Serena' --- verdict --- A (broken config) started: False <- expected False B (fixed, no config) started: True <- expected True C (after real wrap) started: True <- expected True ``` A second `wrap` in the same HOME flips the dashboard without damage: `true` → `false`, `projects` intact, all 153 comment lines intact. The writer was isolated against a pristine 213-line Serena config: **delta 0 newlines**. - **Not tested:** Windows and Linux (macOS only); Serena versions other than 1.6.2.dev0; the JetBrains language backend. The probe needs network + `uvx` (~2 min) so it is a manual verification tool, not wired into CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - **Docs:** N/A — no user-facing docs described the `serena_config.yml` bootstrap or the language scoping. - Also fixes the OpenCode report (#2674). The Codex-side report of the same root cause quotes the `SerenaConfigError` verbatim; OpenCode only surfaces the generic `-32000`, which is why it read as two different bugs. - Users already broken by an affected version are repaired automatically on their next `headroom wrap` — no manual `serena_config.yml` edit needed. - A stacked PR removing the rtk/lean-ctx CLI context tools is based on this branch; this one is deliberately small so it can land first.
2026-07-30 20:55:47 -07:00
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
cfg = tmp_path / ".serena" / "serena_config.yml"
fix(wrap/serena): stop creating serena_config.yml, unbricking Serena on fresh installs (#2676) ## Description `_ensure_serena_dashboard_disabled()` wrote a one-key bootstrap config (`web_dashboard_open_on_launch: false`) into `~/.serena/serena_config.yml` when the file was absent, assuming Serena fills in any key it omits. Verified against **Serena 1.6.2.dev0** (`serena/config/serena_config.py`), that holds for every field except one. Serena autogenerates its own complete config **only when the path does not exist**: ```python if not os.path.exists(config_file_path): cls._generate_config_file(config_file_path) ``` Once any file is present it validates instead. Every other field falls back to a dataclass default via `get_value_or_default`, but a missing `projects` key is fatal (~line 1064): ``` SerenaConfigError: `projects` key not found in Serena configuration. ``` So Headroom's own bootstrap file killed Serena on **every machine without a pre-existing Serena config**. The MCP server exited during handshake — surfacing as `connection closed: initialize response` on Codex and a bare `MCP error -32000: Connection closed` on OpenCode (#2674) — and `serena project index` failed identically. Headroom now leaves that file to Serena. That is immune to Serena adding required keys later; guessing the schema is what caused the outage. The popup never needed the file anyway: `build_serena_spec` passes `--open-web-dashboard False`, which Serena applies *after* loading the config (`serena/mcp.py:361` — `config.web_dashboard_open_on_launch = open_web_dashboard`), so the flag wins regardless of what is on disk. An **existing** config is still edited in place — dashboard key flipped, `projects: []` backfilled to repair machines an affected version already wrote — preserving a populated `projects` list, other keys and comments. Closes #2674 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - `_ensure_serena_dashboard_disabled()` never creates `serena_config.yml`; it only edits an existing one, and backfills `projects: []` there to repair already-broken machines. - Dropped `_scope_serena_languages` + `_detect_repo_languages` + `_EXT_TO_SERENA_LANGUAGE` (**−133 lines**). Dead weight: Serena determines languages itself in `ProjectConfig.autogenerate` (`_determine_project_language_servers`) and records them under `language_servers` — `languages`, which Headroom wrote, is a legacy name Serena migrates via `RENAMED_FIELDS`. Serena's generated file uses a block-style list, so our single-line-flow regex never matched it: on any Serena-generated `project.yml` the function was a **verified no-op**. The only case where it acted was creating the file — the same partial-config trap — which also skipped the `project.local.yml` sidecar Serena writes alongside. - **Test isolation:** the MCP install ledger defaults to `~/.headroom/mcp_installs.json`, so any test registering a server wrote into the developer's real ledger (observed adding a live `claude/serena` entry during a local run). `conftest.py` now redirects it per-test. - **Repo config:** `.serena/project.yml` carried a stale `project_name` (`"feature-opencode-wrap"`) and listed only `typescript`, so Serena's symbol index skipped 1331 Python and 194 Rust files for every contributor. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_serena_boost.py \ tests/test_cli/test_serena_migrate.py tests/test_cli/test_serena_disable.py -q 35 passed, 1 skipped in 0.84s $ SERENA_SRC=<serena checkout> pytest tests/test_wrap_code_memory.py -q 13 passed in 0.62s # the skipped test runs when a Serena source tree is available $ ruff check headroom/ tests/ --exclude headroom/dashboard/templates All checks passed! $ mypy headroom/cli/wrap.py Success: no issues found in 1 source file ``` New tests. The key one asserts the invariant rather than our own key list, so it stays correct even if Serena adds a required key — a test pinning `projects: []` would keep passing while users broke again: - `test_serena_config_is_never_created_by_headroom` — Headroom must not pre-empt Serena's bootstrap - `test_serena_dashboard_disabled_repairs_config_missing_projects` — heals a config an affected version wrote - `test_serena_dashboard_disabled_preserves_registered_projects` — never clobbers the real registry; comments kept, no duplicate key - `test_serena_dashboard_disabled_is_idempotent` - `test_serena_config_required_keys_match_serena_source` — reads Serena's real source and pins the two facts this fix rests on (bootstrap-only-when-absent, `projects` is the sole fatal omission). Skipped unless `SERENA_SRC` is set; deliberately **not** named `HEADROOM_*` because `conftest.py` scrubs that namespace, which would make it silently always-skip. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Serena 1.6.2.dev0 via `uvx --from git+https://github.com/oraios/serena`, Codex CLI 0.146.0. - **Exact command / steps:** a probe doing a real JSON-RPC `initialize` handshake against the exact command `headroom wrap` registers — i.e. what Codex/OpenCode actually do — in a throwaway `HOME` per case. (A) pre-seeded with the one-line config an affected version wrote; (B) no config; (C) real `headroom wrap codex --prepare-only`, then handshake. - **Observed result:** ```text === A. BROKEN: single-key config (Headroom 0.33.0) === MCP handshake: FAIL — no initialize response (exit=1). stderr tail: File ".../serena/config/serena_config.py", line 1064, in from_config_file raise SerenaConfigError("`projects` key not found in Serena configuration. ...") serena.config.serena_config.SerenaConfigError: `projects` key not found ... config after run: 1 lines, has 'projects': False === B. FIXED: no config, Serena bootstraps it === MCP handshake: PASS — initialize OK — serverInfo.name='Serena' config after run: 213 lines, has 'projects': True === C. FULL FLOW: real `headroom wrap codex` then handshake === Serena: no serena_config.yml yet — letting Serena generate it Serena MCP: registered (restart OpenAI Codex CLI if it was already running) Serena: project pre-indexed (symbol cache warmed) serena_config.yml: 213 lines, written by Serena (correct) MCP handshake: PASS — initialize OK — serverInfo.name='Serena' --- verdict --- A (broken config) started: False <- expected False B (fixed, no config) started: True <- expected True C (after real wrap) started: True <- expected True ``` A second `wrap` in the same HOME flips the dashboard without damage: `true` → `false`, `projects` intact, all 153 comment lines intact. The writer was isolated against a pristine 213-line Serena config: **delta 0 newlines**. - **Not tested:** Windows and Linux (macOS only); Serena versions other than 1.6.2.dev0; the JetBrains language backend. The probe needs network + `uvx` (~2 min) so it is a manual verification tool, not wired into CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - **Docs:** N/A — no user-facing docs described the `serena_config.yml` bootstrap or the language scoping. - Also fixes the OpenCode report (#2674). The Codex-side report of the same root cause quotes the `SerenaConfigError` verbatim; OpenCode only surfaces the generic `-32000`, which is why it read as two different bugs. - Users already broken by an affected version are repaired automatically on their next `headroom wrap` — no manual `serena_config.yml` edit needed. - A stacked PR removing the rtk/lean-ctx CLI context tools is based on this branch; this one is deliberately small so it can land first.
2026-07-30 20:55:47 -07:00
assert not cfg.exists(), "Headroom created a config Serena would have generated itself"
def test_serena_dashboard_disabled_repairs_config_missing_projects(tmp_path, monkeypatch) -> None:
"""Backfill ``projects`` into a config an older Headroom already wrote (#2674).
Users who ran an affected version have the single-key file on disk, so simply
not creating new bad files would leave them broken forever.
"""
import yaml
monkeypatch.setenv("HOME", str(tmp_path))
cfg = tmp_path / ".serena" / "serena_config.yml"
cfg.parent.mkdir(parents=True)
cfg.write_text("web_dashboard_open_on_launch: false\n")
wrap._ensure_serena_dashboard_disabled()
parsed = yaml.safe_load(cfg.read_text())
assert parsed["projects"] == []
assert parsed["web_dashboard_open_on_launch"] is False
def test_serena_dashboard_disabled_preserves_registered_projects(tmp_path, monkeypatch) -> None:
"""Never clobber Serena's real project registry — it is user data."""
import yaml
monkeypatch.setenv("HOME", str(tmp_path))
cfg = tmp_path / ".serena" / "serena_config.yml"
cfg.parent.mkdir(parents=True)
cfg.write_text(
"# my serena config\nprojects:\n - /home/me/work/api\n - /home/me/work/web\n"
"web_dashboard_open_on_launch: true\n"
)
wrap._ensure_serena_dashboard_disabled()
text = cfg.read_text()
parsed = yaml.safe_load(text)
assert parsed["projects"] == ["/home/me/work/api", "/home/me/work/web"]
assert parsed["web_dashboard_open_on_launch"] is False
assert "# my serena config" in text # comments preserved
assert text.count("projects:") == 1 # no duplicate key
def test_serena_dashboard_disabled_is_idempotent(tmp_path, monkeypatch) -> None:
monkeypatch.setenv("HOME", str(tmp_path))
cfg = tmp_path / ".serena" / "serena_config.yml"
cfg.parent.mkdir(parents=True)
cfg.write_text("projects: []\nweb_dashboard_open_on_launch: true\n")
wrap._ensure_serena_dashboard_disabled()
first = cfg.read_text()
wrap._ensure_serena_dashboard_disabled()
assert cfg.read_text() == first
def test_serena_config_required_keys_match_serena_source() -> None:
"""Pin the assumption this fix rests on, against Serena's real source (#2674).
Skipped unless a Serena checkout is present. When it is, this proves the claim
the fix depends on that ``projects`` is the *only* hard-required key and that
Serena bootstraps only a missing file rather than trusting a bug report.
Set ``SERENA_SRC`` to a Serena source tree to enable it.
"""
import os
import re
from pathlib import Path
src = os.environ.get("SERENA_SRC", "")
if not src or not (Path(src) / "config" / "serena_config.py").is_file():
pytest.skip("set SERENA_SRC to a Serena source tree to run this check")
text = (Path(src) / "config" / "serena_config.py").read_text(encoding="utf-8")
# Serena bootstraps only when the file is absent — so we must not create one.
assert re.search(r"if not os\.path\.exists\(config_file_path\)", text)
# `projects` is the sole fatal omission; everything else has a default.
fatal = re.findall(r"raise SerenaConfigError\((.*?)\)", text, re.DOTALL)
projects_fatal = [f for f in fatal if "projects" in f]
assert projects_fatal, "Serena no longer rejects a missing `projects` key"
assert "get_value_or_default" in text, "Serena's default-filling path changed"
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
def test_invalid_env_raises() -> None:
with patch.dict(os.environ, {"HEADROOM_CODE_MEMORY": "bogus"}):
try:
wrap._resolve_code_memory({})
except click.ClickException:
pass
else: # pragma: no cover
raise AssertionError("invalid HEADROOM_CODE_MEMORY should raise ClickException")
def _dispatch_calls(selection: str, extra: dict | None = None) -> list[str]:
"""Run the orchestrator with a given selection, recording which setup/disable
helpers fire (all mocked)."""
calls: list[str] = []
env = _clean_env()
env["HEADROOM_CODE_MEMORY"] = selection
with (
patch.dict(os.environ, env, clear=True),
patch.object(wrap, "_setup_serena_mcp", lambda *a, **k: calls.append("serena")),
patch.object(
wrap, "_disable_tokensave_mcp", lambda *a, **k: calls.append("disable_tokensave")
),
patch.object(wrap, "_disable_serena_mcp", lambda *a, **k: calls.append("disable_serena")),
):
wrap._setup_coding_compressor(object(), serena_context="claude-code", **(extra or {}))
return calls
def test_orchestrator_dispatch() -> None:
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499) ## Description `tokensave` was a **downloaded third-party Rust binary** (`aovestdipaperino/tokensave`) that `headroom wrap` registered as a code-graph MCP server. This removes it entirely and standardises on **Serena** as the code-memory MCP — which was already the default in `wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads or executes a binary of its own for code memory. The change is a *removal + safe transition*, not a behaviour flip: Serena was already the default, so existing users move over automatically. This PR also folds in a small README repositioning (Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx are third-party tools we don't control), since it's the same tooling-stack story. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - **Removed the external tool:** `headroom/graph/tokensave_installer.py` (the download-and-execute path), `build_tokensave_spec`, and the `_ensure_tokensave_binary` / `_index_tokensave_project` / `_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`. - **`--code-memory`** now offers `serena` (default) or `none` — the `tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as its (default-on) code-memory MCP. - **Tombstone / transition (ledger-verified):** `headroom wrap` **and** `headroom unwrap` remove a previously Headroom-installed `tokensave` MCP entry so upgrading users stop launching it, and print that the leftover `~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to delete. A user-managed `tokensave` entry is left untouched. Mirrors the existing `codebase-memory-mcp` retirement. - **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and `--no-tokensave` resolve to Serena instead of erroring; `--no-serena` now means "no code memory". **No state migration needed** — both tools' indexes are regenerable caches of the source, so Serena simply re-indexes. - **Docs/README:** replaced the "tokensave binary trust model" section with a Serena note + an "Upgrading from tokensave?" callout; retired the RTK "first-class part of our stack" framing. - **Tests:** deleted the tokensave-only test files (`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`, `test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py` for the new resolver/dispatch, fixed a codex test that patched a removed symbol. Net: **+102 / −1187 lines.** ## Testing - [x] Unit tests pass (`pytest`) — targeted to the affected areas - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py All checks passed! $ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py Success: no issues found in 12 source files $ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \ tests/test_cli/test_wrap_claude_vertex_proxy_env.py \ tests/test_cli/test_wrap_claude_finally_unbound.py -q ======================== 120 passed in 97.86s (0:01:37) ======================== $ pytest tests/test_cli --collect-only -q ========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal ``` ## Real Behavior Proof - **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on branch `tejas/remove-tokensave`. - **Exact command / steps:** - `python -c "from headroom.integrations.strands.bundle import HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False, enable_serena_mcp=False); print(len(b.tools))"` → confirms the module imports after `build_tokensave_spec` removal (the import that my change would otherwise break). - CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`) writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context", "codex"`) to the codex config and **no** tokensave entry. - `_resolve_code_memory` unit tests confirm: default → `serena`; `HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`; `--code-memory bogus` → `ClickException`. - **Observed result:** import OK (`tools: 0`); Serena registered, tokensave absent; resolver behaves as above; 120/120 tests pass. - **Not tested:** a live `headroom wrap` against a real agent on a machine with a *previously-installed* tokensave MCP entry — the tombstone-removal path is covered by unit tests with a fake registrar/ledger, not an end-to-end run. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes - `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden, deprecated flags (no-ops or mapped) so existing scripts don't break. - `--code-graph` is unchanged — it's the proxy's live file-watcher flag and was never the tokensave MCP; only the dead tokensave hook behind it was removed. - `CHANGELOG.md` intentionally left untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-22 20:59:24 -07:00
# A legacy tokensave entry is always retired first, then the selection applies.
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413) ## What Two commits: 1. **Unify code-memory MCP selection behind `--code-memory {tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the `--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector. Old flags remain as hidden deprecated aliases that map into it. Shared across the code-memory-capable subcommands (claude/codex/grok). 2. **Default the engine to Serena**, with its **dashboard browser suppressed**. ## Why Serena as default Serena is a mature, offline, symbol-level code-navigation MCP with broad language coverage (LSP-backed) — the strongest zero-account default for reducing tokens by letting the agent query symbols/definitions/references instead of reading whole files. It attacks the *protected-reads* volume the proxy deliberately doesn't compress, so it's complementary to the pipeline compressors. ## Browser suppression (in Serena's own settings) `_ensure_serena_dashboard_disabled()` sets `web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml` when Serena is set up, so wrapped sessions don't spawn a browser tab. The dashboard backend stays reachable manually at `localhost:24282`. This lives in Serena's config (authoritative), not just a startup flag. ## Schema-overhead note Serena injects tool schemas per request; that cost is deferred by the tool-search deferral the coding profile already enables (`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation benefit without a standing schema tax on turns that don't navigate. ## Selection / escape hatches `--code-memory serena` (default) · `tokensave` (lighter/faster) · `none` (disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still work. ## Testing Updated the primary/backup policy test to the serena-primary default; code-memory selector + serena disable/migrate tests pass. Local: 21 passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00
assert _dispatch_calls(wrap._CODE_MEMORY_SERENA) == ["disable_tokensave", "serena"]
assert set(_dispatch_calls(wrap._CODE_MEMORY_NONE)) == {"disable_tokensave", "disable_serena"}
def test_code_memory_option_present_only_on_code_memory_agents() -> None:
runner = CliRunner()
for tool in ("claude", "codex", "grok"):
out = runner.invoke(wrap.wrap, [tool, "--help"]).output
assert "--code-memory" in out, f"--code-memory missing from `wrap {tool} --help`"
# aider does not register a code-memory MCP → no flag
out = runner.invoke(wrap.wrap, ["aider", "--help"]).output
assert "--code-memory" not in out