headroom/tests/test_wrap_code_memory.py
Tejas Chopra 759209cff3
fix(wrap/serena): stop creating serena_config.yml, unbricking Serena on fresh installs (#2676)
## Description

`_ensure_serena_dashboard_disabled()` wrote a one-key bootstrap config
(`web_dashboard_open_on_launch: false`) into
`~/.serena/serena_config.yml` when the file was absent, assuming Serena
fills in any key it omits.

Verified against **Serena 1.6.2.dev0**
(`serena/config/serena_config.py`), that holds for every field except
one. Serena autogenerates its own complete config **only when the path
does not exist**:

```python
if not os.path.exists(config_file_path):
    cls._generate_config_file(config_file_path)
```

Once any file is present it validates instead. Every other field falls
back to a dataclass default via `get_value_or_default`, but a missing
`projects` key is fatal (~line 1064):

```
SerenaConfigError: `projects` key not found in Serena configuration.
```

So Headroom's own bootstrap file killed Serena on **every machine
without a pre-existing Serena config**. The MCP server exited during
handshake — surfacing as `connection closed: initialize response` on
Codex and a bare `MCP error -32000: Connection closed` on OpenCode
(#2674) — and `serena project index` failed identically.

Headroom now leaves that file to Serena. That is immune to Serena adding
required keys later; guessing the schema is what caused the outage. The
popup never needed the file anyway: `build_serena_spec` passes
`--open-web-dashboard False`, which Serena applies *after* loading the
config (`serena/mcp.py:361` — `config.web_dashboard_open_on_launch =
open_web_dashboard`), so the flag wins regardless of what is on disk.

An **existing** config is still edited in place — dashboard key flipped,
`projects: []` backfilled to repair machines an affected version already
wrote — preserving a populated `projects` list, other keys and comments.

Closes #2674

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- `_ensure_serena_dashboard_disabled()` never creates
`serena_config.yml`; it only edits an existing one, and backfills
`projects: []` there to repair already-broken machines.
- Dropped `_scope_serena_languages` + `_detect_repo_languages` +
`_EXT_TO_SERENA_LANGUAGE` (**−133 lines**). Dead weight: Serena
determines languages itself in `ProjectConfig.autogenerate`
(`_determine_project_language_servers`) and records them under
`language_servers` — `languages`, which Headroom wrote, is a legacy name
Serena migrates via `RENAMED_FIELDS`. Serena's generated file uses a
block-style list, so our single-line-flow regex never matched it: on any
Serena-generated `project.yml` the function was a **verified no-op**.
The only case where it acted was creating the file — the same
partial-config trap — which also skipped the `project.local.yml` sidecar
Serena writes alongside.
- **Test isolation:** the MCP install ledger defaults to
`~/.headroom/mcp_installs.json`, so any test registering a server wrote
into the developer's real ledger (observed adding a live `claude/serena`
entry during a local run). `conftest.py` now redirects it per-test.
- **Repo config:** `.serena/project.yml` carried a stale `project_name`
(`"feature-opencode-wrap"`) and listed only `typescript`, so Serena's
symbol index skipped 1331 Python and 194 Rust files for every
contributor.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_serena_boost.py \
         tests/test_cli/test_serena_migrate.py tests/test_cli/test_serena_disable.py -q
35 passed, 1 skipped in 0.84s

$ SERENA_SRC=<serena checkout> pytest tests/test_wrap_code_memory.py -q
13 passed in 0.62s        # the skipped test runs when a Serena source tree is available

$ ruff check headroom/ tests/ --exclude headroom/dashboard/templates
All checks passed!

$ mypy headroom/cli/wrap.py
Success: no issues found in 1 source file
```

New tests. The key one asserts the invariant rather than our own key
list, so it stays correct even if Serena adds a required key — a test
pinning `projects: []` would keep passing while users broke again:

- `test_serena_config_is_never_created_by_headroom` — Headroom must not
pre-empt Serena's bootstrap
- `test_serena_dashboard_disabled_repairs_config_missing_projects` —
heals a config an affected version wrote
- `test_serena_dashboard_disabled_preserves_registered_projects` — never
clobbers the real registry; comments kept, no duplicate key
- `test_serena_dashboard_disabled_is_idempotent`
- `test_serena_config_required_keys_match_serena_source` — reads
Serena's real source and pins the two facts this fix rests on
(bootstrap-only-when-absent, `projects` is the sole fatal omission).
Skipped unless `SERENA_SRC` is set; deliberately **not** named
`HEADROOM_*` because `conftest.py` scrubs that namespace, which would
make it silently always-skip.

## Real Behavior Proof

- **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Serena
1.6.2.dev0 via `uvx --from git+https://github.com/oraios/serena`, Codex
CLI 0.146.0.
- **Exact command / steps:** a probe doing a real JSON-RPC `initialize`
handshake against the exact command `headroom wrap` registers — i.e.
what Codex/OpenCode actually do — in a throwaway `HOME` per case. (A)
pre-seeded with the one-line config an affected version wrote; (B) no
config; (C) real `headroom wrap codex --prepare-only`, then handshake.
- **Observed result:**

```text
=== A. BROKEN: single-key config (Headroom 0.33.0) ===
  MCP handshake: FAIL — no initialize response (exit=1). stderr tail:
    File ".../serena/config/serena_config.py", line 1064, in from_config_file
      raise SerenaConfigError("`projects` key not found in Serena configuration. ...")
    serena.config.serena_config.SerenaConfigError: `projects` key not found ...
  config after run: 1 lines, has 'projects': False

=== B. FIXED: no config, Serena bootstraps it ===
  MCP handshake: PASS — initialize OK — serverInfo.name='Serena'
  config after run: 213 lines, has 'projects': True

=== C. FULL FLOW: real `headroom wrap codex` then handshake ===
    Serena: no serena_config.yml yet — letting Serena generate it
    Serena MCP: registered (restart OpenAI Codex CLI if it was already running)
    Serena: project pre-indexed (symbol cache warmed)
    serena_config.yml: 213 lines, written by Serena (correct)
    MCP handshake: PASS — initialize OK — serverInfo.name='Serena'

--- verdict ---
  A (broken config)     started: False   <- expected False
  B (fixed, no config)  started: True   <- expected True
  C (after real wrap)   started: True   <- expected True
```

A second `wrap` in the same HOME flips the dashboard without damage:
`true` → `false`, `projects` intact, all 153 comment lines intact. The
writer was isolated against a pristine 213-line Serena config: **delta 0
newlines**.

- **Not tested:** Windows and Linux (macOS only); Serena versions other
than 1.6.2.dev0; the JetBrains language backend. The probe needs network
+ `uvx` (~2 min) so it is a manual verification tool, not wired into CI.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- **Docs:** N/A — no user-facing docs described the `serena_config.yml`
bootstrap or the language scoping.
- Also fixes the OpenCode report (#2674). The Codex-side report of the
same root cause quotes the `SerenaConfigError` verbatim; OpenCode only
surfaces the generic `-32000`, which is why it read as two different
bugs.
- Users already broken by an affected version are repaired automatically
on their next `headroom wrap` — no manual `serena_config.yml` edit
needed.
- A stacked PR removing the rtk/lean-ctx CLI context tools is based on
this branch; this one is deliberately small so it can land first.
2026-07-30 20:55:47 -07:00

210 lines
8.7 KiB
Python

"""Code-memory MCP is selectable via --code-memory (default serena).
Covers the resolver precedence (selector > deprecated flags > default), the
graceful retirement of the removed ``tokensave`` option, the orchestrator
dispatch for each selection, and that --code-memory is exposed on the
code-memory-capable subcommands (claude/codex/grok) but not others.
"""
from __future__ import annotations
import os
from unittest.mock import patch
import click
import pytest
from click.testing import CliRunner
from headroom.cli import wrap
def _clean_env() -> dict[str, str]:
env = dict(os.environ)
env.pop("HEADROOM_CODE_MEMORY", None)
return env
def test_default_is_serena() -> None:
with patch.dict(os.environ, _clean_env(), clear=True):
assert wrap._resolve_code_memory({}) == wrap._CODE_MEMORY_SERENA
def test_selector_env_wins() -> None:
for val in (wrap._CODE_MEMORY_SERENA, wrap._CODE_MEMORY_NONE):
with patch.dict(os.environ, {"HEADROOM_CODE_MEMORY": val}):
# selector beats any legacy flag
assert wrap._resolve_code_memory({"serena": True, "no_serena": True}) == val
def test_deprecated_flags_map_into_selector() -> None:
with patch.dict(os.environ, _clean_env(), clear=True):
assert wrap._resolve_code_memory({"serena": True}) == wrap._CODE_MEMORY_SERENA
# tokensave is retired: --no-tokensave is now a no-op → default serena
assert wrap._resolve_code_memory({"no_tokensave": True}) == wrap._CODE_MEMORY_SERENA
# --no-serena means "no code memory" now that tokensave is gone
assert wrap._resolve_code_memory({"no_serena": True}) == wrap._CODE_MEMORY_NONE
def test_retired_tokensave_selector_maps_to_serena() -> None:
# An explicit HEADROOM_CODE_MEMORY=tokensave (or --code-memory tokensave from
# an old script) degrades gracefully to Serena instead of erroring.
with patch.dict(os.environ, {"HEADROOM_CODE_MEMORY": "tokensave"}):
assert wrap._resolve_code_memory({}) == wrap._CODE_MEMORY_SERENA
def test_serena_dashboard_disabled_flips_existing_config(tmp_path, monkeypatch) -> None:
monkeypatch.setenv("HOME", str(tmp_path))
cfg = tmp_path / ".serena" / "serena_config.yml"
cfg.parent.mkdir(parents=True)
cfg.write_text(
"web_dashboard: true\nweb_dashboard_open_on_launch: true\ngui_log_window: false\n"
)
wrap._ensure_serena_dashboard_disabled()
text = cfg.read_text()
assert "web_dashboard_open_on_launch: false" in text
assert "web_dashboard: true" in text # other keys preserved
def test_serena_config_is_never_created_by_headroom(tmp_path, monkeypatch) -> None:
"""Headroom must NOT pre-empt Serena's own config bootstrap (#2674).
This is the exact invariant, and it is the reason the outage happened.
Verified against Serena 1.6.2.dev0 ``serena/config/serena_config.py``: Serena
autogenerates a complete config only when the path does not exist; once any
file is there it validates instead, and a missing ``projects`` key is fatal
(``SerenaConfigError``). Headroom used to write a one-key bootstrap file,
which killed Serena's MCP handshake on every fresh install.
Asserting "we write nothing" is stronger than asserting which keys we write:
it stays correct even if Serena adds a new required key, whereas a test that
pins our own key list would go on passing while users broke again.
"""
monkeypatch.setenv("HOME", str(tmp_path))
wrap._ensure_serena_dashboard_disabled()
cfg = tmp_path / ".serena" / "serena_config.yml"
assert not cfg.exists(), "Headroom created a config Serena would have generated itself"
def test_serena_dashboard_disabled_repairs_config_missing_projects(tmp_path, monkeypatch) -> None:
"""Backfill ``projects`` into a config an older Headroom already wrote (#2674).
Users who ran an affected version have the single-key file on disk, so simply
not creating new bad files would leave them broken forever.
"""
import yaml
monkeypatch.setenv("HOME", str(tmp_path))
cfg = tmp_path / ".serena" / "serena_config.yml"
cfg.parent.mkdir(parents=True)
cfg.write_text("web_dashboard_open_on_launch: false\n")
wrap._ensure_serena_dashboard_disabled()
parsed = yaml.safe_load(cfg.read_text())
assert parsed["projects"] == []
assert parsed["web_dashboard_open_on_launch"] is False
def test_serena_dashboard_disabled_preserves_registered_projects(tmp_path, monkeypatch) -> None:
"""Never clobber Serena's real project registry — it is user data."""
import yaml
monkeypatch.setenv("HOME", str(tmp_path))
cfg = tmp_path / ".serena" / "serena_config.yml"
cfg.parent.mkdir(parents=True)
cfg.write_text(
"# my serena config\nprojects:\n - /home/me/work/api\n - /home/me/work/web\n"
"web_dashboard_open_on_launch: true\n"
)
wrap._ensure_serena_dashboard_disabled()
text = cfg.read_text()
parsed = yaml.safe_load(text)
assert parsed["projects"] == ["/home/me/work/api", "/home/me/work/web"]
assert parsed["web_dashboard_open_on_launch"] is False
assert "# my serena config" in text # comments preserved
assert text.count("projects:") == 1 # no duplicate key
def test_serena_dashboard_disabled_is_idempotent(tmp_path, monkeypatch) -> None:
monkeypatch.setenv("HOME", str(tmp_path))
cfg = tmp_path / ".serena" / "serena_config.yml"
cfg.parent.mkdir(parents=True)
cfg.write_text("projects: []\nweb_dashboard_open_on_launch: true\n")
wrap._ensure_serena_dashboard_disabled()
first = cfg.read_text()
wrap._ensure_serena_dashboard_disabled()
assert cfg.read_text() == first
def test_serena_config_required_keys_match_serena_source() -> None:
"""Pin the assumption this fix rests on, against Serena's real source (#2674).
Skipped unless a Serena checkout is present. When it is, this proves the claim
the fix depends on — that ``projects`` is the *only* hard-required key and that
Serena bootstraps only a missing file — rather than trusting a bug report.
Set ``SERENA_SRC`` to a Serena source tree to enable it.
"""
import os
import re
from pathlib import Path
src = os.environ.get("SERENA_SRC", "")
if not src or not (Path(src) / "config" / "serena_config.py").is_file():
pytest.skip("set SERENA_SRC to a Serena source tree to run this check")
text = (Path(src) / "config" / "serena_config.py").read_text(encoding="utf-8")
# Serena bootstraps only when the file is absent — so we must not create one.
assert re.search(r"if not os\.path\.exists\(config_file_path\)", text)
# `projects` is the sole fatal omission; everything else has a default.
fatal = re.findall(r"raise SerenaConfigError\((.*?)\)", text, re.DOTALL)
projects_fatal = [f for f in fatal if "projects" in f]
assert projects_fatal, "Serena no longer rejects a missing `projects` key"
assert "get_value_or_default" in text, "Serena's default-filling path changed"
def test_invalid_env_raises() -> None:
with patch.dict(os.environ, {"HEADROOM_CODE_MEMORY": "bogus"}):
try:
wrap._resolve_code_memory({})
except click.ClickException:
pass
else: # pragma: no cover
raise AssertionError("invalid HEADROOM_CODE_MEMORY should raise ClickException")
def _dispatch_calls(selection: str, extra: dict | None = None) -> list[str]:
"""Run the orchestrator with a given selection, recording which setup/disable
helpers fire (all mocked)."""
calls: list[str] = []
env = _clean_env()
env["HEADROOM_CODE_MEMORY"] = selection
with (
patch.dict(os.environ, env, clear=True),
patch.object(wrap, "_setup_serena_mcp", lambda *a, **k: calls.append("serena")),
patch.object(
wrap, "_disable_tokensave_mcp", lambda *a, **k: calls.append("disable_tokensave")
),
patch.object(wrap, "_disable_serena_mcp", lambda *a, **k: calls.append("disable_serena")),
):
wrap._setup_coding_compressor(object(), serena_context="claude-code", **(extra or {}))
return calls
def test_orchestrator_dispatch() -> None:
# A legacy tokensave entry is always retired first, then the selection applies.
assert _dispatch_calls(wrap._CODE_MEMORY_SERENA) == ["disable_tokensave", "serena"]
assert set(_dispatch_calls(wrap._CODE_MEMORY_NONE)) == {"disable_tokensave", "disable_serena"}
def test_code_memory_option_present_only_on_code_memory_agents() -> None:
runner = CliRunner()
for tool in ("claude", "codex", "grok"):
out = runner.invoke(wrap.wrap, [tool, "--help"]).output
assert "--code-memory" in out, f"--code-memory missing from `wrap {tool} --help`"
# aider does not register a code-memory MCP → no flag
out = runner.invoke(wrap.wrap, ["aider", "--help"]).output
assert "--code-memory" not in out