headroom/tests/test_cli
Parideboy 6147883d5e
fix(wrap): stop the Serena pre-index stalling the launch path for 300s (#2945)
## Description

`headroom wrap <agent>` could sit silently for a full 300 seconds before
the agent launched, and leaked one orphaned process every time it did.

`_setup_serena_mcp` runs `serena project index` synchronously on the
launch path, with `capture_output=True`, an inherited stdin and
`timeout=300`. When a project has no `.serena/project.yml`, Serena
auto-creates one — and that auto-creation asks one `[y/N]` question per
additionally-detected language server. Three things then combine:

1. stdin was inherited, so Serena believed it could prompt.
2. stdout was captured, so the question never reached the terminal.
3. the call was synchronous, so the agent waited out the entire timeout.

The user saw no prompt, no progress and no error — only a wrapper that
appeared to hang. The pre-index could never succeed in that state, so
the 300 seconds bought nothing.

On top of that, `subprocess.run` kills only its direct child on timeout.
`uvx` is a launcher that execs the real `serena` executable as a
grandchild, which was never signalled: it reparented to PID 1 and
survived indefinitely. Same class of bug as #615 and #880.

Closes #2938

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `_serena_project_skip_reason` (`headroom/cli/wrap.py`) now returns a
skip reason when `.serena/project.yml` is absent, so the pre-index does
not run in the one state where it cannot succeed.
- `_index_serena_project` passes `stdin=subprocess.DEVNULL`, so a
subprocess that decides to prompt gets EOF and exits in about a second
instead of blocking behind a captured pipe. This is deliberately kept as
a second line of defence even though the skip above already avoids the
known prompt.
- `_index_serena_project` now spawns via `subprocess.Popen` in its own
process group (`start_new_session=True` on POSIX,
`CREATE_NEW_PROCESS_GROUP` on Windows) instead of `run(...)`, so the
whole tree can be signalled.
- New `_kill_serena_index_tree` helper kills that tree on timeout —
`killpg(..., SIGKILL)` on POSIX, `taskkill /F /T /PID` on Windows — then
reaps the child and closes the capture pipes. Best-effort throughout; it
never raises.
- Corrected two comments that asserted the opposite of the observed
behaviour ("a failure or timeout here never blocks the wrap", "neither
blocks the wrap"). Both were accurate about intent and wrong about
effect.
- Added `_SERENA_INDEX_TIMEOUT` (still 300) and a line announcing the
pre-index, so a legitimately long index no longer looks like a hang.
- Tests in `tests/test_cli/test_wrap_serena_boost.py` rewritten for the
`Popen` path and extended to cover the DEVNULL stdin, the process-group
flag, the timeout tree-kill, the new skip reason, and the
`_setup_serena_mcp` wiring on both a fresh project and one that already
has `project.yml`.

### Behaviour change worth a reviewer's attention

**On a project with no `.serena/project.yml`, the pre-index no longer
runs at all.** That is the first wrap of any project, so this is the
common case.

I went this way rather than fixing the prompt because there is no way to
fix it from Headroom's side without re-introducing something the project
deliberately removed. Serena's `project index` command has no
non-interactive switch: `ProjectCommands._create_project` calls
`ProjectConfig.autogenerate(..., interactive=True)` with `interactive`
hardcoded. The only path that skips the prompt is passing
`--ls/--language` explicitly, which means Headroom guessing the
project's languages again — exactly the hand-maintained
extension-to-language map that was removed in #2674, with a comment in
this same function explaining why Serena should own that job.

The cost of skipping is small and self-correcting. Serena's MCP server
(`serena start-mcp-server --project-from-cwd`) generates `project.yml`
itself, non-interactively, on first start, and indexes lazily on demand
— which is the fallback the existing docstring already relied on. So the
first wrap now launches immediately with lazy indexing, and every wrap
after that pre-indexes for real. Previously the first wrap cost 300
seconds *and* still produced no index, so nothing of value is lost.

Happy to switch to passing `--ls` instead if maintainers would rather
keep the pre-index on the first wrap and accept a language map; the
other two changes stand either way.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_cli/test_wrap_serena_boost.py tests/test_cli/test_serena_migrate.py -q
collected 29 items

tests\test_cli\test_wrap_serena_boost.py .............s...........       [ 86%]
tests\test_cli\test_serena_migrate.py ....                               [100%]

======================== 28 passed, 1 skipped in 0.54s ========================

$ python -m ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_serena_boost.py
All checks passed!

$ python -m ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_serena_boost.py
2 files already formatted

$ python -m mypy headroom/cli/wrap.py --ignore-missing-imports --python-version 3.13 --follow-imports=silent
Success: no issues found in 1 source file
```

The single skip is `test_kill_tree_signals_the_group_on_posix`, which is
platform-gated; the Windows counterpart ran. I develop on Windows, so
the POSIX `killpg` branch is covered by unit test only — the end-to-end
tree-kill proof below is the Windows `taskkill` branch.

## Real Behavior Proof

- Environment: Windows 11 Pro 26200, Python 3.13.11, headroom checkout
at 941c25d3 plus this branch, uvx resolving `serena-agent` from PyPI,
throwaway project with bash + TypeScript + Python sources and no
`.serena/`
- Exact command / steps: spawned the real `uvx --from serena-agent
serena project index` against that project twice — once with stdin
readable and never closed (a parent-held pipe, the faithful stand-in for
the idle terminal the old code inherited; `communicate()` cannot be used
here because with `input=None` it closes the child's stdin immediately
and hands it the EOF the real bug never delivers), once with
`stdin=subprocess.DEVNULL`. Then spawned it twice more, stopping it with
a plain `proc.kill()` (the old timeout behaviour) versus the new
`_kill_serena_index_tree`, counting survivors with `Get-CimInstance
Win32_Process` filtered on the command line.
- Observed result: stall reproduced and fixed — `A/open-stdin: STILL
BLOCKED after 30.0s (timeout hit)` versus `B/DEVNULL: exit=1
elapsed=1.1s` with stderr tail `Project configuration auto-generation
failed after 0.000 seconds / Error: EOF when reading a line`. Leak
reproduced and fixed — after spawning a 5-process tree, `after
proc.kill(): still alive: [34468, 35044, 35808]` versus `after
_kill_serena_index_tree: still alive: []`. The 30-second bound in the
first experiment stands in for the shipped 300; the point is that the
child never returns on its own.
- Not tested: the POSIX `killpg`/`start_new_session` branch end-to-end
(no Linux or macOS host available, so it is unit-tested only — and it is
the branch the issue reporter observed failing); a full `headroom wrap
opencode` launch end-to-end, since this machine has no working local
proxy (`tests/test_cli/test_wrap_opencode.py` fails identically with and
without this branch for that reason); and the second-wrap pre-index
success path against a live Serena run, which is covered by unit test
instead.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- The documentation checkbox is unchecked because no user-facing doc
describes the pre-index; the behaviour is explained in source
docstrings, which this PR rewrites.
- `_kill_serena_index_tree` is deliberately total: every step is wrapped
so cleaning up an already-dead child cannot turn a timeout into a crash
on the launch path. There is a test for that.
- #2754 (use a pre-installed Serena) would remove the `uvx` layer this
leak depends on, but not the prompt itself — the stdin guard here is
still needed after that lands.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-08-13 11:46:30 -05:00
..
conftest.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_copilot_auth.py fix(auth): support GitHub Enterprise Copilot OAuth domain (#2192) 2026-07-14 16:07:17 -04:00
test_init_cli.py fix(cli/init): fail clearly on a target settings file with invalid JSON (#2227) 2026-07-15 18:15:45 +00:00
test_init_enable_tool_search.py fix(init): set ENABLE_TOOL_SEARCH=true so Claude Code keeps deferring tools (#746) (#995) 2026-06-19 11:26:26 -05:00
test_install_cli.py fix(install): carry upstream-routing env overrides into supervised deployments (#2429) 2026-07-19 22:18:15 -07:00
test_main_help_version.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_mcp.py feat(mcp): add streamable HTTP MCP transport (#1773) 2026-07-14 13:25:45 -04:00
test_mcp_status.py fix(mcp): mcp status checks ~/.claude.json, not only ~/.claude/mcp.json (#990) 2026-07-15 09:25:52 -05:00
test_recover_codex.py test(recover-codex): bind AF_UNIX socket via short relative path (#2396) 2026-08-02 13:14:31 -07:00
test_serena_disable.py fix(serena): stop the Serena dashboard popup and make --no-serena actually disable Serena (#1003) 2026-06-14 23:32:46 -07:00
test_serena_migrate.py fix(wrap/serena): stop creating serena_config.yml, unbricking Serena on fresh installs (#2676) 2026-07-30 20:55:47 -07:00
test_subprocess_utf8_encoding.py fix(windows): pin UTF-8 encoding on text-mode subprocess calls (#1311) 2026-06-23 12:52:49 -05:00
test_unwrap_claude.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_aider.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_bridge.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_claude.py fix(claude): stop forcing tool search on Foundry (#2477) 2026-08-12 00:15:07 -05:00
test_wrap_claude_base_url.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_claude_finally_unbound.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_claude_vertex_proxy_env.py fix(wrap/claude): keep --1m effective when an explicit --model is passed through 2026-08-11 18:15:40 -07:00
test_wrap_codex.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_copilot.py fix(wrap): honor Copilot OAuth wire-api override and model default (#2387) 2026-08-12 00:04:42 -05:00
test_wrap_dead_marker_selfheal.py fix(wrap): self-heal a stale ANTHROPIC_BASE_URL left by a dead proxy (#2223) 2026-07-15 19:58:10 +00:00
test_wrap_encoding.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_goose.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_grok.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_helpers.py fix(wrap): serialize shared proxy startup (#2946) 2026-08-12 12:50:09 -07:00
test_wrap_kimi.py feat(wrap): add headroom wrap kimi for Kimi CLI (#1426) 2026-07-15 21:40:58 +00:00
test_wrap_omp.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_openclaude.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_openclaw.py fix(wrap): use canonical headroom-openclaw npm package for wrap openclaw (#1969) (#2120) 2026-07-13 16:33:44 -04:00
test_wrap_opencode.py fix(wrap/opencode): verify the opencode binary before mutating config 2026-08-11 18:15:37 -07:00
test_wrap_openhands.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_persistent.py feat(opencode): support Copilot subscription backend for headroom models (#2441) (#2445) 2026-07-20 11:02:14 -07:00
test_wrap_proxy_detach.py fix(wrap): detach the shared proxy on Windows so it survives an ungraceful agent close (#1464) 2026-06-30 13:49:28 -05:00
test_wrap_serena_boost.py fix(wrap): stop the Serena pre-index stalling the launch path for 300s (#2945) 2026-08-13 11:46:30 -05:00
test_wrap_stale_marker.py fix: detect and clear stale ANTHROPIC_BASE_URL from crashed wrap sessions (#1768) (#1837) 2026-07-06 08:35:40 -07:00
test_wrap_vibe.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
test_wrap_vscode.py feat(copilot): proxy VS Code models transparently (#2687) 2026-08-03 04:42:48 -07:00
test_wrap_vscode_claude.py feat(claude): support Claude Code in VS Code (#2752) 2026-08-03 20:14:13 -07:00
test_wrap_zcode.py fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00