headroom/tests/test_cli/test_unwrap_claude.py
Tejas Chopra f27f235032
fix(wrap): stop concurrent wrap sessions clobbering settings.local.json (#3232)
## Description

Several `headroom wrap` sessions in one project each write the proxy URL
into
`.claude/settings.local.json` and restore it on exit. That
read-modify-write was
unsynchronised. The write itself is atomic so the file never tears, but
the
updates were still lost against each other:

- **Live sessions were silently unrouted.** The first session to exit
deleted the
key while its siblings were still running. They kept working, but their
traffic
  stopped going through the proxy — no error, no warning, no savings.
- **A dead proxy was written back into the project.** A session that
started
second captured the *first* session's proxy URL as "the original", so
its exit
restored a URL pointing at a port that was already gone. Every later
session in
  that project then failed to connect.
- **SIGTERM/SIGHUP never ran the restore at all.** `cleanup` was
registered as the
handler, but a Python signal handler that returns normally does not
unwind the
stack — under PEP 475 the interrupted `waitpid` is simply retried. The
`finally`
block that restores `settings.local.json` never ran, while the handler
had
  already terminated the proxy underneath a child that was still alive.

Closes #3205

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- **`_wrap_settings_lock`** — an exclusive OS lock (flock /
`msvcrt.locking`) held
across the settings read-modify-write. A workspace that cannot hold lock
state
  degrades to the previous behaviour rather than failing, matching
  `_proxy_start_lock`.
- **`.headroom_wrap_owners.json`** — a sidecar recording, per env key,
the true
pre-wrap `original` plus the live sessions holding it. The first writer
records
the original; later writers inherit it and are flagged `inherited`, so
no
session restores a value it did not observe first-hand. A session exits
without
restoring while a sibling still holds the key. Dead holders are pruned
with the
same conservative PID+identity liveness the proxy-client markers use, so
a
  SIGKILLed session cannot wedge the key.
- **`unwrap` passes `force=True`** — unwrap is the user explicitly
asking for
their settings back, so it drops every claim instead of deferring to a
live
sibling and silently printing success while leaving the proxy URL in the
file.
- **The #2221 self-heal passes `dead_ports`** — a wrapper process can
outlive its
proxy (proxy alone SIGKILLed). Its claim would otherwise veto the
self-heal and
  leave `ANTHROPIC_BASE_URL` pointing at a port just proven dead.
- **`_rehome_wrap_marker`** — the wrap marker has one slot, won by the
last
writer. When that writer exits while a sibling still owns the key, the
marker is
rewritten to describe the survivor (carrying the record's true
original), so the
survivor keeps its #2221 self-heal record instead of being left with a
marker
  describing a dead process.
- **`_exit_on_signal`** replaces `cleanup` as the SIGTERM/SIGHUP
handler. Raising
`SystemExit` unwinds, so the settings restore actually runs and cleanup
happens
  exactly once from `finally`.
- **`_proxy_start_lock` now shares `_locked_file`** with the new
settings lock
  rather than carrying a second verbatim copy of the platform branches.

## Testing

- [x] Unit tests pass (`pytest`) — full suite, 11518 passed / 588
skipped
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

`tests/test_wrap_concurrent_settings.py` (14 tests) covers: a sibling
exit leaving
survivors routed, the last session out restoring the true original, a
pre-existing
user URL surviving the whole cycle, three sessions in every exit order,
a crashed
session not wedging the key, forced unwrap past a live session, a holder
that
outlived its proxy not vetoing the self-heal, marker rehoming, and the
signal-handler unwind.

### Test Output

```text
$ uv run pytest tests/test_cli/test_unwrap_claude.py tests/test_cli/test_wrap_claude_base_url.py \
    tests/test_cli/test_wrap_claude_finally_unbound.py tests/test_cli/test_wrap_claude_vertex_proxy_env.py \
    tests/test_cli/test_wrap_claude.py tests/test_cli/test_wrap_dead_marker_selfheal.py \
    tests/test_cli/test_wrap_helpers.py tests/test_cli/test_wrap_stale_marker.py \
    tests/test_cli/test_wrap_persistent.py tests/test_wrap_concurrent_settings.py tests/test_cli_doctor.py -q

tests/test_wrap_concurrent_settings.py ..............                    [ 72%]
tests/test_cli_doctor.py ............................................... [ 89%]
...............................                                          [100%]

============================= 285 passed in 3.01s ==============================

$ uv run pytest tests/ -q
======== 11518 passed, 588 skipped, 6036 warnings in 1831.34s (0:30:31) ========

$ uv run ruff check .
All checks passed!

$ uv run mypy headroom
Success: no issues found in 527 source files
```

## Real Behavior Proof

- **Environment:** macOS 15 (Darwin 25.4.0), Python 3.12.13, repo venv,
Claude
  provider path (`ANTHROPIC_BASE_URL` in `.claude/settings.local.json`).
- **Exact command / steps:** a script spawning **two real OS processes**
— no
  mocks, real PIDs, real files — that call the same
`_write_claude_wrap_base_url` / `_restore_claude_wrap_base_url` helpers
`wrap claude` uses. The project starts with a real user gateway already
set.
Session A (port 8787) starts, session B (port 8788) starts 0.7s later, A
exits
while B is still running, then B exits. Run identically on `main` and on
this
  branch.

**Before (on `main`) — both bugs visible:**

```text
start                       : {"ANTHROPIC_BASE_URL": "https://my-gateway.example.com"}
  session port=8787 started, remembers previous='https://my-gateway.example.com'
  session port=8788 started, remembers previous='http://127.0.0.1:8787'
both sessions running       : {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8788"}
  session port=8787 exited
after FIRST session exits   : {"ANTHROPIC_BASE_URL": "https://my-gateway.example.com"}
  session port=8788 exited
after LAST session exits    : {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}
```

Session B is still running, but after A exits the proxy URL is gone from
under it
— B is unrouted with no error. And the final state is
`http://127.0.0.1:8787`: a
dead proxy left permanently in the user's project, with their real
gateway lost.

**After (this branch):**

```text
start                       : {"ANTHROPIC_BASE_URL": "https://my-gateway.example.com"}
  session port=8787 started, remembers previous='https://my-gateway.example.com'
  session port=8788 started, remembers previous='http://127.0.0.1:8787'
both sessions running       : {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8788"}
  session port=8787 exited
after FIRST session exits   : {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8788"}
  session port=8788 exited
after LAST session exits    : {"ANTHROPIC_BASE_URL": "https://my-gateway.example.com"}
```

B stays routed after A exits, and the last session out restores the
user's real
gateway.

- **Observed result:** matches the intent on both counts — no unrouting,
no dead
  proxy residue, user's pre-existing URL preserved.
- **Not tested:** Windows (`msvcrt.locking`) — the lock and dead-holder
pruning
  are exercised on POSIX only; the Windows branch is the same code path
`_proxy_start_lock` has shipped with. No live end-to-end run against a
real
Anthropic endpoint with two concurrent `claude` CLIs; the proof above
drives the
same helpers out of two real processes instead. Foundry/Vertex key
variants are
covered by unit tests, not by a live run. Real SIGTERM/SIGHUP delivery
to a
running `wrap claude` was not exercised end to end — the handler's
unwind is
covered by a unit test, and full signal delivery would need a spawned
and
  killed subprocess, which the existing #1768 test also declined to do.

## Runtime Rollout Safety

- **Rollout-managed feature(s):** none — this is an unconditional
correctness fix
  on the wrap settings path.
- **Minimum rollout channel:** n/a.
- **Stable/default behavior changed:** yes, three ways. (1) A wrap
session exiting
while a sibling holds the key now leaves the key in place instead of
removing
  it. (2) SIGTERM/SIGHUP now unwinds, so the child CLI is terminated by
  `subprocess.run`'s cleanup rather than being left running against a
  torn-down proxy. (3) Two new sidecar files appear next to
`settings.local.json`: `.headroom_wrap_owners.json` (removed when the
last
holder exits) and `.headroom_wrap_settings.lock` (retained by design —
deleting
  a live lock file creates an inode-replacement race).
- **Kill switch / disable path:** none. A workspace where the lock file
cannot be
created degrades to the previous unsynchronised behaviour automatically.
- **Unsafe override required:** no.
- **Qualification impact:** none beyond the wrap settings path.
- **Rollback path:** revert the commit; the sidecar files are ignored by
older
  versions and can be deleted safely.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

- The ownership record is keyed per env key, so `ANTHROPIC_BASE_URL`,
the
Foundry/Vertex variants and the tool-search entry are tracked
independently.
- Documentation: the behaviour is documented in the helper docstrings
rather than
user-facing docs — the sidecar files are internal state a user never
configures.
- Follow-up worth considering: `.headroom_wrap_settings.lock` is
intentionally
never deleted (matching `_proxy_start_lock`'s retention rationale), so
it stays
in `.claude/` after `unwrap`. Removing it safely needs a separate think
about
  the inode-replacement race.

Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-23 22:36:17 -07:00

478 lines
16 KiB
Python

from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import patch
import pytest
from click.testing import CliRunner
from headroom import paths
from headroom.cli import wrap as wrap_cli
from headroom.cli.main import main
@pytest.fixture
def runner() -> CliRunner:
return CliRunner()
@pytest.fixture(autouse=True)
def _no_persistent_manifest(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda _port: None)
def test_remove_claude_managed_hooks_preserves_unrelated_hooks(tmp_path: Path) -> None:
settings = tmp_path / "settings.json"
settings.write_text(
json.dumps(
{
"model": "opus",
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": (
"headroom init hook ensure --marker headroom-init-claude"
),
},
{"type": "command", "command": "echo keep"},
],
}
],
"SessionStart": [
{"matcher": "startup", "hooks": [{"type": "command", "command": "keep"}]}
],
},
}
)
+ "\n",
encoding="utf-8",
)
assert wrap_cli._remove_claude_managed_hooks(settings) is True
payload = json.loads(settings.read_text(encoding="utf-8"))
pre_tool_hooks = payload["hooks"]["PreToolUse"][0]["hooks"]
assert pre_tool_hooks == [{"type": "command", "command": "echo keep"}]
assert payload["hooks"]["SessionStart"][0]["hooks"][0]["command"] == "keep"
def test_unwrap_claude_removes_mcp_purges_retired_hook_and_stops_proxy(
runner: CliRunner,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
home = str(tmp_path)
monkeypatch.setenv("HOME", home)
monkeypatch.setenv("USERPROFILE", home)
monkeypatch.delenv("HEADROOM_WORKSPACE_DIR", raising=False)
bin_dir = paths.bin_dir()
claude_dir = tmp_path / ".claude"
claude_dir.mkdir()
hooks_dir = claude_dir / "hooks"
hooks_dir.mkdir()
hook_script = hooks_dir / "rtk-rewrite.sh"
hook_script.write_text(f'#!/bin/sh\nexec {bin_dir / "rtk"} "$@"\n', encoding="utf-8")
settings = claude_dir / "settings.json"
settings.write_text(
json.dumps(
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [{"type": "command", "command": str(hook_script)}],
}
]
}
}
)
+ "\n",
encoding="utf-8",
)
stopped: list[int] = []
unregistered: list[str] = []
class Registrar:
name = "claude"
def detect(self) -> bool:
return True
def unregister_server(self, server_name: str) -> bool:
unregistered.append(server_name)
return True
def get_server(self, server_name: str):
return None
with (
patch("headroom.mcp_registry.ClaudeRegistrar", return_value=Registrar()),
patch(
"headroom.cli.wrap._stop_local_proxy_for_unwrap",
side_effect=lambda port: stopped.append(port) or "stopped",
),
):
result = runner.invoke(main, ["unwrap", "claude", "--port", "9999"])
assert result.exit_code == 0, result.output
assert unregistered == ["headroom", "codebase-memory-mcp"]
assert stopped == [9999]
assert "Stopped local Headroom proxy on port 9999" in result.output
# The leftover retired context-tool hook is purged end-to-end by unwrap
# (via purge_context_tool_artifacts), leaving no hooks behind.
assert "hooks" not in json.loads(settings.read_text(encoding="utf-8"))
def test_unwrap_claude_preserves_user_managed_serena(
runner: CliRunner,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path / ".headroom"))
unregistered: list[str] = []
class Registrar:
name = "claude"
def detect(self) -> bool:
return True
def unregister_server(self, server_name: str) -> bool:
unregistered.append(server_name)
return True
def get_server(self, server_name: str):
if server_name == "serena":
from headroom.mcp_registry.base import ServerSpec
return ServerSpec(name="serena", command="/usr/local/bin/custom-serena")
return None
with (
patch("headroom.mcp_registry.ClaudeRegistrar", return_value=Registrar()),
patch("headroom.cli.wrap._remove_claude_managed_hooks", return_value=False),
patch("headroom.cli.wrap._stop_local_proxy_for_unwrap"),
):
result = runner.invoke(main, ["unwrap", "claude"])
assert result.exit_code == 0, result.output
assert unregistered == ["headroom", "codebase-memory-mcp"]
def test_unwrap_claude_removes_headroom_installed_serena(
runner: CliRunner,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path / ".headroom"))
from headroom.mcp_registry import build_serena_spec
from headroom.mcp_registry.ledger import record_install
serena_spec = build_serena_spec("claude-code")
record_install("claude", serena_spec)
unregistered: list[str] = []
class Registrar:
name = "claude"
def detect(self) -> bool:
return True
def unregister_server(self, server_name: str) -> bool:
unregistered.append(server_name)
return True
def get_server(self, server_name: str):
if server_name == "serena":
return serena_spec
return None
with (
patch("headroom.mcp_registry.ClaudeRegistrar", return_value=Registrar()),
patch("headroom.cli.wrap._remove_claude_managed_hooks", return_value=False),
patch("headroom.cli.wrap._stop_local_proxy_for_unwrap"),
):
result = runner.invoke(main, ["unwrap", "claude"])
assert result.exit_code == 0, result.output
assert unregistered == ["headroom", "codebase-memory-mcp", "serena"]
assert "Removed Headroom-installed Serena MCP server" in result.output
def test_unwrap_claude_keep_flags_skip_cleanup(
runner: CliRunner,
) -> None:
with (
patch("headroom.mcp_registry.ClaudeRegistrar") as registrar,
patch("headroom.cli.wrap._remove_claude_managed_hooks", return_value=False),
patch("headroom.cli.wrap._stop_local_proxy_for_unwrap") as stop_proxy,
):
result = runner.invoke(
main,
["unwrap", "claude", "--keep-mcp", "--no-stop-proxy"],
)
assert result.exit_code == 0, result.output
registrar.assert_not_called()
stop_proxy.assert_not_called()
def test_unwrap_claude_restores_all_base_url_modes(runner: CliRunner) -> None:
restore_calls: list[dict[str, object]] = []
def restore_base_url(previous: str | None, **kwargs: object) -> None:
restore_calls.append({"previous": previous, **kwargs})
with patch("headroom.cli.wrap._restore_claude_wrap_base_url", side_effect=restore_base_url):
result = runner.invoke(
main,
["unwrap", "claude", "--keep-mcp", "--no-stop-proxy"],
)
assert result.exit_code == 0, result.output
settings_path = Path.cwd() / ".claude" / "settings.local.json"
assert restore_calls == [
{
"previous": None,
"foundry_mode": False,
"vertex_mode": False,
"settings_path": settings_path,
# unwrap is the user asking for their settings back, so it drops
# every wrap session's ownership claim instead of deferring to a
# live sibling and silently doing nothing (#3205).
"force": True,
},
{
"previous": None,
"foundry_mode": True,
"vertex_mode": False,
"settings_path": settings_path,
"force": True,
},
{
"previous": None,
"foundry_mode": False,
"vertex_mode": True,
"settings_path": settings_path,
"force": True,
},
]
def test_unwrap_claude_stops_claude_owned_persistent_deployment(
runner: CliRunner,
monkeypatch: pytest.MonkeyPatch,
) -> None:
class Manifest:
profile = "unwrap-2340"
targets = ["claude"]
tool_envs = {"claude": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}
mutations: list[object] = []
supervisor_kind = "service"
stopped: list[str] = []
deactivated: list[str] = []
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: Manifest())
monkeypatch.setattr(
"headroom.cli.install._deactivate_deployment_mutations",
lambda manifest: deactivated.append(manifest.profile),
)
monkeypatch.setattr(
"headroom.cli.install._stop_deployment",
lambda manifest: stopped.append(manifest.profile),
)
with (
patch("headroom.cli.wrap._stop_local_proxy_for_unwrap") as stop_local,
):
result = runner.invoke(
main,
["unwrap", "claude", "--keep-mcp", "--port", "8787"],
)
assert result.exit_code == 0, result.output
stop_local.assert_not_called()
assert deactivated == ["unwrap-2340"]
assert stopped == ["unwrap-2340"]
assert "Stopped Claude-owned persistent deployment 'unwrap-2340' on port 8787." in result.output
assert "Claude is no longer durably wrapped by Headroom." in result.output
def test_unwrap_claude_reports_ambiguous_same_port_persistent_deployment(
runner: CliRunner,
monkeypatch: pytest.MonkeyPatch,
) -> None:
class Manifest:
profile = "shared-proxy"
targets = ["codex"]
tool_envs = {"codex": {"OPENAI_BASE_URL": "http://127.0.0.1:8787"}}
mutations: list[object] = []
supervisor_kind = "service"
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: Manifest())
with patch("headroom.cli.wrap._stop_local_proxy_for_unwrap") as stop_local:
result = runner.invoke(
main,
["unwrap", "claude", "--keep-mcp", "--port", "8787"],
)
assert result.exit_code == 0, result.output
stop_local.assert_not_called()
assert "same-port persistent deployment 'shared-proxy' still owns port 8787" in result.output
assert "headroom install stop --profile shared-proxy" in result.output
assert "Claude is no longer durably wrapped by Headroom." not in result.output
def test_unwrap_claude_warns_about_same_port_inherited_env(
runner: CliRunner,
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("ANTHROPIC_BASE_URL", "http://127.0.0.1:8787")
with patch("headroom.cli.wrap._stop_local_proxy_for_unwrap", return_value="stopped"):
result = runner.invoke(
main,
["unwrap", "claude", "--keep-mcp", "--port", "8787"],
)
assert result.exit_code == 0, result.output
assert "current shell still exports ANTHROPIC_BASE_URL for port 8787" in result.output
assert "Claude is no longer durably wrapped by Headroom." not in result.output
def test_unwrap_claude_ignores_malformed_inherited_env_port(
runner: CliRunner,
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("ANTHROPIC_BASE_URL", "http://127.0.0.1:notaport")
with patch("headroom.cli.wrap._stop_local_proxy_for_unwrap", return_value="stopped"):
result = runner.invoke(
main,
["unwrap", "claude", "--keep-mcp", "--port", "8787"],
)
assert result.exit_code == 0, result.output
assert "current shell still exports ANTHROPIC_BASE_URL" not in result.output
assert "Claude is no longer durably wrapped by Headroom." in result.output
def test_remove_claude_managed_hooks_removes_init_hooks_and_env(tmp_path: Path) -> None:
settings = tmp_path / "settings.json"
settings.write_text(
json.dumps(
{
"model": "opus",
"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787", "FOO": "bar"},
"hooks": {
"SessionStart": [
{
"matcher": "startup|resume",
"hooks": [
{
"type": "command",
"command": (
"/home/u/.local/bin/headroom init hook ensure "
"--profile init-user --marker headroom-init-claude"
),
"timeout": 15,
}
],
}
],
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "headroom init hook ensure --marker headroom-init-claude",
},
{"type": "command", "command": "echo keep-me"},
],
}
],
},
}
)
+ "\n",
encoding="utf-8",
)
assert wrap_cli._remove_claude_managed_hooks(settings) is True
payload = json.loads(settings.read_text(encoding="utf-8"))
# ANTHROPIC_BASE_URL stripped; unrelated env var preserved
assert payload.get("env") == {"FOO": "bar"}
# SessionStart removed entirely (its only hook was the init marker)
assert "SessionStart" not in payload.get("hooks", {})
# PreToolUse: init-marker hook gone, unrelated hook kept
assert payload["hooks"]["PreToolUse"][0]["hooks"] == [
{"type": "command", "command": "echo keep-me"}
]
assert payload["model"] == "opus"
def test_remove_claude_managed_hooks_strips_env_without_hooks(tmp_path: Path) -> None:
# Regression: unwrap previously returned early when no hooks existed,
# leaving init's ANTHROPIC_BASE_URL behind in settings.json.
settings = tmp_path / "settings.json"
settings.write_text(
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}) + "\n",
encoding="utf-8",
)
assert wrap_cli._remove_claude_managed_hooks(settings) is True
payload = json.loads(settings.read_text(encoding="utf-8"))
assert "env" not in payload # emptied env dict is dropped
def test_remove_claude_managed_hooks_noop_when_nothing_managed(tmp_path: Path) -> None:
settings = tmp_path / "settings.json"
original = {
"model": "opus",
"env": {"FOO": "bar"},
"hooks": {
"PreToolUse": [
{"matcher": "Bash", "hooks": [{"type": "command", "command": "echo hi"}]}
]
},
}
settings.write_text(json.dumps(original) + "\n", encoding="utf-8")
assert wrap_cli._remove_claude_managed_hooks(settings) is False
# nothing managed -> file untouched
assert json.loads(settings.read_text(encoding="utf-8")) == original
def test_remove_claude_managed_hooks_strips_enable_tool_search(tmp_path: Path) -> None:
# unwrap must remove BOTH env vars init writes (ANTHROPIC_BASE_URL +
# ENABLE_TOOL_SEARCH, GH #746), leaving user-set vars intact.
settings = tmp_path / "settings.json"
settings.write_text(
json.dumps(
{
"env": {
"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787",
"ENABLE_TOOL_SEARCH": "true",
"KEEP": "1",
}
}
)
+ "\n",
encoding="utf-8",
)
assert wrap_cli._remove_claude_managed_hooks(settings) is True
payload = json.loads(settings.read_text(encoding="utf-8"))
assert payload["env"] == {"KEEP": "1"}