headroom/tests/test_graph_tokensave.py
Paperinik dca9853ed9
feat(wrap): make tokensave the primary coding-task compressor, Serena the backup (#1230)
## Description

Makes **tokensave**
([github.com/aovestdipaperino/tokensave](https://github.com/aovestdipaperino/tokensave))
the **primary coding-task compressor** that `headroom wrap` installs,
and demotes **Serena** to a **backup**. tokensave is a local semantic
code-graph MCP server (`tokensave serve`): the agent queries it for
symbols, call chains, and impact analysis instead of grepping/reading
whole files — the same role Serena filled, but as a pre-indexed graph.
Serena now only registers when tokensave is unavailable (or when forced
with `--serena`).

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/graph/tokensave_installer.py` (new): fetch the prebuilt
tokensave release binary for the platform (release-binary only — no
`cargo` compile at wrap time); honors `HEADROOM_BINARIES_OFFLINE`;
returns `None` (→ Serena) when no asset exists (e.g. x86_64 macOS) or
the download fails.
- `mcp_registry`: `build_tokensave_spec()`; registration/disable/migrate
go through the existing `ServerSpec` + ownership-ledger flow, identical
to Serena.
- `cli/wrap.py`: new `_setup_coding_compressor` primary/backup policy;
tokensave setup/disable/migrate/index helpers. New flags
`--no-tokensave` (skip primary) and `--serena` (force backup on);
`--no-serena` now means "never register the backup". Default wrap
removes a previously Headroom-installed Serena entry once tokensave is
primary (user-managed entries preserved). `--code-graph` repointed to
tokensave; the legacy `codebase-memory-mcp` install path is dropped
(unwrap still cleans up legacy entries). `unwrap claude|codex` remove a
ledger-owned tokensave entry.
- Strands `HeadroomBundle`: `enable_tokensave_mcp=True` (primary);
`enable_serena_mcp` now defaults `False` (backup).
- `docs/content/docs/proxy.mdx`: `--code-graph` description updated from
codebase-memory-mcp to tokensave.
- Tests: tokensave installer (incl. error paths),
register/disable/migrate, primary/backup policy, and the
binary-resolution/indexing helpers. A scoped
`tests/test_cli/conftest.py` offline guard keeps the CLI suite hermetic.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ uv run pytest -q tests/test_graph_tokensave.py tests/test_cli/test_tokensave_setup.py tests/test_cli/test_tokensave_helpers.py
41 passed

$ uv run pytest -q tests/test_cli/ tests/test_graph.py tests/test_graph_tokensave.py
421 passed   # full CLI + graph suites, incl. all pre-existing Serena/unwrap/registry tests

$ uv run pytest -q tests/test_mcp_registry/ tests/test_proxy_healthchecks.py
passed

$ uv run ruff format --check headroom/ tests/      # 822 files already formatted
$ uv run ruff check <changed files>                # All checks passed!
$ uv run mypy headroom/graph/tokensave_installer.py headroom/mcp_registry/install.py
Success: no issues found in 2 source files

# Coverage on new module
headroom/graph/tokensave_installer.py    99%
```

## Real Behavior Proof

- Environment: macOS (darwin arm64), Python 3.14, `uv` dev env;
tokensave 7.0.2 binary present on PATH and exercised against this repo's
`.tokensave/` graph during development. The installer pins release
**v7.0.2** (SHA-256-verified) across macOS arm64, Linux aarch64/x86_64,
and Windows x86_64/aarch64.
- Exact command / steps: `headroom wrap claude` registers `tokensave
serve` as the primary MCP code-graph server and indexes the project;
with the binary removed from PATH and `HEADROOM_BINARIES_OFFLINE=1`, the
same command falls back to registering Serena. Behavior is pinned by the
unit tests (binary-present → tokensave registered + Serena entry
removed; binary-absent → Serena fallback; `--serena` forces backup on;
`--no-serena` suppresses it; `--no-tokensave` disables primary).
- Observed result: tokensave registered as primary on the binary-present
path; Serena registered on the unavailable path; unwrap removes only
ledger-owned entries.
- Not tested: live end-to-end agent session inside Claude Code / Codex
against a real provider API; Windows/Linux release-asset download
(covered by unit tests with mocked archives, not a live fetch).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

- CHANGELOG is left untouched: this repo generates it via release-please
from Conventional Commits, so a manual edit is N/A.
- `strands/bundle.py` shows 0% patch coverage because that module
hard-imports the optional `strands` SDK, which CI does not install (the
pre-existing `_make_serena_client` was likewise uncovered) — not a
regression.
- A `test (3)` shard failure on `headroom.memory.bridge` is a
pre-existing offline-CI flake (cannot reach huggingface.co); it touches
no file in this PR and the scoped offline guard only applies under
`tests/test_cli/`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 16:55:37 -05:00

284 lines
12 KiB
Python

"""Tests for the tokensave release-binary installer."""
from __future__ import annotations
import io
import tarfile
import zipfile
from pathlib import Path
from types import SimpleNamespace
import pytest
from headroom.graph import tokensave_installer as ts
def _tar_archive(member_name: str = ts.TOKENSAVE_BIN_NAME) -> bytes:
payload = io.BytesIO()
with tarfile.open(fileobj=payload, mode="w:gz") as tar:
data = b"#!/bin/sh\necho version\n"
info = tarfile.TarInfo(name=member_name)
info.size = len(data)
tar.addfile(info, io.BytesIO(data))
return payload.getvalue()
def _zip_archive(member_name: str = "tokensave.exe") -> bytes:
payload = io.BytesIO()
with zipfile.ZipFile(payload, "w") as zf:
zf.writestr(member_name, b"binary")
return payload.getvalue()
class FakeResponse:
def __init__(self, data: bytes) -> None:
self._data = data
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb) -> None:
return None
def read(self) -> bytes:
return self._data
@pytest.mark.parametrize(
("system", "machine", "expected"),
[
("darwin", "arm64", ("tokensave-v9-aarch64-macos.tar.gz", "tar.gz")),
("linux", "aarch64", ("tokensave-v9-aarch64-linux.tar.gz", "tar.gz")),
("linux", "arm64", ("tokensave-v9-aarch64-linux.tar.gz", "tar.gz")),
("linux", "x86_64", ("tokensave-v9-x86_64-linux.tar.gz", "tar.gz")),
("windows", "amd64", ("tokensave-v9-x86_64-windows.zip", "zip")),
("windows", "arm64", ("tokensave-v9-aarch64-windows.zip", "zip")),
],
)
def test_detect_asset_variants(monkeypatch, system, machine, expected) -> None:
monkeypatch.setattr(ts.platform, "system", lambda: system)
monkeypatch.setattr(ts.platform, "machine", lambda: machine)
assert ts._detect_asset("v9") == expected
def test_detect_asset_returns_none_for_intel_mac_and_unknown(monkeypatch) -> None:
monkeypatch.setattr(ts.platform, "system", lambda: "darwin")
monkeypatch.setattr(ts.platform, "machine", lambda: "x86_64")
assert ts._detect_asset("v9") is None # no x86_64-macos asset is published
monkeypatch.setattr(ts.platform, "system", lambda: "solaris")
monkeypatch.setattr(ts.platform, "machine", lambda: "sparc")
assert ts._detect_asset("v9") is None
def test_get_tokensave_path_prefers_path_then_install_dir(monkeypatch, tmp_path: Path) -> None:
on_path = tmp_path / "on-path"
installed = tmp_path / ts.TOKENSAVE_BIN_NAME
installed.write_text("bin")
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr("shutil.which", lambda name: str(on_path))
assert ts.get_tokensave_path() == on_path
monkeypatch.setattr("shutil.which", lambda name: None)
assert ts.get_tokensave_path() == installed
installed.unlink()
assert ts.get_tokensave_path() is None
def test_ensure_offline_returns_none_when_absent(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr("shutil.which", lambda name: None)
monkeypatch.setenv("HEADROOM_BINARIES_OFFLINE", "1")
def _boom(*a, **k):
raise AssertionError("download must not run when offline")
monkeypatch.setattr(ts, "download_tokensave", _boom)
assert ts.ensure_tokensave() is None
def test_ensure_returns_existing_without_download(monkeypatch, tmp_path: Path) -> None:
existing = tmp_path / ts.TOKENSAVE_BIN_NAME
existing.write_text("bin")
monkeypatch.setattr(ts, "get_tokensave_path", lambda: existing)
def _boom(*a, **k):
raise AssertionError("download must not run when binary present")
monkeypatch.setattr(ts, "download_tokensave", _boom)
assert ts.ensure_tokensave() == existing
def test_ensure_returns_none_on_unsupported_platform(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "get_tokensave_path", lambda: None)
monkeypatch.delenv("HEADROOM_BINARIES_OFFLINE", raising=False)
monkeypatch.setattr(ts.platform, "system", lambda: "darwin")
monkeypatch.setattr(ts.platform, "machine", lambda: "x86_64") # no asset
assert ts.ensure_tokensave() is None
def test_download_tokensave_tarball(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr(ts.platform, "system", lambda: "linux")
monkeypatch.setattr(ts.platform, "machine", lambda: "x86_64")
# Synthetic archive bytes won't match the pinned digest; this test covers
# extraction, not integrity, so opt out of verification explicitly.
monkeypatch.setenv("HEADROOM_TOKENSAVE_ALLOW_UNVERIFIED", "1")
monkeypatch.setattr(ts, "urlopen", lambda url, timeout=60: FakeResponse(_tar_archive()))
monkeypatch.setattr(
"subprocess.run", lambda *a, **k: SimpleNamespace(returncode=0, stdout="tokensave 6\n")
)
path = ts.download_tokensave(version="v0.0.0-test")
assert path == tmp_path / ts.TOKENSAVE_BIN_NAME
assert path.exists()
def test_download_tokensave_zip_windows(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr(ts.platform, "system", lambda: "windows")
monkeypatch.setattr(ts.platform, "machine", lambda: "amd64")
monkeypatch.setenv("HEADROOM_TOKENSAVE_ALLOW_UNVERIFIED", "1")
monkeypatch.setattr(ts, "urlopen", lambda url, timeout=60: FakeResponse(_zip_archive()))
monkeypatch.setattr(
"subprocess.run", lambda *a, **k: SimpleNamespace(returncode=0, stdout="tokensave 6\n")
)
path = ts.download_tokensave(version="v0.0.0-test")
assert path == tmp_path / "tokensave.exe"
assert path.exists()
def test_download_raises_for_unsupported_platform(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr(ts.platform, "system", lambda: "darwin")
monkeypatch.setattr(ts.platform, "machine", lambda: "x86_64")
with pytest.raises(RuntimeError, match="no prebuilt tokensave asset"):
ts.download_tokensave(version="v7.0.0")
def test_download_wraps_network_failure(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr(ts.platform, "system", lambda: "linux")
monkeypatch.setattr(ts.platform, "machine", lambda: "x86_64")
def _boom(url, timeout=60):
raise OSError("connection refused")
monkeypatch.setattr(ts, "urlopen", _boom)
with pytest.raises(RuntimeError, match="Failed to download tokensave"):
ts.download_tokensave(version="v7.0.0")
def test_download_raises_when_binary_missing_from_tarball(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr(ts.platform, "system", lambda: "linux")
monkeypatch.setattr(ts.platform, "machine", lambda: "x86_64")
# Archive contains an unrelated member, not the tokensave binary.
monkeypatch.setenv("HEADROOM_TOKENSAVE_ALLOW_UNVERIFIED", "1")
monkeypatch.setattr(
ts, "urlopen", lambda url, timeout=60: FakeResponse(_tar_archive("README.md"))
)
with pytest.raises(RuntimeError, match="binary not found in archive"):
ts.download_tokensave(version="v0.0.0-test")
def test_download_raises_when_binary_missing_from_zip(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr(ts.platform, "system", lambda: "windows")
monkeypatch.setattr(ts.platform, "machine", lambda: "amd64")
monkeypatch.setenv("HEADROOM_TOKENSAVE_ALLOW_UNVERIFIED", "1")
monkeypatch.setattr(
ts, "urlopen", lambda url, timeout=60: FakeResponse(_zip_archive("notes.txt"))
)
with pytest.raises(RuntimeError, match="binary not found in archive"):
ts.download_tokensave(version="v0.0.0-test")
def test_download_tolerates_failed_version_check(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr(ts.platform, "system", lambda: "linux")
monkeypatch.setattr(ts.platform, "machine", lambda: "x86_64")
monkeypatch.setenv("HEADROOM_TOKENSAVE_ALLOW_UNVERIFIED", "1")
monkeypatch.setattr(ts, "urlopen", lambda url, timeout=60: FakeResponse(_tar_archive()))
# Non-zero return code and a raising probe must both be non-fatal.
monkeypatch.setattr(
"subprocess.run", lambda *a, **k: SimpleNamespace(returncode=1, stdout="", stderr="x")
)
assert ts.download_tokensave(version="v0.0.0-test") == tmp_path / ts.TOKENSAVE_BIN_NAME
monkeypatch.setattr(
"subprocess.run", lambda *a, **k: (_ for _ in ()).throw(RuntimeError("probe boom"))
)
assert ts.download_tokensave(version="v0.0.0-test") == tmp_path / ts.TOKENSAVE_BIN_NAME
def test_verify_asset_digest_accepts_matching_hash(monkeypatch) -> None:
import hashlib
data = b"some-release-bytes"
digest = hashlib.sha256(data).hexdigest()
monkeypatch.setattr(ts, "TOKENSAVE_ASSET_DIGESTS", {"asset.tar.gz": digest})
# No exception => verification passed.
ts._verify_asset_digest("asset.tar.gz", data)
def test_verify_asset_digest_rejects_mismatch(monkeypatch) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_ASSET_DIGESTS", {"asset.tar.gz": "00" * 32})
with pytest.raises(RuntimeError, match="failed integrity check"):
ts._verify_asset_digest("asset.tar.gz", b"tampered")
def test_verify_asset_digest_refuses_unpinned_without_optout(monkeypatch) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_ASSET_DIGESTS", {})
monkeypatch.delenv("HEADROOM_TOKENSAVE_ALLOW_UNVERIFIED", raising=False)
with pytest.raises(RuntimeError, match="no pinned SHA-256 digest"):
ts._verify_asset_digest("unknown.tar.gz", b"bytes")
def test_verify_asset_digest_allows_unpinned_with_optout(monkeypatch) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_ASSET_DIGESTS", {})
monkeypatch.setenv("HEADROOM_TOKENSAVE_ALLOW_UNVERIFIED", "1")
ts._verify_asset_digest("unknown.tar.gz", b"bytes") # no exception
def test_download_aborts_on_digest_mismatch(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr(ts.platform, "system", lambda: "linux")
monkeypatch.setattr(ts.platform, "machine", lambda: "x86_64")
monkeypatch.delenv("HEADROOM_TOKENSAVE_ALLOW_UNVERIFIED", raising=False)
# Pin a digest that the synthetic archive cannot match.
monkeypatch.setattr(
ts, "TOKENSAVE_ASSET_DIGESTS", {"tokensave-v7.0.0-x86_64-linux.tar.gz": "00" * 32}
)
monkeypatch.setattr(ts, "urlopen", lambda url, timeout=60: FakeResponse(_tar_archive()))
with pytest.raises(RuntimeError, match="failed integrity check"):
ts.download_tokensave(version="v7.0.0")
# The unverified binary must not have been written.
assert not (tmp_path / ts.TOKENSAVE_BIN_NAME).exists()
def test_download_honors_invalid_url_scheme(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "TOKENSAVE_BIN_DIR", tmp_path)
monkeypatch.setattr(ts.platform, "system", lambda: "linux")
monkeypatch.setattr(ts.platform, "machine", lambda: "x86_64")
monkeypatch.setattr(ts, "GITHUB_RELEASE_URL", "ftp://example.test/releases")
with pytest.raises(RuntimeError, match="Failed to download tokensave"):
ts.download_tokensave(version="v7.0.0")
def test_ensure_returns_none_when_download_fails(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setattr(ts, "get_tokensave_path", lambda: None)
monkeypatch.delenv("HEADROOM_BINARIES_OFFLINE", raising=False)
def _raise(version=None):
raise RuntimeError("download failed")
monkeypatch.setattr(ts, "download_tokensave", _raise)
assert ts.ensure_tokensave() is None
def test_pinned_version_env_override(monkeypatch) -> None:
monkeypatch.setenv("HEADROOM_TOKENSAVE_VERSION", "v9.9.9")
assert ts._pinned_version() == "v9.9.9"
monkeypatch.delenv("HEADROOM_TOKENSAVE_VERSION", raising=False)
assert ts._pinned_version() == ts.TOKENSAVE_VERSION