headroom/tests/test_ccr_mcp_server.py
Rod Boev ad7993bf15
fix(codex): stop pinning Codex memory MCP to one project db (#1269)
## Description

Stop `headroom wrap codex --memory` from pinning the global
`headroom_memory` MCP server to one absolute SQLite path. Today the
wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into
`~/.codex/config.toml`, which makes later Codex sessions either reopen a
stale project-local DB or fail with `unable to open database file` when
that original path disappears. This change lets the MCP server use its
existing per-cwd default again, so each Codex session resolves
`.headroom/memory.db` from the active project instead of a serialized
past cwd. Closes #1147

The current Codex-memory config surface was shaped by
https://github.com/chopratejas/headroom/issues/462 and
https://github.com/chopratejas/headroom/issues/730; this PR keeps that
surface project-scoped again instead of globally pinning one DB.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- remove the injected `--db` argument from the global `headroom_memory`
Codex MCP block while keeping `--user` intact
- preserve the wrap-time local `.headroom/memory.db` setup and
Claude-memory import path for the current project
- treat only wrap-owned Codex markers as snapshot-suppression and
unwrap-cleanup signals, so pre-existing named MCP blocks still back up
and restore
- log a startup diagnostic from `headroom.memory.mcp_server` that
records the configured DB path, config source, cwd/project root,
resolved storage scope, path existence/readability, and whether the path
was static or cwd-derived
- add a shared MCP SDK test stub so both the memory MCP and CCR MCP test
surfaces still run in CI when `mcp` is absent
- make the shared MCP stub re-import target modules under the stubbed
dependency set and restore any pre-existing target module object plus
dotted parent-package attribute state after cleanup
- add focused regressions and guard coverage for the persisted Codex
config shape, named-MCP marker backup and restore, the no-backup
memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the
failed-wrap memory-only cleanup path, the startup-diagnostic path
classification, the shared-store CCR retrieval path, and the shared MCP
stub import lifecycle
- add a `CHANGELOG.md` entry for the user-visible Codex memory scoping
fix

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`uv run mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py
======================== 78 passed, 1 warning in 5.96s ========================
Pytest warning:
PytestConfigWarning: Unknown config option: asyncio_mode
Pytest post-success atexit noise:
PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current'

uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py
All checks passed!

uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check
7 files already formatted
```

## Real Behavior Proof

- Environment: isolated temp project directories, a temp Codex home, the
real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked
missing-`codex` launch path for the failed-wrap cleanup case, and shared
MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still
exercises those paths without a real `mcp` install.
- Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py
tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py
tests/test_mcp_stub.py`; prove the persisted config shape with
`TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`;
prove prepare-only wrap cleanup with
`test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`;
prove failed-wrap cleanup with
`test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`;
guard pre-existing named Codex MCP preservation with
`test_memory_only_wrap_restores_preexisting_named_mcp_block` and
`test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove
the startup diagnostic classifications with
`test_memory_mcp_startup_context_reports_dynamic_project_db` and
`test_memory_mcp_startup_context_reports_static_external_db`; prove the
shared-store CCR retrieval path with
`test_mcp_uses_shared_singleton_store` and
`test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup
with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`,
`test_import_module_with_mcp_stub_reimports_target_and_restores_originals`,
and
`test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`.
- Observed result: the persisted global `headroom_memory` block now
keeps `--user` but omits `--db`; prepare-only memory setup still
bootstraps the current project's `.headroom/memory.db`; `headroom unwrap
codex --no-stop-proxy` now removes both the prepare-only generated
config and the failed-wrap memory-only config instead of leaving
`[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP
blocks remain restorable across both normal and no-backup memory-only
unwrap paths because only wrap-owned markers suppress backups or trigger
named-block cleanup; the memory MCP server now logs whether its DB path
came from the cwd default or an explicit static path, along with the
resolved path and scope it will open; CI can exercise both MCP test
modules even when the `mcp` package is absent from the shard
environment, and the shared stub now re-imports target modules under the
stubbed SDK while restoring both dependency and dotted parent-package
target-module import state after cleanup.
- Not tested: full end-to-end interactive Codex CLI launch.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

The code change stays narrowly scoped to Codex memory config
persistence, cleanup, and startup observability. It does not widen into
larger memory-routing redesign or startup-failure recovery logic.
2026-06-23 07:49:07 -05:00

116 lines
4.9 KiB
Python

from __future__ import annotations
import asyncio
import json
import pytest
from headroom.cache.compression_store import (
get_compression_store,
reset_compression_store,
)
from tests._mcp_stub import import_module_with_mcp_stub
mcp_server = import_module_with_mcp_stub("headroom.ccr.mcp_server")
def test_shared_stats_work_without_fcntl(monkeypatch, tmp_path) -> None:
monkeypatch.setattr(mcp_server, "_HAS_FCNTL", False)
monkeypatch.setattr(mcp_server, "fcntl", None)
monkeypatch.setattr(mcp_server, "SHARED_STATS_DIR", tmp_path)
monkeypatch.setattr(mcp_server, "SHARED_STATS_FILE", tmp_path / "session_stats.jsonl")
monkeypatch.setattr(mcp_server.os, "getpid", lambda: 4242)
monkeypatch.setattr(mcp_server.time, "time", lambda: 1001.0)
event = {"type": "compress", "timestamp": 1000.0}
mcp_server._append_shared_event(event)
raw_lines = mcp_server.SHARED_STATS_FILE.read_text(encoding="utf-8").splitlines()
assert len(raw_lines) == 1
assert json.loads(raw_lines[0]) == {"type": "compress", "timestamp": 1000.0, "pid": 4242}
events = mcp_server._read_shared_events(window_seconds=60)
assert events == [{"type": "compress", "timestamp": 1000.0, "pid": 4242}]
# --- Shared compression store wiring ---------------------------------------
# MCP's _get_local_store() must return the get_compression_store() singleton —
# the same instance the proxy and response_handler use — so content compressed
# on either side is retrievable in-process. These pin that wiring so a private
# store can't creep back.
@pytest.fixture
def fresh_store():
reset_compression_store()
yield
reset_compression_store()
def test_mcp_uses_shared_singleton_store(fresh_store) -> None:
"""MCP's store is the global singleton, not a private instance."""
server = mcp_server.HeadroomMCPServer(check_proxy=False)
assert server._get_local_store() is get_compression_store()
def test_mcp_retrieves_proxy_stored_content(fresh_store) -> None:
"""Content stored via the singleton (as the proxy does) is retrievable
through MCP's local-store path. The HTTP fallback is disabled so this
passes only via the shared store."""
original = '{"some": "original proxy-compressed content"}'
hash_key = get_compression_store().store(original, '{"compressed": true}')
server = mcp_server.HeadroomMCPServer(check_proxy=False)
result = asyncio.run(server._retrieve_content(hash_key, query=None))
assert result.get("source") == "local"
assert result["original_content"] == original
def test_compress_savings_percent_tracks_token_counts(fresh_store) -> None:
"""``savings_percent`` must be the *removed* percentage derived from the
token counts — never the retained percentage. Regression for the inversion
where ``(1 - compression_ratio)`` reported a no-op (0% saved) as 100%."""
pytest.importorskip("mcp", reason="MCP SDK required")
server = mcp_server.HeadroomMCPServer(check_proxy=False)
# Repetitive JSON array — the shape the engine actually compresses.
content = json.dumps([{"id": i, "status": "ok", "kind": "run"} for i in range(40)])
result = server._compress_content(content)
orig = result["original_tokens"]
comp = result["compressed_tokens"]
expected = round((1 - comp / orig) * 100, 1) if orig > 0 else 0
# Reported savings agrees with the token fields (and with tokens_saved).
assert result["savings_percent"] == expected
assert 0.0 <= result["savings_percent"] <= 100.0
if result["tokens_saved"] == 0:
assert result["savings_percent"] == 0.0 # not inverted to 100
else:
assert result["savings_percent"] > 0.0
def test_mcp_retrieve_with_nonmatching_query_returns_full_content(fresh_store) -> None:
"""A query that matches no item above the relevance floor must still return
the stored entry (it exists and is unexpired) rather than the "Content not
found" error, which is reserved for genuine misses."""
original = "the the the the the the the the the the\n" * 5
hash_key = get_compression_store().store(original, "<<small>>")
# Precondition: the query genuinely matches nothing above the BM25 floor.
assert get_compression_store().search(hash_key, "zzqx_nonmatching_token") == []
server = mcp_server.HeadroomMCPServer(check_proxy=False)
result = asyncio.run(server._retrieve_content(hash_key, query="zzqx_nonmatching_token"))
assert "error" not in result
assert result.get("source") == "local"
assert result["original_content"] == original
assert result["count"] == 0
def test_mcp_retrieve_missing_hash_still_errors(fresh_store) -> None:
"""A genuinely missing hash must still report "Content not found"."""
server = mcp_server.HeadroomMCPServer(check_proxy=False)
result = asyncio.run(server._retrieve_content("nonexistent_hash", query="anything"))
assert "Content not found" in result.get("error", "")