mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic.
116 lines
4.9 KiB
Python
116 lines
4.9 KiB
Python
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import json
|
|
|
|
import pytest
|
|
|
|
from headroom.cache.compression_store import (
|
|
get_compression_store,
|
|
reset_compression_store,
|
|
)
|
|
from tests._mcp_stub import import_module_with_mcp_stub
|
|
|
|
mcp_server = import_module_with_mcp_stub("headroom.ccr.mcp_server")
|
|
|
|
|
|
def test_shared_stats_work_without_fcntl(monkeypatch, tmp_path) -> None:
|
|
monkeypatch.setattr(mcp_server, "_HAS_FCNTL", False)
|
|
monkeypatch.setattr(mcp_server, "fcntl", None)
|
|
monkeypatch.setattr(mcp_server, "SHARED_STATS_DIR", tmp_path)
|
|
monkeypatch.setattr(mcp_server, "SHARED_STATS_FILE", tmp_path / "session_stats.jsonl")
|
|
monkeypatch.setattr(mcp_server.os, "getpid", lambda: 4242)
|
|
monkeypatch.setattr(mcp_server.time, "time", lambda: 1001.0)
|
|
|
|
event = {"type": "compress", "timestamp": 1000.0}
|
|
mcp_server._append_shared_event(event)
|
|
|
|
raw_lines = mcp_server.SHARED_STATS_FILE.read_text(encoding="utf-8").splitlines()
|
|
assert len(raw_lines) == 1
|
|
assert json.loads(raw_lines[0]) == {"type": "compress", "timestamp": 1000.0, "pid": 4242}
|
|
|
|
events = mcp_server._read_shared_events(window_seconds=60)
|
|
assert events == [{"type": "compress", "timestamp": 1000.0, "pid": 4242}]
|
|
|
|
|
|
# --- Shared compression store wiring ---------------------------------------
|
|
# MCP's _get_local_store() must return the get_compression_store() singleton —
|
|
# the same instance the proxy and response_handler use — so content compressed
|
|
# on either side is retrievable in-process. These pin that wiring so a private
|
|
# store can't creep back.
|
|
|
|
|
|
@pytest.fixture
|
|
def fresh_store():
|
|
reset_compression_store()
|
|
yield
|
|
reset_compression_store()
|
|
|
|
|
|
def test_mcp_uses_shared_singleton_store(fresh_store) -> None:
|
|
"""MCP's store is the global singleton, not a private instance."""
|
|
server = mcp_server.HeadroomMCPServer(check_proxy=False)
|
|
assert server._get_local_store() is get_compression_store()
|
|
|
|
|
|
def test_mcp_retrieves_proxy_stored_content(fresh_store) -> None:
|
|
"""Content stored via the singleton (as the proxy does) is retrievable
|
|
through MCP's local-store path. The HTTP fallback is disabled so this
|
|
passes only via the shared store."""
|
|
original = '{"some": "original proxy-compressed content"}'
|
|
hash_key = get_compression_store().store(original, '{"compressed": true}')
|
|
|
|
server = mcp_server.HeadroomMCPServer(check_proxy=False)
|
|
result = asyncio.run(server._retrieve_content(hash_key, query=None))
|
|
|
|
assert result.get("source") == "local"
|
|
assert result["original_content"] == original
|
|
|
|
|
|
def test_compress_savings_percent_tracks_token_counts(fresh_store) -> None:
|
|
"""``savings_percent`` must be the *removed* percentage derived from the
|
|
token counts — never the retained percentage. Regression for the inversion
|
|
where ``(1 - compression_ratio)`` reported a no-op (0% saved) as 100%."""
|
|
pytest.importorskip("mcp", reason="MCP SDK required")
|
|
server = mcp_server.HeadroomMCPServer(check_proxy=False)
|
|
|
|
# Repetitive JSON array — the shape the engine actually compresses.
|
|
content = json.dumps([{"id": i, "status": "ok", "kind": "run"} for i in range(40)])
|
|
result = server._compress_content(content)
|
|
|
|
orig = result["original_tokens"]
|
|
comp = result["compressed_tokens"]
|
|
expected = round((1 - comp / orig) * 100, 1) if orig > 0 else 0
|
|
|
|
# Reported savings agrees with the token fields (and with tokens_saved).
|
|
assert result["savings_percent"] == expected
|
|
assert 0.0 <= result["savings_percent"] <= 100.0
|
|
if result["tokens_saved"] == 0:
|
|
assert result["savings_percent"] == 0.0 # not inverted to 100
|
|
else:
|
|
assert result["savings_percent"] > 0.0
|
|
|
|
|
|
def test_mcp_retrieve_with_nonmatching_query_returns_full_content(fresh_store) -> None:
|
|
"""A query that matches no item above the relevance floor must still return
|
|
the stored entry (it exists and is unexpired) rather than the "Content not
|
|
found" error, which is reserved for genuine misses."""
|
|
original = "the the the the the the the the the the\n" * 5
|
|
hash_key = get_compression_store().store(original, "<<small>>")
|
|
# Precondition: the query genuinely matches nothing above the BM25 floor.
|
|
assert get_compression_store().search(hash_key, "zzqx_nonmatching_token") == []
|
|
|
|
server = mcp_server.HeadroomMCPServer(check_proxy=False)
|
|
result = asyncio.run(server._retrieve_content(hash_key, query="zzqx_nonmatching_token"))
|
|
|
|
assert "error" not in result
|
|
assert result.get("source") == "local"
|
|
assert result["original_content"] == original
|
|
assert result["count"] == 0
|
|
|
|
|
|
def test_mcp_retrieve_missing_hash_still_errors(fresh_store) -> None:
|
|
"""A genuinely missing hash must still report "Content not found"."""
|
|
server = mcp_server.HeadroomMCPServer(check_proxy=False)
|
|
result = asyncio.run(server._retrieve_content("nonexistent_hash", query="anything"))
|
|
assert "Content not found" in result.get("error", "")
|