headroom/tests/test_memory_sync.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

761 lines
26 KiB
Python
Raw Normal View History

Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
"""Comprehensive tests for the universal memory sync engine.
Tests cover:
- Core sync: import, export, bidirectional
- Idempotency and deduplication
- Fast no-op detection
- Lineage and governance metadata
- Claude Code adapter: read/write frontmatter files
- Codex adapter: read/write AGENTS.md sections
- Cross-agent interop: save in one agent, find in another
"""
from __future__ import annotations
import hashlib
import json
import time
from dataclasses import dataclass, field
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
import pytest
from headroom.memory.sync import (
fix(memory): use ONNX embedder for `wrap --memory` sync (#1092) (#1262) ## Description `headroom wrap --memory` could never import memories: the startup sync subprocess (`python -m headroom.memory.sync`) and the in-process Codex memory import both built their backend with `LocalBackendConfig(db_path=...)`, which defaults `embedder_backend` to `"local"` — sentence-transformers + PyTorch (~2 GB). On the proxy extras that dependency is absent, so sync crashed with `ImportError: sentence-transformers is required for LocalEmbedder` while the proxy itself served memory fine via the torch-free ONNX backend. This routes both paths through a shared `_build_sync_backend` helper that uses `embedder_backend="onnx"`, matching the proxy MCP server (`headroom/memory/mcp_server.py`). Closes #1092 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync.py`: added `_build_sync_backend(db_path)` that constructs the backend with `embedder_backend="onnx"`; the sync CLI subprocess now uses it. - `headroom/cli/wrap.py`: the in-process Claude→DB memory import (Codex wrap path) now uses the same helper instead of the LOCAL-defaulting `LocalBackendConfig`. - `tests/test_memory_sync.py`: added `test_sync_backend_uses_onnx_embedder` regression test. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality ### Test Output ```text $ python -m pytest tests/test_memory_sync.py -q 31 passed $ python -m ruff check headroom/memory/sync.py headroom/cli/wrap.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, headroom on branch fix/1092-memory-sync-onnx-embedder - Exact command / steps: Ran the memory-sync suite + ruff, and an import smoke that builds the sync backend: `python -c "from headroom.memory.sync import _build_sync_backend; print(_build_sync_backend('x.db')._config.embedder_backend)"`. - Observed result: 31 tests pass (incl. the new regression test), ruff clean, and the smoke prints `onnx` — the sync backend no longer defaults to the sentence-transformers embedder. - Not tested: Did not run a full live `headroom wrap claude --memory` end to end (needs the ONNX model download + Claude memory files); the same-model (all-MiniLM-L6-v2, 384-dim) ONNX backend the proxy already uses keeps vectors DB-compatible, so no migration is involved. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 05:08:12 +02:00
_build_sync_backend,
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
sync,
sync_export,
sync_import,
)
from headroom.memory.sync_adapters.claude_code import (
ClaudeCodeAdapter,
_parse_frontmatter,
encode_claude_project_path,
get_claude_memory_dir,
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
)
from headroom.memory.sync_adapters.codex_agent import CodexAdapter
# ---------------------------------------------------------------------------
# Fake backend for testing (no real DB/embeddings needed)
# ---------------------------------------------------------------------------
@dataclass
class FakeMemory:
id: str = ""
content: str = ""
user_id: str = ""
category: str = ""
importance: float = 0.5
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
metadata: dict[str, Any] = field(default_factory=dict)
class FakeBackend:
"""In-memory backend for testing sync without real DB."""
def __init__(self) -> None:
self._memories: list[FakeMemory] = []
self._next_id = 1
async def get_user_memories(self, user_id: str, limit: int = 500) -> list[FakeMemory]:
return [m for m in self._memories if m.user_id == user_id][:limit]
async def save_memory(
self,
content: str,
user_id: str,
importance: float = 0.5,
metadata: dict[str, Any] | None = None,
**kwargs: Any,
) -> FakeMemory:
mem = FakeMemory(
id=f"mem_{self._next_id:04d}",
content=content,
user_id=user_id,
importance=importance,
metadata=metadata or {},
)
self._next_id += 1
self._memories.append(mem)
return mem
def add_memory(self, content: str, user_id: str = "tcms", **kwargs: Any) -> FakeMemory:
"""Sync helper to pre-populate memories."""
mem = FakeMemory(
id=f"mem_{self._next_id:04d}",
content=content,
user_id=user_id,
metadata=kwargs.get("metadata", {}),
importance=kwargs.get("importance", 0.5),
)
self._next_id += 1
self._memories.append(mem)
return mem
# ---------------------------------------------------------------------------
# Core sync tests
# ---------------------------------------------------------------------------
class TestSyncImport:
"""Test importing from agent files into DB."""
@pytest.fixture
def backend(self):
return FakeBackend()
@pytest.fixture
def claude_dir(self, tmp_path):
d = tmp_path / "memory"
d.mkdir()
return d
def _write_claude_memory(
self, memory_dir: Path, name: str, content: str, **fm_fields: str
) -> None:
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
slug = name.lower().replace(" ", "_")
fields = {"name": name, "description": content[:80], "type": "project", **fm_fields}
fm_lines = ["---"]
for k, v in fields.items():
fm_lines.append(f"{k}: {v}")
fm_lines.append("---")
(memory_dir / f"{slug}.md").write_text("\n".join(fm_lines) + f"\n\n{content}\n")
@pytest.mark.asyncio
async def test_import_claude_files_to_db(self, backend, claude_dir):
self._write_claude_memory(claude_dir, "Project codename", "The secret name is TC")
self._write_claude_memory(claude_dir, "Dark mode", "User prefers dark mode")
adapter = ClaudeCodeAdapter(claude_dir)
imported = await sync_import(backend, adapter, "tcms")
assert imported == 2
mems = await backend.get_user_memories("tcms")
contents = {m.content for m in mems}
assert "The secret name is TC" in contents
assert "User prefers dark mode" in contents
@pytest.mark.asyncio
async def test_import_skips_existing(self, backend, claude_dir):
"""Memories already in DB are not re-imported."""
backend.add_memory(
"The secret name is TC",
metadata={"content_hash": hashlib.sha256(b"The secret name is TC").hexdigest()[:16]},
)
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
self._write_claude_memory(claude_dir, "Project codename", "The secret name is TC")
self._write_claude_memory(claude_dir, "New fact", "Something new")
adapter = ClaudeCodeAdapter(claude_dir)
imported = await sync_import(backend, adapter, "tcms")
assert imported == 1 # Only "Something new"
@pytest.mark.asyncio
async def test_import_preserves_lineage(self, backend, claude_dir):
self._write_claude_memory(claude_dir, "Fact", "Important fact")
adapter = ClaudeCodeAdapter(claude_dir)
await sync_import(backend, adapter, "tcms")
mems = await backend.get_user_memories("tcms")
assert len(mems) == 1
assert mems[0].metadata["source_agent"] == "claude"
assert mems[0].metadata["source_file"] == "fact.md"
assert "content_hash" in mems[0].metadata
assert mems[0].metadata["sync_direction"] == "import"
class TestSyncExport:
"""Test exporting from DB to agent files."""
@pytest.fixture
def backend(self):
return FakeBackend()
@pytest.fixture
def claude_dir(self, tmp_path):
d = tmp_path / "memory"
d.mkdir()
return d
@pytest.mark.asyncio
async def test_export_new_memory_to_claude_files(self, backend, claude_dir):
backend.add_memory(
"Project uses Python 3.12",
metadata={
"source_agent": "codex",
"sync_direction": "export", # Not from claude import
},
)
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
adapter = ClaudeCodeAdapter(claude_dir)
exported = await sync_export(backend, adapter, "tcms")
assert exported == 1
# Check file was created
md_files = list(claude_dir.glob("headroom_*.md"))
assert len(md_files) == 1
content = md_files[0].read_text()
assert "Python 3.12" in content
assert "headroom_id: mem_0001" in content
assert "source_agent: codex" in content
@pytest.mark.asyncio
async def test_export_skips_claude_originated(self, backend, claude_dir):
"""Don't re-export memories that were imported FROM claude (anti-echo)."""
backend.add_memory(
"From claude",
metadata={
"source_agent": "claude",
"sync_direction": "import",
},
)
backend.add_memory(
"From codex",
metadata={
"source_agent": "codex",
},
)
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
adapter = ClaudeCodeAdapter(claude_dir)
exported = await sync_export(backend, adapter, "tcms")
assert exported == 1 # Only "From codex"
@pytest.mark.asyncio
async def test_export_updates_memory_md_index(self, backend, claude_dir):
# Create an existing MEMORY.md
(claude_dir / "MEMORY.md").write_text("# Memory\n\n## User\n- Some existing entry\n")
backend.add_memory("New fact from codex", metadata={"source_agent": "codex"})
adapter = ClaudeCodeAdapter(claude_dir)
await sync_export(backend, adapter, "tcms")
memory_md = (claude_dir / "MEMORY.md").read_text()
assert "Headroom Shared Memory" in memory_md
assert "New fact from codex" in memory_md
assert "Some existing entry" in memory_md # Preserved
class TestBidirectionalSync:
"""Test full bidirectional sync."""
@pytest.fixture
def backend(self):
return FakeBackend()
@pytest.fixture
def claude_dir(self, tmp_path):
d = tmp_path / "memory"
d.mkdir()
return d
@pytest.fixture
def state_path(self, tmp_path):
return tmp_path / "sync_state.json"
def _write_claude_memory(self, memory_dir: Path, name: str, content: str) -> None:
slug = name.lower().replace(" ", "_")
fm = f"---\nname: {name}\ndescription: {content[:80]}\ntype: project\n---"
(memory_dir / f"{slug}.md").write_text(f"{fm}\n\n{content}\n")
@pytest.mark.asyncio
async def test_bidirectional_sync(self, backend, claude_dir, state_path):
# Claude has a memory file
self._write_claude_memory(claude_dir, "Convention", "Always use ruff for linting")
# DB has a memory from Codex
backend.add_memory("Secret name is TC", metadata={"source_agent": "codex"})
adapter = ClaudeCodeAdapter(claude_dir)
result = await sync(backend, adapter, "tcms", state_path=state_path, force=True)
assert result.imported == 1 # Claude file → DB
assert result.exported == 1 # Codex memory → Claude file
# Verify DB has both
mems = await backend.get_user_memories("tcms")
contents = {m.content for m in mems}
assert "Always use ruff for linting" in contents
assert "Secret name is TC" in contents
# Verify Claude dir has the exported file
all_files = list(claude_dir.glob("headroom_*.md"))
assert len(all_files) >= 1
exported_content = " ".join(f.read_text() for f in all_files)
assert "TC" in exported_content
@pytest.mark.asyncio
async def test_sync_idempotent(self, backend, claude_dir, state_path):
"""Running sync twice produces no duplicates."""
self._write_claude_memory(claude_dir, "Fact", "Python 3.12 is required")
backend.add_memory("Port 8787 is default", metadata={"source_agent": "codex"})
adapter = ClaudeCodeAdapter(claude_dir)
r1 = await sync(backend, adapter, "tcms", state_path=state_path, force=True)
assert r1.imported == 1
assert r1.exported == 1
r2 = await sync(backend, adapter, "tcms", state_path=state_path, force=True)
assert r2.imported == 0 # Already imported
assert r2.exported == 0 # Already exported
# No duplicates in DB
mems = await backend.get_user_memories("tcms")
assert len(mems) == 2
@pytest.mark.asyncio
async def test_fast_noop_when_unchanged(self, backend, claude_dir, state_path):
"""Second sync with no changes completes in < 10ms."""
self._write_claude_memory(claude_dir, "Fact", "Some fact")
adapter = ClaudeCodeAdapter(claude_dir)
# First sync (populates state)
await sync(backend, adapter, "tcms", state_path=state_path, force=True)
# Second sync (should be fast no-op)
start = time.monotonic()
r = await sync(backend, adapter, "tcms", state_path=state_path)
elapsed = (time.monotonic() - start) * 1000
assert r.imported == 0
assert r.exported == 0
assert elapsed < 50 # Generous threshold for CI
class TestLineageAndGovernance:
"""Test metadata tracking for audit and lineage."""
@pytest.fixture
def backend(self):
return FakeBackend()
@pytest.fixture
def claude_dir(self, tmp_path):
d = tmp_path / "memory"
d.mkdir()
return d
@pytest.mark.asyncio
async def test_lineage_tracks_source_agent(self, backend, claude_dir):
fm = "---\nname: test\ndescription: test\ntype: project\n---"
(claude_dir / "test.md").write_text(f"{fm}\n\nClaude discovered this\n")
adapter = ClaudeCodeAdapter(claude_dir)
await sync_import(backend, adapter, "tcms")
mems = await backend.get_user_memories("tcms")
assert mems[0].metadata["source_agent"] == "claude"
@pytest.mark.asyncio
async def test_exported_files_have_headroom_id(self, backend, claude_dir):
backend.add_memory("From codex", metadata={"source_agent": "codex"})
adapter = ClaudeCodeAdapter(claude_dir)
await sync_export(backend, adapter, "tcms")
md_files = list(claude_dir.glob("headroom_*.md"))
assert len(md_files) == 1
content = md_files[0].read_text()
assert "headroom_id:" in content
@pytest.mark.asyncio
async def test_sync_state_records_timestamps(self, backend, claude_dir, tmp_path):
state_path = tmp_path / "state.json"
fm = "---\nname: t\ndescription: t\ntype: project\n---"
(claude_dir / "t.md").write_text(f"{fm}\n\nFact\n")
adapter = ClaudeCodeAdapter(claude_dir)
await sync(backend, adapter, "tcms", state_path=state_path, force=True)
state = json.loads(state_path.read_text())
key = "claude:tcms"
assert key in state
assert "last_sync" in state[key]
assert "agent_fingerprint" in state[key]
assert "db_fingerprint" in state[key]
# ---------------------------------------------------------------------------
# Claude Code adapter tests
# ---------------------------------------------------------------------------
class TestClaudeCodeAdapter:
"""Test Claude Code adapter read/write."""
@pytest.fixture
def memory_dir(self, tmp_path):
d = tmp_path / "memory"
d.mkdir()
return d
def test_parse_frontmatter(self):
content = "---\nname: Test\ntype: project\n---\n\nBody content here."
fm, body = _parse_frontmatter(content)
assert fm["name"] == "Test"
assert fm["type"] == "project"
assert body == "Body content here."
def test_parse_frontmatter_no_frontmatter(self):
content = "Just plain content."
fm, body = _parse_frontmatter(content)
assert fm == {}
assert body == "Just plain content."
def test_encode_claude_project_path_windows_user_with_dot(self):
assert encode_claude_project_path(r"C:\Users\john.doe\work") == "-C-Users-john.doe-work"
def test_get_claude_memory_dir_uses_windows_safe_project_encoding(self):
memory_dir = get_claude_memory_dir(Path(r"C:\Users\john.doe\work"))
rendered = str(memory_dir)
assert "-C-Users-john.doe-work" in rendered
assert "john-doe" not in rendered
assert rendered.endswith("memory")
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
@pytest.mark.asyncio
async def test_read_memories_skips_memory_md(self, memory_dir):
(memory_dir / "MEMORY.md").write_text("# Index\n- entry")
(memory_dir / "fact.md").write_text(
"---\nname: Fact\ntype: project\n---\n\nImportant fact."
)
adapter = ClaudeCodeAdapter(memory_dir)
mems = await adapter.read_memories()
assert len(mems) == 1
assert mems[0].content == "Important fact."
assert mems[0].source_file == "fact.md"
@pytest.mark.asyncio
async def test_write_creates_valid_md(self, memory_dir):
adapter = ClaudeCodeAdapter(memory_dir)
written = await adapter.write_memories(
[
{
"content": "Project uses FastAPI",
"category": "architecture",
"headroom_id": "mem_001",
"source_agent": "codex",
"content_hash": "abc123",
}
]
)
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
assert written == 1
files = list(memory_dir.glob("headroom_*.md"))
assert len(files) == 1
content = files[0].read_text()
fm, body = _parse_frontmatter(content)
assert fm["type"] == "architecture"
assert fm["headroom_id"] == "mem_001"
assert fm["source_agent"] == "codex"
assert "FastAPI" in body
fix(memory/sync): don't clobber memories sharing a first line (#1976) ## Description `ClaudeCodeAdapter.write_memories` (`headroom/memory/sync_adapters/claude_code.py`) picks each memory's file name from the **first line of its content only**: ```python first_line = content.split("\n")[0][:60].strip() slug = _sanitize_for_filename(first_line) filename = f"headroom_{slug}.md" ``` So two *distinct* DB memories whose first lines slugify to the same value map to the same file. The existing "already on disk?" guard only skips when the on-disk content hash equals this memory's hash — for a genuine collision (same slug, different body) it falls through and `target.write_text(...)` overwrites the other memory. Data loss. It also never converges. The overwritten memory never lands on disk, so on the next `sync_export` the adapter reads back the agent's files, doesn't find that memory's hash in `agent_hashes`, and re-exports it — overwriting the other one this time. The pair ping-pongs on every sync, and each round appends a fresh line to `MEMORY.md`. This is realistic for headed/structured memories (e.g. several entries that begin `# Project conventions` or `The user prefers …`). Closes: no issue filed — found while auditing the memory sync adapters. ## Fix When the slug is already taken by a **different** memory (a distinct `headroom_id` in the existing file's frontmatter), disambiguate the file name with a short content-hash suffix so both survive. A matching `headroom_id` means it's an update of the same memory, so the plain slug file is rewritten as before — existing file names don't change, so there's no migration churn for the common (no-collision) case: ```python existing_id = existing_fm.get("headroom_id", "") if headroom_id and existing_id and existing_id != headroom_id: suffix = (content_hash or hashlib.sha256(content.encode()).hexdigest()[:16])[:8] filename = f"headroom_{slug}_{suffix}.md" ... ``` ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync_adapters/claude_code.py`: in `write_memories`, disambiguate the file name with a content-hash suffix when the slug already belongs to a different `headroom_id`; same-id updates still rewrite the slug file in place. - `tests/test_memory_sync.py`: add `test_write_distinct_memories_sharing_first_line_do_not_clobber` (two files survive) and `test_write_same_memory_updates_in_place` (no duplicate on update). ## Testing - [x] New regression tests added (`tests/test_memory_sync.py`) - [x] Linting passes (`ruff check`) and formatting is clean (`ruff format --check`) - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uv run ruff check headroom/memory/sync_adapters/claude_code.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.10, headroom from this branch. Importing `headroom` pulls in the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the write logic with a dependency-free script (replicating `_sanitize_for_filename` / `_parse_frontmatter` / the write loop against a real temp dir) and left the full pytest to CI. - Exact command / steps: wrote two memories that share the first line `# Project conventions` but differ in body (distinct `headroom_id`), through both the old and new logic, then wrote a same-id update. - Observed result: the old logic reports `written=2` but leaves **one** file (the first memory's body is gone); the new logic keeps both, and a same-id update rewrites in place instead of duplicating: ```text OLD: written=2 files=1 tabs=False fridays=True NEW: written=2 files=2 tabs=True fridays=True UPDATE: files=1 second=True MEMORY COLLISION FIX VERIFIED ``` - Not tested: a full `sync_export`/`sync_import` round-trip through the DB backend (needs the heavy stack). The fix is confined to the file-naming decision in `write_memories`, and the new tests drive that method directly. Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + a standalone logic check; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - No new dependencies; a small, migration-safe naming guard plus tests. - @JerrettDavis tagging you — this one is a quiet data-loss path in the Claude memory sync (a collision drops one memory and then thrashes on every sync), so it may be worth a look when you get a chance.
2026-07-10 21:17:31 +05:30
@pytest.mark.asyncio
async def test_write_distinct_memories_sharing_first_line_do_not_clobber(self, memory_dir):
"""Two different memories that share a first line must not overwrite one
another. The filename slug is derived from the first line only, so before
the fix the second write clobbered the first (data loss)."""
adapter = ClaudeCodeAdapter(memory_dir)
written = await adapter.write_memories(
[
{
"content": "# Project conventions\nUse tabs for indentation.",
"headroom_id": "mem_a",
"content_hash": "hash_a",
},
{
"content": "# Project conventions\nDeploy on Fridays only.",
"headroom_id": "mem_b",
"content_hash": "hash_b",
},
]
)
assert written == 2
files = sorted(memory_dir.glob("headroom_*.md"))
# Both memories must survive on disk (distinct files).
assert len(files) == 2
bodies = "\n".join(f.read_text() for f in files)
assert "tabs for indentation" in bodies
assert "Deploy on Fridays only" in bodies
@pytest.mark.asyncio
async def test_write_same_memory_updates_in_place(self, memory_dir):
"""An update to the *same* memory (matching headroom_id) rewrites the
original slug file rather than spawning a disambiguated duplicate."""
adapter = ClaudeCodeAdapter(memory_dir)
await adapter.write_memories(
[{"content": "# Note\nfirst version", "headroom_id": "mem_x", "content_hash": "h1"}]
)
await adapter.write_memories(
[{"content": "# Note\nsecond version", "headroom_id": "mem_x", "content_hash": "h2"}]
)
files = list(memory_dir.glob("headroom_*.md"))
assert len(files) == 1
assert "second version" in files[0].read_text()
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
def test_fingerprint_changes_on_modification(self, memory_dir):
(memory_dir / "test.md").write_text("content 1")
adapter = ClaudeCodeAdapter(memory_dir)
fp1 = adapter.fingerprint()
(memory_dir / "test.md").write_text("content 2")
fp2 = adapter.fingerprint()
assert fp1 != fp2
def test_fingerprint_stable_when_unchanged(self, memory_dir):
(memory_dir / "test.md").write_text("stable content")
adapter = ClaudeCodeAdapter(memory_dir)
assert adapter.fingerprint() == adapter.fingerprint()
def test_fingerprint_empty_dir(self, tmp_path):
empty = tmp_path / "empty"
empty.mkdir()
adapter = ClaudeCodeAdapter(empty)
assert adapter.fingerprint() == "empty"
# ---------------------------------------------------------------------------
# Codex adapter tests
# ---------------------------------------------------------------------------
class TestCodexAdapter:
"""Test Codex AGENTS.md adapter."""
@pytest.fixture
def agents_md(self, tmp_path):
return tmp_path / "AGENTS.md"
@pytest.mark.asyncio
async def test_read_from_agents_md(self, agents_md):
agents_md.write_text(
"# Instructions\n\n"
"<!-- headroom:memory:start -->\n"
"## Headroom Shared Memory\n\n"
"- Secret name is TC\n"
"- Uses Python 3.12\n"
"<!-- headroom:memory:end -->\n"
)
adapter = CodexAdapter(agents_md)
mems = await adapter.read_memories()
assert len(mems) == 2
assert mems[0].content == "Secret name is TC"
assert mems[1].content == "Uses Python 3.12"
@pytest.mark.asyncio
async def test_write_to_agents_md(self, agents_md):
agents_md.write_text("# Existing instructions\n")
adapter = CodexAdapter(agents_md)
written = await adapter.write_memories(
[
{"content": "Port 8787 is default"},
{"content": "Uses ruff for linting"},
]
)
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
assert written == 2
content = agents_md.read_text()
assert "headroom:memory:start" in content
assert "Port 8787 is default" in content
assert "Uses ruff for linting" in content
assert "Existing instructions" in content # Preserved
@pytest.mark.asyncio
fix(memory/sync): make Codex AGENTS.md adapter additive (stop wiping memories) (#1674) ## Description `sync_export` (in `headroom/memory/sync.py`) hands each adapter only the **delta** — the memories the agent doesn't already have. It reads the agent's current memories, builds `agent_hashes`, and only puts a memory in `to_export` if its hash isn't already there: ```python agent_hashes = {am.content_hash for am in await adapter.read_memories()} for mem in existing_memories: if content_hash in agent_hashes: continue # skip: agent already has it to_export.append(...) exported = await adapter.write_memories(to_export) # ← delta only ``` The `ClaudeCodeAdapter` is additive (a file per memory + index append), so a delta is correct for it. But `CodexAdapter.write_memories` rebuilt its **entire** `<!-- headroom:memory --> … <!-- /… -->` section from just the passed delta and spliced it back with `_MARKER_PATTERN.sub`. So every export **overwrote** the section with only the new items. Concrete thrash: - DB has A, B → first sync exports `[A, B]` → section = A, B ✅ - Add C → next sync's delta is `[C]` → section becomes **just C** (A, B erased) - Now the agent only has C → next sync's delta is `[A, B]` → section becomes **A, B** (C erased) … The file bounces between disjoint subsets and never holds the full set — silent memory loss on every sync. Closes: no issue filed — found while auditing the memory sync adapters. ## Fix Make `CodexAdapter.write_memories` additive, matching the adapter contract the ClaudeCode adapter already follows: read the facts already in the managed section, merge the incoming delta into them (dedup by rendered first-line), and write the union. Return the count actually added. The function-based `re.sub` is kept so literal backslashes / `\u` in a memory aren't treated as regex escapes. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync_adapters/codex_agent.py`: `write_memories` now merges the delta into the existing section instead of replacing the whole section. - `tests/test_memory_sync.py`: **two existing tests asserted the old replace-the-whole-section behavior — i.e. they codified this bug.** Updated them to the additive semantics (an existing managed fact is preserved) and added `test_write_accumulates_across_syncs` covering the delta-export-across-syncs scenario. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] New regression test added; two behavior-codifying tests corrected - [x] Linting passes (`ruff check`) and formatting is clean (`ruff format --check`) - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uv run ruff check headroom/memory/sync_adapters/codex_agent.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom from this branch. Importing `headroom` loads the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the merge logic with a dependency-free script (only stdlib) and left the full pytest to CI. - Exact command / steps: replicated `write_memories` (read existing section bullets → merge delta → splice) against real temp files, then ran the multi-sync scenario: export `[A, B]`, then export the delta `[C]`, then re-export an existing fact; plus a literal-backslash memory and a no-marker file. - Observed result: after the delta export of C, A and B are still present (no wipe); re-exporting an existing fact adds nothing; backslashes land literally; a file with no marker keeps its surrounding content: ```text OK: A,B preserved after delta-export of C (no wipe) OK: re-writing existing fact -> added 0, others intact OK: literal backslashes preserved OK: no-marker file -> section appended, existing preserved CODEX MERGE LOGIC VERIFIED ``` - Not tested: a full DB→adapter `sync_export` run end-to-end (needs a memory backend/embedder = the heavy stack); the delta contract is confirmed by reading `sync.py`, and the adapter merge is covered by the unit tests. Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + a standalone logic check; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The most reviewer-sensitive part is that I changed two existing tests. They were asserting `"old fact" not in content` after a write — i.e. they locked in the replace-the-whole-section behavior that causes the wipe. Given `sync_export` only ever passes the delta, that behavior is the bug; the updated tests assert the fact is preserved. Happy to discuss if you'd rather fix this on the `sync_export` side instead (e.g. pass the full set to replace-style adapters), but making the adapter additive matches the existing ClaudeCode adapter and keeps the contract uniform. - @JerrettDavis tagging you — flagging the test change up front so it's not a surprise in the diff. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-09 23:13:01 +05:30
async def test_write_merges_into_existing_section(self, agents_md):
"""Additive: an existing managed fact is preserved when a new one is
written. ``sync_export`` hands the adapter only the delta, so a
replace-the-whole-section write would erase prior memories."""
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
agents_md.write_text(
"# Instructions\n\n"
"<!-- headroom:memory:start -->\n"
fix(memory/sync): make Codex AGENTS.md adapter additive (stop wiping memories) (#1674) ## Description `sync_export` (in `headroom/memory/sync.py`) hands each adapter only the **delta** — the memories the agent doesn't already have. It reads the agent's current memories, builds `agent_hashes`, and only puts a memory in `to_export` if its hash isn't already there: ```python agent_hashes = {am.content_hash for am in await adapter.read_memories()} for mem in existing_memories: if content_hash in agent_hashes: continue # skip: agent already has it to_export.append(...) exported = await adapter.write_memories(to_export) # ← delta only ``` The `ClaudeCodeAdapter` is additive (a file per memory + index append), so a delta is correct for it. But `CodexAdapter.write_memories` rebuilt its **entire** `<!-- headroom:memory --> … <!-- /… -->` section from just the passed delta and spliced it back with `_MARKER_PATTERN.sub`. So every export **overwrote** the section with only the new items. Concrete thrash: - DB has A, B → first sync exports `[A, B]` → section = A, B ✅ - Add C → next sync's delta is `[C]` → section becomes **just C** (A, B erased) - Now the agent only has C → next sync's delta is `[A, B]` → section becomes **A, B** (C erased) … The file bounces between disjoint subsets and never holds the full set — silent memory loss on every sync. Closes: no issue filed — found while auditing the memory sync adapters. ## Fix Make `CodexAdapter.write_memories` additive, matching the adapter contract the ClaudeCode adapter already follows: read the facts already in the managed section, merge the incoming delta into them (dedup by rendered first-line), and write the union. Return the count actually added. The function-based `re.sub` is kept so literal backslashes / `\u` in a memory aren't treated as regex escapes. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync_adapters/codex_agent.py`: `write_memories` now merges the delta into the existing section instead of replacing the whole section. - `tests/test_memory_sync.py`: **two existing tests asserted the old replace-the-whole-section behavior — i.e. they codified this bug.** Updated them to the additive semantics (an existing managed fact is preserved) and added `test_write_accumulates_across_syncs` covering the delta-export-across-syncs scenario. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] New regression test added; two behavior-codifying tests corrected - [x] Linting passes (`ruff check`) and formatting is clean (`ruff format --check`) - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uv run ruff check headroom/memory/sync_adapters/codex_agent.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom from this branch. Importing `headroom` loads the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the merge logic with a dependency-free script (only stdlib) and left the full pytest to CI. - Exact command / steps: replicated `write_memories` (read existing section bullets → merge delta → splice) against real temp files, then ran the multi-sync scenario: export `[A, B]`, then export the delta `[C]`, then re-export an existing fact; plus a literal-backslash memory and a no-marker file. - Observed result: after the delta export of C, A and B are still present (no wipe); re-exporting an existing fact adds nothing; backslashes land literally; a file with no marker keeps its surrounding content: ```text OK: A,B preserved after delta-export of C (no wipe) OK: re-writing existing fact -> added 0, others intact OK: literal backslashes preserved OK: no-marker file -> section appended, existing preserved CODEX MERGE LOGIC VERIFIED ``` - Not tested: a full DB→adapter `sync_export` run end-to-end (needs a memory backend/embedder = the heavy stack); the delta contract is confirmed by reading `sync.py`, and the adapter merge is covered by the unit tests. Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + a standalone logic check; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The most reviewer-sensitive part is that I changed two existing tests. They were asserting `"old fact" not in content` after a write — i.e. they locked in the replace-the-whole-section behavior that causes the wipe. Given `sync_export` only ever passes the delta, that behavior is the bug; the updated tests assert the fact is preserved. Happy to discuss if you'd rather fix this on the `sync_export` side instead (e.g. pass the full set to replace-style adapters), but making the adapter additive matches the existing ClaudeCode adapter and keeps the contract uniform. - @JerrettDavis tagging you — flagging the test change up front so it's not a surprise in the diff. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-09 23:13:01 +05:30
"## Headroom Shared Memory\n\n- old fact\n"
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
"<!-- headroom:memory:end -->\n"
)
adapter = CodexAdapter(agents_md)
await adapter.write_memories([{"content": "new fact"}])
content = agents_md.read_text()
assert "new fact" in content
fix(memory/sync): make Codex AGENTS.md adapter additive (stop wiping memories) (#1674) ## Description `sync_export` (in `headroom/memory/sync.py`) hands each adapter only the **delta** — the memories the agent doesn't already have. It reads the agent's current memories, builds `agent_hashes`, and only puts a memory in `to_export` if its hash isn't already there: ```python agent_hashes = {am.content_hash for am in await adapter.read_memories()} for mem in existing_memories: if content_hash in agent_hashes: continue # skip: agent already has it to_export.append(...) exported = await adapter.write_memories(to_export) # ← delta only ``` The `ClaudeCodeAdapter` is additive (a file per memory + index append), so a delta is correct for it. But `CodexAdapter.write_memories` rebuilt its **entire** `<!-- headroom:memory --> … <!-- /… -->` section from just the passed delta and spliced it back with `_MARKER_PATTERN.sub`. So every export **overwrote** the section with only the new items. Concrete thrash: - DB has A, B → first sync exports `[A, B]` → section = A, B ✅ - Add C → next sync's delta is `[C]` → section becomes **just C** (A, B erased) - Now the agent only has C → next sync's delta is `[A, B]` → section becomes **A, B** (C erased) … The file bounces between disjoint subsets and never holds the full set — silent memory loss on every sync. Closes: no issue filed — found while auditing the memory sync adapters. ## Fix Make `CodexAdapter.write_memories` additive, matching the adapter contract the ClaudeCode adapter already follows: read the facts already in the managed section, merge the incoming delta into them (dedup by rendered first-line), and write the union. Return the count actually added. The function-based `re.sub` is kept so literal backslashes / `\u` in a memory aren't treated as regex escapes. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync_adapters/codex_agent.py`: `write_memories` now merges the delta into the existing section instead of replacing the whole section. - `tests/test_memory_sync.py`: **two existing tests asserted the old replace-the-whole-section behavior — i.e. they codified this bug.** Updated them to the additive semantics (an existing managed fact is preserved) and added `test_write_accumulates_across_syncs` covering the delta-export-across-syncs scenario. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] New regression test added; two behavior-codifying tests corrected - [x] Linting passes (`ruff check`) and formatting is clean (`ruff format --check`) - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uv run ruff check headroom/memory/sync_adapters/codex_agent.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom from this branch. Importing `headroom` loads the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the merge logic with a dependency-free script (only stdlib) and left the full pytest to CI. - Exact command / steps: replicated `write_memories` (read existing section bullets → merge delta → splice) against real temp files, then ran the multi-sync scenario: export `[A, B]`, then export the delta `[C]`, then re-export an existing fact; plus a literal-backslash memory and a no-marker file. - Observed result: after the delta export of C, A and B are still present (no wipe); re-exporting an existing fact adds nothing; backslashes land literally; a file with no marker keeps its surrounding content: ```text OK: A,B preserved after delta-export of C (no wipe) OK: re-writing existing fact -> added 0, others intact OK: literal backslashes preserved OK: no-marker file -> section appended, existing preserved CODEX MERGE LOGIC VERIFIED ``` - Not tested: a full DB→adapter `sync_export` run end-to-end (needs a memory backend/embedder = the heavy stack); the delta contract is confirmed by reading `sync.py`, and the adapter merge is covered by the unit tests. Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + a standalone logic check; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The most reviewer-sensitive part is that I changed two existing tests. They were asserting `"old fact" not in content` after a write — i.e. they locked in the replace-the-whole-section behavior that causes the wipe. Given `sync_export` only ever passes the delta, that behavior is the bug; the updated tests assert the fact is preserved. Happy to discuss if you'd rather fix this on the `sync_export` side instead (e.g. pass the full set to replace-style adapters), but making the adapter additive matches the existing ClaudeCode adapter and keeps the contract uniform. - @JerrettDavis tagging you — flagging the test change up front so it's not a surprise in the diff. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-09 23:13:01 +05:30
assert "old fact" in content # preserved, not clobbered
@pytest.mark.asyncio
fix(memory/sync): make Codex AGENTS.md adapter additive (stop wiping memories) (#1674) ## Description `sync_export` (in `headroom/memory/sync.py`) hands each adapter only the **delta** — the memories the agent doesn't already have. It reads the agent's current memories, builds `agent_hashes`, and only puts a memory in `to_export` if its hash isn't already there: ```python agent_hashes = {am.content_hash for am in await adapter.read_memories()} for mem in existing_memories: if content_hash in agent_hashes: continue # skip: agent already has it to_export.append(...) exported = await adapter.write_memories(to_export) # ← delta only ``` The `ClaudeCodeAdapter` is additive (a file per memory + index append), so a delta is correct for it. But `CodexAdapter.write_memories` rebuilt its **entire** `<!-- headroom:memory --> … <!-- /… -->` section from just the passed delta and spliced it back with `_MARKER_PATTERN.sub`. So every export **overwrote** the section with only the new items. Concrete thrash: - DB has A, B → first sync exports `[A, B]` → section = A, B ✅ - Add C → next sync's delta is `[C]` → section becomes **just C** (A, B erased) - Now the agent only has C → next sync's delta is `[A, B]` → section becomes **A, B** (C erased) … The file bounces between disjoint subsets and never holds the full set — silent memory loss on every sync. Closes: no issue filed — found while auditing the memory sync adapters. ## Fix Make `CodexAdapter.write_memories` additive, matching the adapter contract the ClaudeCode adapter already follows: read the facts already in the managed section, merge the incoming delta into them (dedup by rendered first-line), and write the union. Return the count actually added. The function-based `re.sub` is kept so literal backslashes / `\u` in a memory aren't treated as regex escapes. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync_adapters/codex_agent.py`: `write_memories` now merges the delta into the existing section instead of replacing the whole section. - `tests/test_memory_sync.py`: **two existing tests asserted the old replace-the-whole-section behavior — i.e. they codified this bug.** Updated them to the additive semantics (an existing managed fact is preserved) and added `test_write_accumulates_across_syncs` covering the delta-export-across-syncs scenario. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] New regression test added; two behavior-codifying tests corrected - [x] Linting passes (`ruff check`) and formatting is clean (`ruff format --check`) - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uv run ruff check headroom/memory/sync_adapters/codex_agent.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom from this branch. Importing `headroom` loads the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the merge logic with a dependency-free script (only stdlib) and left the full pytest to CI. - Exact command / steps: replicated `write_memories` (read existing section bullets → merge delta → splice) against real temp files, then ran the multi-sync scenario: export `[A, B]`, then export the delta `[C]`, then re-export an existing fact; plus a literal-backslash memory and a no-marker file. - Observed result: after the delta export of C, A and B are still present (no wipe); re-exporting an existing fact adds nothing; backslashes land literally; a file with no marker keeps its surrounding content: ```text OK: A,B preserved after delta-export of C (no wipe) OK: re-writing existing fact -> added 0, others intact OK: literal backslashes preserved OK: no-marker file -> section appended, existing preserved CODEX MERGE LOGIC VERIFIED ``` - Not tested: a full DB→adapter `sync_export` run end-to-end (needs a memory backend/embedder = the heavy stack); the delta contract is confirmed by reading `sync.py`, and the adapter merge is covered by the unit tests. Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + a standalone logic check; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The most reviewer-sensitive part is that I changed two existing tests. They were asserting `"old fact" not in content` after a write — i.e. they locked in the replace-the-whole-section behavior that causes the wipe. Given `sync_export` only ever passes the delta, that behavior is the bug; the updated tests assert the fact is preserved. Happy to discuss if you'd rather fix this on the `sync_export` side instead (e.g. pass the full set to replace-style adapters), but making the adapter additive matches the existing ClaudeCode adapter and keeps the contract uniform. - @JerrettDavis tagging you — flagging the test change up front so it's not a surprise in the diff. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-09 23:13:01 +05:30
async def test_write_preserves_existing_fact_with_literal_backslashes(self, agents_md):
agents_md.write_text(
"# Instructions\n\n"
"<!-- headroom:memory:start -->\n"
fix(memory/sync): make Codex AGENTS.md adapter additive (stop wiping memories) (#1674) ## Description `sync_export` (in `headroom/memory/sync.py`) hands each adapter only the **delta** — the memories the agent doesn't already have. It reads the agent's current memories, builds `agent_hashes`, and only puts a memory in `to_export` if its hash isn't already there: ```python agent_hashes = {am.content_hash for am in await adapter.read_memories()} for mem in existing_memories: if content_hash in agent_hashes: continue # skip: agent already has it to_export.append(...) exported = await adapter.write_memories(to_export) # ← delta only ``` The `ClaudeCodeAdapter` is additive (a file per memory + index append), so a delta is correct for it. But `CodexAdapter.write_memories` rebuilt its **entire** `<!-- headroom:memory --> … <!-- /… -->` section from just the passed delta and spliced it back with `_MARKER_PATTERN.sub`. So every export **overwrote** the section with only the new items. Concrete thrash: - DB has A, B → first sync exports `[A, B]` → section = A, B ✅ - Add C → next sync's delta is `[C]` → section becomes **just C** (A, B erased) - Now the agent only has C → next sync's delta is `[A, B]` → section becomes **A, B** (C erased) … The file bounces between disjoint subsets and never holds the full set — silent memory loss on every sync. Closes: no issue filed — found while auditing the memory sync adapters. ## Fix Make `CodexAdapter.write_memories` additive, matching the adapter contract the ClaudeCode adapter already follows: read the facts already in the managed section, merge the incoming delta into them (dedup by rendered first-line), and write the union. Return the count actually added. The function-based `re.sub` is kept so literal backslashes / `\u` in a memory aren't treated as regex escapes. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync_adapters/codex_agent.py`: `write_memories` now merges the delta into the existing section instead of replacing the whole section. - `tests/test_memory_sync.py`: **two existing tests asserted the old replace-the-whole-section behavior — i.e. they codified this bug.** Updated them to the additive semantics (an existing managed fact is preserved) and added `test_write_accumulates_across_syncs` covering the delta-export-across-syncs scenario. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] New regression test added; two behavior-codifying tests corrected - [x] Linting passes (`ruff check`) and formatting is clean (`ruff format --check`) - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uv run ruff check headroom/memory/sync_adapters/codex_agent.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom from this branch. Importing `headroom` loads the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the merge logic with a dependency-free script (only stdlib) and left the full pytest to CI. - Exact command / steps: replicated `write_memories` (read existing section bullets → merge delta → splice) against real temp files, then ran the multi-sync scenario: export `[A, B]`, then export the delta `[C]`, then re-export an existing fact; plus a literal-backslash memory and a no-marker file. - Observed result: after the delta export of C, A and B are still present (no wipe); re-exporting an existing fact adds nothing; backslashes land literally; a file with no marker keeps its surrounding content: ```text OK: A,B preserved after delta-export of C (no wipe) OK: re-writing existing fact -> added 0, others intact OK: literal backslashes preserved OK: no-marker file -> section appended, existing preserved CODEX MERGE LOGIC VERIFIED ``` - Not tested: a full DB→adapter `sync_export` run end-to-end (needs a memory backend/embedder = the heavy stack); the delta contract is confirmed by reading `sync.py`, and the adapter merge is covered by the unit tests. Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + a standalone logic check; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The most reviewer-sensitive part is that I changed two existing tests. They were asserting `"old fact" not in content` after a write — i.e. they locked in the replace-the-whole-section behavior that causes the wipe. Given `sync_export` only ever passes the delta, that behavior is the bug; the updated tests assert the fact is preserved. Happy to discuss if you'd rather fix this on the `sync_export` side instead (e.g. pass the full set to replace-style adapters), but making the adapter additive matches the existing ClaudeCode adapter and keeps the contract uniform. - @JerrettDavis tagging you — flagging the test change up front so it's not a surprise in the diff. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-09 23:13:01 +05:30
"## Headroom Shared Memory\n\n- old fact\n"
"<!-- headroom:memory:end -->\n"
)
adapter = CodexAdapter(agents_md)
await adapter.write_memories([{"content": r"Use C:\Users\john.doe\repo and literal \u"}])
content = agents_md.read_text()
fix(memory/sync): make Codex AGENTS.md adapter additive (stop wiping memories) (#1674) ## Description `sync_export` (in `headroom/memory/sync.py`) hands each adapter only the **delta** — the memories the agent doesn't already have. It reads the agent's current memories, builds `agent_hashes`, and only puts a memory in `to_export` if its hash isn't already there: ```python agent_hashes = {am.content_hash for am in await adapter.read_memories()} for mem in existing_memories: if content_hash in agent_hashes: continue # skip: agent already has it to_export.append(...) exported = await adapter.write_memories(to_export) # ← delta only ``` The `ClaudeCodeAdapter` is additive (a file per memory + index append), so a delta is correct for it. But `CodexAdapter.write_memories` rebuilt its **entire** `<!-- headroom:memory --> … <!-- /… -->` section from just the passed delta and spliced it back with `_MARKER_PATTERN.sub`. So every export **overwrote** the section with only the new items. Concrete thrash: - DB has A, B → first sync exports `[A, B]` → section = A, B ✅ - Add C → next sync's delta is `[C]` → section becomes **just C** (A, B erased) - Now the agent only has C → next sync's delta is `[A, B]` → section becomes **A, B** (C erased) … The file bounces between disjoint subsets and never holds the full set — silent memory loss on every sync. Closes: no issue filed — found while auditing the memory sync adapters. ## Fix Make `CodexAdapter.write_memories` additive, matching the adapter contract the ClaudeCode adapter already follows: read the facts already in the managed section, merge the incoming delta into them (dedup by rendered first-line), and write the union. Return the count actually added. The function-based `re.sub` is kept so literal backslashes / `\u` in a memory aren't treated as regex escapes. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync_adapters/codex_agent.py`: `write_memories` now merges the delta into the existing section instead of replacing the whole section. - `tests/test_memory_sync.py`: **two existing tests asserted the old replace-the-whole-section behavior — i.e. they codified this bug.** Updated them to the additive semantics (an existing managed fact is preserved) and added `test_write_accumulates_across_syncs` covering the delta-export-across-syncs scenario. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] New regression test added; two behavior-codifying tests corrected - [x] Linting passes (`ruff check`) and formatting is clean (`ruff format --check`) - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uv run ruff check headroom/memory/sync_adapters/codex_agent.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom from this branch. Importing `headroom` loads the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the merge logic with a dependency-free script (only stdlib) and left the full pytest to CI. - Exact command / steps: replicated `write_memories` (read existing section bullets → merge delta → splice) against real temp files, then ran the multi-sync scenario: export `[A, B]`, then export the delta `[C]`, then re-export an existing fact; plus a literal-backslash memory and a no-marker file. - Observed result: after the delta export of C, A and B are still present (no wipe); re-exporting an existing fact adds nothing; backslashes land literally; a file with no marker keeps its surrounding content: ```text OK: A,B preserved after delta-export of C (no wipe) OK: re-writing existing fact -> added 0, others intact OK: literal backslashes preserved OK: no-marker file -> section appended, existing preserved CODEX MERGE LOGIC VERIFIED ``` - Not tested: a full DB→adapter `sync_export` run end-to-end (needs a memory backend/embedder = the heavy stack); the delta contract is confirmed by reading `sync.py`, and the adapter merge is covered by the unit tests. Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + a standalone logic check; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The most reviewer-sensitive part is that I changed two existing tests. They were asserting `"old fact" not in content` after a write — i.e. they locked in the replace-the-whole-section behavior that causes the wipe. Given `sync_export` only ever passes the delta, that behavior is the bug; the updated tests assert the fact is preserved. Happy to discuss if you'd rather fix this on the `sync_export` side instead (e.g. pass the full set to replace-style adapters), but making the adapter additive matches the existing ClaudeCode adapter and keeps the contract uniform. - @JerrettDavis tagging you — flagging the test change up front so it's not a surprise in the diff. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-09 23:13:01 +05:30
# Backslashes / \u land literally (function replacement, not a template).
assert r"C:\Users\john.doe\repo" in content
assert r"literal \u" in content
fix(memory/sync): make Codex AGENTS.md adapter additive (stop wiping memories) (#1674) ## Description `sync_export` (in `headroom/memory/sync.py`) hands each adapter only the **delta** — the memories the agent doesn't already have. It reads the agent's current memories, builds `agent_hashes`, and only puts a memory in `to_export` if its hash isn't already there: ```python agent_hashes = {am.content_hash for am in await adapter.read_memories()} for mem in existing_memories: if content_hash in agent_hashes: continue # skip: agent already has it to_export.append(...) exported = await adapter.write_memories(to_export) # ← delta only ``` The `ClaudeCodeAdapter` is additive (a file per memory + index append), so a delta is correct for it. But `CodexAdapter.write_memories` rebuilt its **entire** `<!-- headroom:memory --> … <!-- /… -->` section from just the passed delta and spliced it back with `_MARKER_PATTERN.sub`. So every export **overwrote** the section with only the new items. Concrete thrash: - DB has A, B → first sync exports `[A, B]` → section = A, B ✅ - Add C → next sync's delta is `[C]` → section becomes **just C** (A, B erased) - Now the agent only has C → next sync's delta is `[A, B]` → section becomes **A, B** (C erased) … The file bounces between disjoint subsets and never holds the full set — silent memory loss on every sync. Closes: no issue filed — found while auditing the memory sync adapters. ## Fix Make `CodexAdapter.write_memories` additive, matching the adapter contract the ClaudeCode adapter already follows: read the facts already in the managed section, merge the incoming delta into them (dedup by rendered first-line), and write the union. Return the count actually added. The function-based `re.sub` is kept so literal backslashes / `\u` in a memory aren't treated as regex escapes. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync_adapters/codex_agent.py`: `write_memories` now merges the delta into the existing section instead of replacing the whole section. - `tests/test_memory_sync.py`: **two existing tests asserted the old replace-the-whole-section behavior — i.e. they codified this bug.** Updated them to the additive semantics (an existing managed fact is preserved) and added `test_write_accumulates_across_syncs` covering the delta-export-across-syncs scenario. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] New regression test added; two behavior-codifying tests corrected - [x] Linting passes (`ruff check`) and formatting is clean (`ruff format --check`) - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uv run ruff check headroom/memory/sync_adapters/codex_agent.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom from this branch. Importing `headroom` loads the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the merge logic with a dependency-free script (only stdlib) and left the full pytest to CI. - Exact command / steps: replicated `write_memories` (read existing section bullets → merge delta → splice) against real temp files, then ran the multi-sync scenario: export `[A, B]`, then export the delta `[C]`, then re-export an existing fact; plus a literal-backslash memory and a no-marker file. - Observed result: after the delta export of C, A and B are still present (no wipe); re-exporting an existing fact adds nothing; backslashes land literally; a file with no marker keeps its surrounding content: ```text OK: A,B preserved after delta-export of C (no wipe) OK: re-writing existing fact -> added 0, others intact OK: literal backslashes preserved OK: no-marker file -> section appended, existing preserved CODEX MERGE LOGIC VERIFIED ``` - Not tested: a full DB→adapter `sync_export` run end-to-end (needs a memory backend/embedder = the heavy stack); the delta contract is confirmed by reading `sync.py`, and the adapter merge is covered by the unit tests. Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + a standalone logic check; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The most reviewer-sensitive part is that I changed two existing tests. They were asserting `"old fact" not in content` after a write — i.e. they locked in the replace-the-whole-section behavior that causes the wipe. Given `sync_export` only ever passes the delta, that behavior is the bug; the updated tests assert the fact is preserved. Happy to discuss if you'd rather fix this on the `sync_export` side instead (e.g. pass the full set to replace-style adapters), but making the adapter additive matches the existing ClaudeCode adapter and keeps the contract uniform. - @JerrettDavis tagging you — flagging the test change up front so it's not a surprise in the diff. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-09 23:13:01 +05:30
assert "old fact" in content # preserved
@pytest.mark.asyncio
async def test_write_accumulates_across_syncs(self, agents_md):
"""Regression: exporting deltas across successive syncs must accumulate,
not thrash between disjoint subsets."""
adapter = CodexAdapter(agents_md)
await adapter.write_memories([{"content": "fact A"}, {"content": "fact B"}])
# Second sync only sees the new memory as a delta.
added = await adapter.write_memories([{"content": "fact C"}])
content = agents_md.read_text()
assert "fact A" in content
assert "fact B" in content
assert "fact C" in content
assert added == 1
# Re-writing an already-present fact adds nothing and keeps the rest.
again = await adapter.write_memories([{"content": "fact A"}])
assert again == 0
assert (await adapter.read_memories()).__len__() == 3
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
@pytest.mark.asyncio
async def test_read_empty_agents_md(self, agents_md):
agents_md.write_text("# No memory section\n")
adapter = CodexAdapter(agents_md)
mems = await adapter.read_memories()
assert mems == []
@pytest.mark.asyncio
async def test_read_nonexistent_file(self, tmp_path):
adapter = CodexAdapter(tmp_path / "nonexistent.md")
mems = await adapter.read_memories()
assert mems == []
# ---------------------------------------------------------------------------
# Cross-agent integration tests
# ---------------------------------------------------------------------------
class TestCrossAgentInterop:
"""Test that memories flow between agents via sync."""
@pytest.fixture
def backend(self):
return FakeBackend()
@pytest.fixture
def claude_dir(self, tmp_path):
d = tmp_path / "claude_memory"
d.mkdir()
return d
@pytest.fixture
def agents_md(self, tmp_path):
return tmp_path / "AGENTS.md"
@pytest.fixture
def state_path(self, tmp_path):
return tmp_path / "state.json"
@pytest.mark.asyncio
async def test_codex_saves_claude_finds(self, backend, claude_dir, state_path):
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
"""Memory saved via Codex MCP appears in Claude's files after sync."""
# Simulate Codex saving via MCP (directly to backend)
backend.add_memory(
"Secret name is TC",
metadata={"source_agent": "codex", "content_hash": "x"},
)
# Sync to Claude
adapter = ClaudeCodeAdapter(claude_dir)
result = await sync(backend, adapter, "tcms", state_path=state_path, force=True)
assert result.exported == 1
# Claude's memory dir should have the file
files = list(claude_dir.glob("headroom_*.md"))
assert len(files) == 1
assert "TC" in files[0].read_text()
@pytest.mark.asyncio
async def test_claude_saves_codex_finds(self, backend, claude_dir, agents_md, state_path):
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
"""Memory saved in Claude's files appears in Codex AGENTS.md after sync."""
# Claude has a memory
fm = "---\nname: Linting\ndescription: use ruff\ntype: project\n---"
(claude_dir / "linting.md").write_text(f"{fm}\n\nAlways use ruff for linting\n")
# Sync Claude → DB
claude_adapter = ClaudeCodeAdapter(claude_dir)
await sync(backend, claude_adapter, "tcms", state_path=state_path, force=True)
# Sync DB → Codex AGENTS.md
codex_adapter = CodexAdapter(agents_md)
result = await sync(backend, codex_adapter, "tcms", state_path=state_path, force=True)
assert result.exported >= 1
assert "ruff" in agents_md.read_text()
@pytest.mark.asyncio
async def test_full_round_trip(self, backend, claude_dir, agents_md, state_path):
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop
2026-04-13 23:38:44 -07:00
"""Full round trip: Claude → DB → Codex, Codex → DB → Claude."""
# Claude has a memory
fm = "---\nname: Framework\ntype: project\n---"
(claude_dir / "framework.md").write_text(f"{fm}\n\nUses FastAPI\n")
# Codex has a memory (in DB via MCP)
backend.add_memory("Port is 8787", metadata={"source_agent": "codex"})
# Sync both adapters
claude_adapter = ClaudeCodeAdapter(claude_dir)
codex_adapter = CodexAdapter(agents_md)
await sync(backend, claude_adapter, "tcms", state_path=state_path, force=True)
await sync(backend, codex_adapter, "tcms", state_path=state_path, force=True)
# DB has both memories
mems = await backend.get_user_memories("tcms")
contents = {m.content for m in mems}
assert "Uses FastAPI" in contents
assert "Port is 8787" in contents
# Claude files have Codex's memory
all_claude = " ".join(f.read_text() for f in claude_dir.glob("headroom_*.md"))
assert "8787" in all_claude
# AGENTS.md has both (from DB)
agents_content = agents_md.read_text()
assert "FastAPI" in agents_content or "8787" in agents_content
fix(memory): use ONNX embedder for `wrap --memory` sync (#1092) (#1262) ## Description `headroom wrap --memory` could never import memories: the startup sync subprocess (`python -m headroom.memory.sync`) and the in-process Codex memory import both built their backend with `LocalBackendConfig(db_path=...)`, which defaults `embedder_backend` to `"local"` — sentence-transformers + PyTorch (~2 GB). On the proxy extras that dependency is absent, so sync crashed with `ImportError: sentence-transformers is required for LocalEmbedder` while the proxy itself served memory fine via the torch-free ONNX backend. This routes both paths through a shared `_build_sync_backend` helper that uses `embedder_backend="onnx"`, matching the proxy MCP server (`headroom/memory/mcp_server.py`). Closes #1092 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync.py`: added `_build_sync_backend(db_path)` that constructs the backend with `embedder_backend="onnx"`; the sync CLI subprocess now uses it. - `headroom/cli/wrap.py`: the in-process Claude→DB memory import (Codex wrap path) now uses the same helper instead of the LOCAL-defaulting `LocalBackendConfig`. - `tests/test_memory_sync.py`: added `test_sync_backend_uses_onnx_embedder` regression test. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality ### Test Output ```text $ python -m pytest tests/test_memory_sync.py -q 31 passed $ python -m ruff check headroom/memory/sync.py headroom/cli/wrap.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, headroom on branch fix/1092-memory-sync-onnx-embedder - Exact command / steps: Ran the memory-sync suite + ruff, and an import smoke that builds the sync backend: `python -c "from headroom.memory.sync import _build_sync_backend; print(_build_sync_backend('x.db')._config.embedder_backend)"`. - Observed result: 31 tests pass (incl. the new regression test), ruff clean, and the smoke prints `onnx` — the sync backend no longer defaults to the sentence-transformers embedder. - Not tested: Did not run a full live `headroom wrap claude --memory` end to end (needs the ONNX model download + Claude memory files); the same-model (all-MiniLM-L6-v2, 384-dim) ONNX backend the proxy already uses keeps vectors DB-compatible, so no migration is involved. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 05:08:12 +02:00
def test_sync_backend_uses_onnx_embedder(tmp_path):
"""#1092: the sync subprocess must pick the torch-free ONNX embedder.
Defaulting to the LOCAL (sentence-transformers) embedder makes
`wrap --memory` crash with an ImportError on the proxy extras. The backend
must match the proxy MCP server, which uses ONNX.
"""
backend = _build_sync_backend(str(tmp_path / "memory.db"))
assert backend._config.embedder_backend == "onnx"