mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
6 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Abhay Singh
|
5e14b8c0f2
|
fix(memory/sync): don't clobber memories sharing a first line (#1976)
## Description
`ClaudeCodeAdapter.write_memories`
(`headroom/memory/sync_adapters/claude_code.py`) picks
each memory's file name from the **first line of its content only**:
```python
first_line = content.split("\n")[0][:60].strip()
slug = _sanitize_for_filename(first_line)
filename = f"headroom_{slug}.md"
```
So two *distinct* DB memories whose first lines slugify to the same
value map to the same
file. The existing "already on disk?" guard only skips when the on-disk
content hash equals
this memory's hash — for a genuine collision (same slug, different body)
it falls through and
`target.write_text(...)` overwrites the other memory. Data loss.
It also never converges. The overwritten memory never lands on disk, so
on the next
`sync_export` the adapter reads back the agent's files, doesn't find
that memory's hash in
`agent_hashes`, and re-exports it — overwriting the other one this time.
The pair ping-pongs
on every sync, and each round appends a fresh line to `MEMORY.md`.
This is realistic for headed/structured memories (e.g. several entries
that begin
`# Project conventions` or `The user prefers …`).
Closes: no issue filed — found while auditing the memory sync adapters.
## Fix
When the slug is already taken by a **different** memory (a distinct
`headroom_id` in the
existing file's frontmatter), disambiguate the file name with a short
content-hash suffix so
both survive. A matching `headroom_id` means it's an update of the same
memory, so the plain
slug file is rewritten as before — existing file names don't change, so
there's no migration
churn for the common (no-collision) case:
```python
existing_id = existing_fm.get("headroom_id", "")
if headroom_id and existing_id and existing_id != headroom_id:
suffix = (content_hash or hashlib.sha256(content.encode()).hexdigest()[:16])[:8]
filename = f"headroom_{slug}_{suffix}.md"
...
```
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/memory/sync_adapters/claude_code.py`: in `write_memories`,
disambiguate the file name with a content-hash suffix when the slug
already belongs to a different `headroom_id`; same-id updates still
rewrite the slug file in place.
- `tests/test_memory_sync.py`: add
`test_write_distinct_memories_sharing_first_line_do_not_clobber` (two
files survive) and `test_write_same_memory_updates_in_place` (no
duplicate on update).
## Testing
- [x] New regression tests added (`tests/test_memory_sync.py`)
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`)
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uv run ruff check headroom/memory/sync_adapters/claude_code.py tests/test_memory_sync.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the write logic with
a dependency-free script (replicating `_sanitize_for_filename` /
`_parse_frontmatter` / the write loop against a real temp dir) and left
the full pytest to CI.
- Exact command / steps: wrote two memories that share the first line `#
Project conventions` but differ in body (distinct `headroom_id`),
through both the old and new logic, then wrote a same-id update.
- Observed result: the old logic reports `written=2` but leaves **one**
file (the first memory's body is gone); the new logic keeps both, and a
same-id update rewrites in place instead of duplicating:
```text
OLD: written=2 files=1 tabs=False fridays=True
NEW: written=2 files=2 tabs=True fridays=True
UPDATE: files=1 second=True
MEMORY COLLISION FIX VERIFIED
```
- Not tested: a full `sync_export`/`sync_import` round-trip through the
DB backend (needs the heavy stack). The fix is confined to the
file-naming decision in `write_memories`, and the new tests drive that
method directly. Full local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- No new dependencies; a small, migration-safe naming guard plus tests.
- @JerrettDavis tagging you — this one is a quiet data-loss path in the
Claude memory sync (a collision drops one memory and then thrashes on
every sync), so it may be worth a look when you get a chance.
|
||
|
Abhay Singh
|
7fd0c42ced
|
fix(memory/sync): make Codex AGENTS.md adapter additive (stop wiping memories) (#1674)
## Description
`sync_export` (in `headroom/memory/sync.py`) hands each adapter only the
**delta** — the memories the agent doesn't already have. It reads the
agent's
current memories, builds `agent_hashes`, and only puts a memory in
`to_export`
if its hash isn't already there:
```python
agent_hashes = {am.content_hash for am in await adapter.read_memories()}
for mem in existing_memories:
if content_hash in agent_hashes:
continue # skip: agent already has it
to_export.append(...)
exported = await adapter.write_memories(to_export) # ← delta only
```
The `ClaudeCodeAdapter` is additive (a file per memory + index append),
so a
delta is correct for it. But `CodexAdapter.write_memories` rebuilt its
**entire**
`<!-- headroom:memory --> … <!-- /… -->` section from just the passed
delta and
spliced it back with `_MARKER_PATTERN.sub`. So every export
**overwrote** the
section with only the new items.
Concrete thrash:
- DB has A, B → first sync exports `[A, B]` → section = A, B ✅
- Add C → next sync's delta is `[C]` → section becomes **just C** (A, B
erased)
- Now the agent only has C → next sync's delta is `[A, B]` → section
becomes
**A, B** (C erased) …
The file bounces between disjoint subsets and never holds the full set —
silent
memory loss on every sync.
Closes: no issue filed — found while auditing the memory sync adapters.
## Fix
Make `CodexAdapter.write_memories` additive, matching the adapter
contract the
ClaudeCode adapter already follows: read the facts already in the
managed
section, merge the incoming delta into them (dedup by rendered
first-line), and
write the union. Return the count actually added. The function-based
`re.sub`
is kept so literal backslashes / `\u` in a memory aren't treated as
regex
escapes.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/memory/sync_adapters/codex_agent.py`: `write_memories` now
merges the delta into the existing section instead of replacing the
whole section.
- `tests/test_memory_sync.py`: **two existing tests asserted the old
replace-the-whole-section behavior — i.e. they codified this bug.**
Updated them to the additive semantics (an existing managed fact is
preserved) and added `test_write_accumulates_across_syncs` covering the
delta-export-across-syncs scenario.
- `CHANGELOG.md`: Bug Fixes entry under Unreleased.
## Testing
- [x] New regression test added; two behavior-codifying tests corrected
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`)
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uv run ruff check headroom/memory/sync_adapters/codex_agent.py tests/test_memory_sync.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, headroom from this branch.
Importing `headroom` loads the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the merge logic with
a dependency-free script (only stdlib) and left the full pytest to CI.
- Exact command / steps: replicated `write_memories` (read existing
section bullets → merge delta → splice) against real temp files, then
ran the multi-sync scenario: export `[A, B]`, then export the delta
`[C]`, then re-export an existing fact; plus a literal-backslash memory
and a no-marker file.
- Observed result: after the delta export of C, A and B are still
present (no wipe); re-exporting an existing fact adds nothing;
backslashes land literally; a file with no marker keeps its surrounding
content:
```text
OK: A,B preserved after delta-export of C (no wipe)
OK: re-writing existing fact -> added 0, others intact
OK: literal backslashes preserved
OK: no-marker file -> section appended, existing preserved
CODEX MERGE LOGIC VERIFIED
```
- Not tested: a full DB→adapter `sync_export` run end-to-end (needs a
memory backend/embedder = the heavy stack); the delta contract is
confirmed by reading `sync.py`, and the adapter merge is covered by the
unit tests. Full local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- The most reviewer-sensitive part is that I changed two existing tests.
They were asserting `"old fact" not in content` after a write — i.e.
they locked in the replace-the-whole-section behavior that causes the
wipe. Given `sync_export` only ever passes the delta, that behavior is
the bug; the updated tests assert the fact is preserved. Happy to
discuss if you'd rather fix this on the `sync_export` side instead (e.g.
pass the full set to replace-style adapters), but making the adapter
additive matches the existing ClaudeCode adapter and keeps the contract
uniform.
- @JerrettDavis tagging you — flagging the test change up front so it's
not a surprise in the diff.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Parideboy
|
4f9fedaa7a
|
fix(memory): use ONNX embedder for wrap --memory sync (#1092) (#1262)
## Description `headroom wrap --memory` could never import memories: the startup sync subprocess (`python -m headroom.memory.sync`) and the in-process Codex memory import both built their backend with `LocalBackendConfig(db_path=...)`, which defaults `embedder_backend` to `"local"` — sentence-transformers + PyTorch (~2 GB). On the proxy extras that dependency is absent, so sync crashed with `ImportError: sentence-transformers is required for LocalEmbedder` while the proxy itself served memory fine via the torch-free ONNX backend. This routes both paths through a shared `_build_sync_backend` helper that uses `embedder_backend="onnx"`, matching the proxy MCP server (`headroom/memory/mcp_server.py`). Closes #1092 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/memory/sync.py`: added `_build_sync_backend(db_path)` that constructs the backend with `embedder_backend="onnx"`; the sync CLI subprocess now uses it. - `headroom/cli/wrap.py`: the in-process Claude→DB memory import (Codex wrap path) now uses the same helper instead of the LOCAL-defaulting `LocalBackendConfig`. - `tests/test_memory_sync.py`: added `test_sync_backend_uses_onnx_embedder` regression test. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality ### Test Output ```text $ python -m pytest tests/test_memory_sync.py -q 31 passed $ python -m ruff check headroom/memory/sync.py headroom/cli/wrap.py tests/test_memory_sync.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, headroom on branch fix/1092-memory-sync-onnx-embedder - Exact command / steps: Ran the memory-sync suite + ruff, and an import smoke that builds the sync backend: `python -c "from headroom.memory.sync import _build_sync_backend; print(_build_sync_backend('x.db')._config.embedder_backend)"`. - Observed result: 31 tests pass (incl. the new regression test), ruff clean, and the smoke prints `onnx` — the sync backend no longer defaults to the sentence-transformers embedder. - Not tested: Did not run a full live `headroom wrap claude --memory` end to end (needs the ONNX model download + Claude memory files); the same-model (all-MiniLM-L6-v2, 384-dim) ONNX backend the proxy already uses keeps vectors DB-compatible, so no migration is involved. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Tejas Chopra
|
5ceca13c65 | fix: harden learn path handling across platforms | ||
|
chopratejas
|
89061fd430 |
Fix CI lint errors and test failures
- test_memory_sync.py: remove unused imports (asyncio, MagicMock,
AgentMemory, AgentMemoryAdapter, SyncResult), fix import sorting
- test_ws_memory_relay.py: remove unused pytest import and unused
output_index variable, fix import sorting
- test_wrap_copilot.py: provide dummy API keys in test env — the
BYOK validation added in
|
||
|
Tejas Chopra
|
82301bbf76 |
Add cross-agent memory interoperability via MCP, sync engine, and atomic facts
Memory saved in one agent (Codex, Claude Code, Aider) is now accessible from any other agent through a shared headroom DB. Three integration layers: MCP Server (headroom.memory.mcp_server): - stdio MCP server that Codex/Claude discover natively via config.toml - memory_search with supersession filtering (only active memories returned) - memory_save accepts atomic facts array — each fact stored/indexed individually - Auto-supersession: new facts that match existing ones (≥0.70 similarity) retire the old entry via the supersedes/superseded_by lineage chain - ONNX embedder pre-loaded at startup (no cold-start on first query) - HuggingFace offline mode eliminates network latency on startup Sync Engine (headroom.memory.sync): - Bidirectional sync: DB ↔ agent-native memory files - Pluggable adapters: ClaudeCodeAdapter (frontmatter .md files + MEMORY.md index), CodexAdapter (AGENTS.md sections) - Fast no-op: fingerprint comparison skips sync when nothing changed (<5ms) - Content-hash dedup prevents duplicate memories across agents - Lineage metadata: source_agent, source_file, content_hash, synced_at - Anti-echo: memories imported from an agent are not re-exported to that agent - CLI entry point: python -m headroom.memory.sync --agent claude|codex Wrap CLI integration: - `wrap codex --memory`: registers MCP server + AGENTS.md guidance + syncs Claude memories into DB for MCP search - `wrap claude --memory`: bidirectional sync at startup (DB ↔ Claude files) - MCP config re-injected after provider config to survive file rewrite - Cross-platform: Windows path handling in TOML configs and path sanitization Proxy improvements: - Responses API: tool format conversion (Chat Completions → Responses API) - Responses API: memory tool calls handled with proper continuation - WebSocket: buffer-then-decide relay suppresses memory tool events from Codex, executes them transparently, relays only the final answer - memory_handler: supports Responses API function_call format (call_id, top-level arguments, output[] extraction) - HNSW vector index now persists to disk via auto_save + save_path Tests: 37 new tests covering WS relay event suppression, sync import/export, bidirectional sync, idempotency, fast no-op, lineage, cross-agent interop |