mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
6 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Abhinav Kumar Singh
|
4bd8ecd1e3
|
fix(memory): close MCP backend on shutdown
## Description Closes the initialized LocalBackend and cancels in-flight initialization whenever the memory MCP stdio transport exits. Fixes #2898 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added an explicit server cleanup callback that cancels and awaits pending backend initialization. - Closes an initialized backend exactly once and clears the backend/task references. - Runs cleanup in `_run()` through a `finally` block after the stdio transport exits, including transport errors. - Added regression coverage for initialized cleanup, pending initialization cancellation, idempotence, and `_run()` shutdown behavior. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text python -m pytest -q tests/test_memory/test_mcp_server.py 15 passed, 20 warnings ruff check . All checks passed! ruff format --check . 1382 files already formatted python -m mypy headroom Success: no issues found in 515 source files python -m pytest -q Collected 8881 items / 174 errors / 18 skipped. Interrupted during collection because this Windows environment lacks the compiled headroom._core extension. ``` ## Real Behavior Proof - Environment: Windows, Python 3.12, async MCP server lifecycle test with the real `create_memory_server()` closure and an embedded server transport stub. - Exact command / steps: Ran `python -m pytest -q tests/test_memory/test_mcp_server.py`; the regression tests initialized a backend through the server's registered tool lifecycle, returned the stdio transport, and invoked the cleanup callback from `_run()`'s `finally` path. - Observed result: 15 tests passed. Initialized backends were closed once, pending initialization was cancelled and awaited, and transport exit invoked cleanup even when the server run returned. - Who maintains it: Headroom Labs maintains this active upstream repository and memory MCP server. - Install surface: No dependencies or install behavior changed. The fix uses existing asyncio lifecycle handling and `LocalBackend.close()`; no native code or runtime network access is introduced. - Not tested: The complete repository suite could not run past collection because this Windows environment lacks the compiled `headroom._core` extension. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes (full suite blocked by missing native extension; targeted tests pass) - [x] I did not edit `CHANGELOG.md` - it is generated by release-please from my Conventional Commit PR title. ## Screenshots (if applicable) Not applicable. ## Additional Notes Cleanup is attached to each created memory MCP server and is idempotent, so embedded callers can invoke the same lifecycle callback safely if needed. |
||
|
Gautam Sharma
|
0924755591
|
fix(memory): serialize MCP backend initialization (#2309)
## Description
The Memory MCP server previously assigned its backend before
asynchronous embedder and vector-index warm-up completed. A tool call
arriving
during the handshake could therefore receive a partially initialized
backend.
Backend initialization is now atomic and shared between concurrent
callers. The backend is published only after warm-up succeeds. Failed
candidates are closed and discarded so later calls can retry with a
fresh backend.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation-only change
- [ ] Refactoring
## Changes Made
- Keep the initializing backend local until warm-up completes
successfully.
- Share one initialization task between handshake and concurrent tool
calls.
- Await the shared task before exposing the backend to tool handlers.
- Shield shared initialization from cancellation by an individual tool
caller.
- Close failed or cancelled backend candidates.
- Clear failed initialization state so subsequent calls can retry.
- Retrieve and log background initialization failures.
- Add regression tests for handshake races, failure recovery, and
concurrent initialization.
- Add an Unreleased changelog entry.
## Testing
- [x] Added regression tests
- [x] Focused test suite passes
- [x] Ruff checks pass
- [x] Mypy passes
- [x] Changed files pass formatting checks
- [ ] Entire repository test suite passes without baseline failures
Commands and results:
- `uv run --extra dev --frozen pytest
tests/test_memory/test_mcp_server.py -q`
- `12 passed`
- `uv run --extra dev --frozen ruff check .`
- Passed
- `uv run --extra dev --frozen ruff format --check
headroom/memory/mcp_server.py tests/test_memory/test_mcp_server.py`
- Passed
- `uv run --extra dev --frozen mypy headroom --ignore-missing-imports`
- Success across 504 source files
- `uv run --extra dev --frozen pytest -q`
- `9363 passed, 565 skipped, 4 failed`
- The four failures are existing, unrelated failures outside the changed
code:
- `test_l2_appends_transform_label`
- `test_recovery_records_sockets_and_secures_both_backups`
- `test_dashboard_uses_cached_stats_and_lazy_history_feed_polling`
- `test_smart_crusher_log_fallback_runs_for_valid_json`
Repository-wide `ruff format --check .` also identifies pre-existing
formatting drift in the untouched
`headroom/proxy/handlers/anthropic.py`.
## Real Behavior Proof
The regression tests exercise the affected lifecycle directly:
1. Start backend initialization through the MCP handshake.
2. Suspend warm-up before it completes.
3. Issue a memory tool call and verify its handler is not invoked.
4. Release warm-up and verify the tool receives the initialized backend.
5. Force background initialization to fail and verify the candidate is
closed.
6. Issue another tool call and verify initialization retries with a
fresh backend.
7. Start two tool calls concurrently and verify only one backend is
constructed.
Observed behavior:
- Tool calls remain pending while handshake warm-up is incomplete.
- A partially initialized backend never reaches a tool handler.
- Failed candidates are closed and discarded.
- A later tool call successfully retries initialization.
- Concurrent calls share one initialization task and backend.
Environment: macOS arm64, CPython 3.12.13.
Not tested: a live stdio MCP client using the real ONNX model and
database. The affected initialization lifecycle is covered with
deterministic asynchronous regression tests.
## Review Readiness
- [x] I have performed a self-review before requesting human review.
- [x] This PR is ready for human review.
## Checklist
- [x] The implementation follows the repository’s existing style and
error-handling conventions.
- [x] Tests cover the reported race, concurrent initialization, and
failure recovery.
- [x] Failed initialization does not leave a partially published
backend.
- [x] Failed backend candidates are closed before retry.
- [x] No unrelated files or formatting changes are included.
- [x] No temporary logging, debug code, or commented-out code remains.
- [x] Public behavior changes are documented in the changelog.
- [x] The branch has been rebased from the intended base and is ready
for review.
## Additional Notes
The four full-suite failures listed above occur outside the changed
Memory MCP code and are unrelated to this PR. All tests covering the
modified initialization lifecycle pass.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Chester
|
6d897e8eaa
|
fix(memory): require explicit updates for supersession (#2188)
## Description The standalone Memory MCP `memory_save` handler currently treats vector similarity as update identity. A score of `0.70` can therefore supersede a valid but distinct memory that merely shares domain vocabulary. This change makes `memory_save` append-only. Supersession remains available through explicit update paths that receive an existing memory ID. Closes #2187. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that causes existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Remove vector-similarity-based auto-supersession from the standalone MCP `memory_save` handler. - Clarify in the tool description that corrections require an explicit update path with the existing memory ID. - Add a regression test proving that a high-scoring but distinct memory is neither searched for replacement nor updated. - Preserve the existing save result summary shape for compatibility. ## Testing - [x] Focused unit tests pass - [x] Linting passes (`ruff check` on changed files) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual focused test execution performed ### Test Output ```text uv run --with pytest --with numpy pytest tests/test_memory/test_mcp_server.py -q 9 passed, 21 warnings in 0.70s uvx ruff check headroom/memory/mcp_server.py tests/test_memory/test_mcp_server.py All checks passed! uvx ruff format --check headroom/memory/mcp_server.py tests/test_memory/test_mcp_server.py 2 files already formatted ``` The warnings are pre-existing pytest configuration and `datetime.utcnow()` deprecation warnings in the test environment. ## Real Behavior Proof - Environment: Python 3.13 with the MCP module stub and an async recording backend. - Exact command / steps: run `tests/test_memory/test_mcp_server.py`; the new regression supplies a search result with similarity `0.91`, then saves a distinct fact. - Observed result: `search_memories` and `update_memory` are not called; `save_memory` is called once with the new fact and requested importance. - Not tested: live embedding backends or migration of supersession chains created by earlier versions. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my code - [x] I have commented the non-obvious identity boundary - [ ] Documentation changes are limited to the MCP tool description - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [ ] New and existing unit tests pass locally; the focused MCP suite passes and full CI is pending - [ ] CHANGELOG update is not included because release notes are generated from conventional commits ## Screenshots (if applicable) Not applicable. ## Additional Notes This patch intentionally does not infer replacement identity from category, entity references, or a higher vector threshold: none of those alone proves that two statements are versions of the same fact. Exposing an explicit update tool from the standalone MCP server can be considered separately without retaining the unsafe automatic behavior. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Chester
|
d0ecc9a556
|
fix(memory): track MCP retrieval access (#2065)
## Description Track successful native MCP `memory_search` retrievals in persistent memory metadata. Returned memories now increment `access_count` and update `last_accessed`, so MCP usage contributes to memory budget and retention signals. Closes #2061 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add an atomic, deduplicated `MemoryStore.record_access` operation. - Expose access recording through `HierarchicalMemory` and `LocalBackend`, invalidating stale cache entries. - Record only the final active memories actually returned by MCP search. - Fail open if usage metadata cannot be written. - Add SQLite and MCP regression coverage. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text pytest tests/test_memory --ignore=tests/test_memory/test_learn_flag.py -q 368 passed, 142 skipped, 158 warnings in 3.28s pytest tests/test_memory/test_hierarchical.py tests/test_memory/test_mcp_server.py tests/test_memory/test_factory.py -q 40 passed, 53 skipped, 158 warnings in 0.75s ``` ## Real Behavior Proof - Environment: macOS, Python 3.13, SQLite memory store. - Exact command / steps: save two memories; call `record_access` with duplicate IDs plus a missing ID; read both rows; call it again for one row. - Observed result: each existing memory increments once per call, duplicates do not double-count, missing IDs are ignored, and `last_accessed` advances to the supplied timestamp. - Not tested: the full repository suite and `tests/test_memory/test_learn_flag.py`; the source checkout does not include the compiled `headroom._core` Rust extension. Ruff and mypy were not available in the local development environment; CI remains authoritative for those checks. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes Documentation and changelog changes are not included because this is an internal retrieval-metadata correction with no user-facing configuration change. Access tracking is intentionally fail-open so a metadata write failure cannot suppress a valid memory search result. --------- Co-authored-by: xuyidiao <xuyidiao@bytedance.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Rod Boev
|
ad7993bf15
|
fix(codex): stop pinning Codex memory MCP to one project db (#1269)
## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic. |
||
|
Kayzo
|
f5cea7c51e |
fix(memory): batch onnx embeddings and sqlite-vec ops
Make the ONNX + sqlite-vec memory path truly batched. Batch ONNX embed_batch calls, batch sqlite-vec index/remove work under a single cached connection, and update MCP warm-up to use batch embed/save/index flows. Add focused regression tests for ONNX batching, sqlite-vec single-connection batch behavior, and MCP warm-up batching. Skip the MCP-specific test when optional MCP dependencies are not installed. Refs #240 |