Hardens client-selected upstreams, memory identity resolution, downloaded binary integrity, telemetry import, Docker defaults, Neo4j credentials, and archive extraction. Refreshes the branch against current main and preserves newer same-origin and loopback protections.
## Description
Prevent memory_save from automatically deleting semantically similar but
distinct memories. The previous fire-and-forget deduplication path
deleted existing memories at cosine similarity scores of 0.92 or higher
after the save had already returned success. Similarity remains
available as a consolidation hint, while supersession now requires an
explicit memory_update or memory_delete operation.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Removed the automatic background deletion scheduled by memory_save.
- Removed the automatic-dedup threshold and background coroutine that
were no longer needed.
- Preserved the existing similarity search and consolidation hint.
- Kept explicit memory_update and memory_delete behavior unchanged.
- Added a regression test proving that distinct memories survive even at
0.99 simulated similarity.
- Added an Unreleased changelog entry.
## Testing
- [x] Unit tests pass (pytest)
- [x] Linting passes (ruff check .)
- [x] Type checking passes (mypy headroom)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
$ uv run --extra dev --frozen pytest
tests/test_memory_handler_native_ops.py
33 passed
$ uv run --extra dev --frozen ruff check .
All checks passed!
$ uv run --extra dev --frozen ruff format --check
headroom/proxy/memory_handler.py tests/test_memory_handler_native_ops.py
2 files already formatted
$ uv run --extra dev --frozen mypy headroom --ignore-missing-imports
Success: no issues found in 504 source files
$ uv run --extra dev --frozen pytest
9361 passed, 565 skipped, 4 failed
The four full-suite failures are unrelated to this diff: the Anthropic
compaction test passed in isolation; the Codex recovery test exceeded
the macOS AF_UNIX path limit; the dashboard test expects text absent
from the existing implementation; and the content-router test expects a
fallback absent from the existing strategy chain.
The repository-wide format check also flags pre-existing formatting in
the untouched headroom/proxy/handlers/anthropic.py.
## Real Behavior Proof
- Environment: macOS on Apple Silicon, CPython 3.12.13, real
LocalBackend, temporary SQLite database, and the local
sentence-transformers
embedding backend; no external provider or model API.
- Exact command / steps: Ran uv run --extra dev --frozen python with a
temporary database, saved User's primary backend framework at work is
FastAPI., queried its similarity to User's primary backend framework at
home is FastAPI., saved the second fact through
MemoryHandler._execute_save, and listed the user's memories.
- Observed result: The real embedding similarity was 0.9387, above the
former 0.92 deletion threshold. The second save returned saved,
included the consolidation hint, retained the original memory, and left
both distinct facts in the database (memory_count: 2).
- Not tested: Live OpenAI or Anthropic provider calls, a deployed proxy
or MCP client session, and Qdrant or Neo4j memory backends. These
paths share the handler policy changed here; backend-specific explicit
update and delete behavior is unchanged.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The documentation and code-comment checklist items are not applicable
because this change removes unsafe behavior without introducing a new
public interface or complex implementation. The full-suite checkbox
remains unchecked because four unrelated tests failed locally, as
documented above.
## Description
Close the passive-memory observability loop by recording access for
context rows that survive the final injection budget and tagging
requests where context is actually appended.
Closes#2211
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Track only memory IDs retained after ranking, similarity filtering,
entry limits, and final text truncation.
- Call optional backend `record_access` with stable de-duplication and
fail-open error handling.
- Extend structured injection logging to stamp `memory_injected=true`
when injected bytes are positive.
- Thread request tags through successful Anthropic, OpenAI Chat, OpenAI
Responses, Gemini, and Codex WebSocket injection sites.
- Add a static contract test that all current successful handler
injection logs pass tags.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uv run --with pytest --with pytest-asyncio --with numpy --with fastapi pytest \
tests/test_memory_handler_native_ops.py \
tests/test_memory_auto_tail.py \
tests/test_memory_handler_project_isolation.py \
tests/test_memory_injection_logging.py -q
51 passed
$ uv run --with ruff ruff check <touched Python files>
All checks passed!
$ git diff --check
(no output)
```
## Real Behavior Proof
- Environment: Python 3.13 with synthetic backend and handler fixtures
- Exact command / steps: run the focused test set above
- Observed result: only IDs present after the final text budget are
access-recorded; access-write failures remain fail-open; positive
injection logs stamp `memory_injected=true`; all six current successful
injection call sites pass tags
- Not tested: live provider requests, full repository suite, third-party
backends without `record_access`
## Review Readiness
- [x] I have performed a self-review
- [ ] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A
## Additional Notes
Access accounting is intentionally best-effort: unsupported backends and
write failures do not delay or fail the upstream model request.
Documentation and changelog changes are not needed for this internal
observability fix.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Pre-this-PR the auto-injected memory block rendered rows as `1. <content>`
with no addressable handle. To UPDATE or DELETE a row the model first had
to call memory_search to discover its ID — two round trips, against the
model-as-judge architecture.
This PR adds three tightly-coupled affordances so the model can act on
memory directly:
1. Auto-tail rows now carry the memory ID:
`1. [mem_alpha_001] User prefers Python`
The bracketed token is the canonical ID — same identifier accepted by
memory_update and memory_delete.
2. New `memory_list` tool — chronological browse (vs `memory_search`'s
semantic lookup). Returns recent memories with their IDs. Backend
dispatches to `Backend.list_memories` if available, else falls back
to an empty-query `search_memories`. Caps at 100 entries.
3. ID-usage guidance text appended to the auto-tail block. Tells the
model that bracketed IDs can go straight to memory_update /
memory_delete with no intervening search. The guidance lives in the
user-message tail (never system) — preserves cache-prefix byte
stability (invariant I2).
`memory_update` and `memory_delete` tool descriptions also point at the
[id] block as a valid ID source — keeps tool docs consistent with the
new affordance.
Verification:
- 10/10 tests pass in tests/test_memory_auto_tail.py (incl. 2 new
guidance tests + 2 new ID-format tests)
- 31/31 tests pass in tests/test_memory_handler_native_ops.py (incl. 4
new memory_list dispatch tests + existing assertions updated for the
[id] format change)
- Golden fixtures regenerated for the tool-description copy changes
(tests/fixtures/memory_tool_definitions/{anthropic,openai}.json)
- Live end-to-end test against real Anthropic API
(tests/test_proxy_memory_integration.py::TestMemoryIdAutoTailAndUpdate):
seeded memory → auto-tail → Claude → memory_update with exact ID.
PASSED.
PR-this removed the 500-char query truncation in
``memory_handler._extract_user_query``. The pre-existing assertion at
``test_memory_handler_native_ops.py:845`` was testing the old buggy
behaviour (output truncated at 500) and CI caught it. Updated to
assert the full-fidelity return.
Memory retrieval was partitioned only by `x-headroom-user-id`. Claude
Code never sets that header, so every project a user worked on landed
in one global `default` bucket; the proxy then injected semantically
similar memories from that mixed bucket into every `/v1/messages`
request, regardless of which repo the session was actually about. The
injected `## Relevant Memories` block reads like a prompt-injection
payload and Claude has been seen to refuse to act on it, defeating the
feature.
This change makes leakage structurally impossible by giving each
resolved workspace its own SQLite database file. The wrong DB is
simply not open during a request.
- `headroom/memory/storage_router.py` (new) — `MemoryStorageMode`
(project/user/global), `ProjectResolver` (x-headroom-project-id →
x-headroom-cwd → --memory-project-root CLI override → env-block
parse: `Primary working directory:` / `Working directory:` / `cwd:`,
no regex), and `BackendRouter` with an LRU of open `LocalBackend`s
keyed by db_path.
- `proxy/memory_handler.py` — `MemoryConfig.storage_mode` defaults to
`PROJECT`. Provider handlers build a `RequestContext` once and pass
it through; `search_and_format_context`, `handle_memory_tool_calls`,
and the `_execute_*` methods route save/search/update/delete on the
per-project backend. Qdrant-neo4j gets a composite
`user::project_key` partition so external Mem0-style deployments
also isolate per project without a parallel collection.
- Fix C — injected block carries provenance:
`## Relevant Memories (workspace: <basename>, scope: project)`.
CCR proactive-expansion block gets a matching workspace tag.
- `memory/factory.py` — process-wide embedder cache so opening N
project DBs doesn't load the embedder N times. OpenAI key
validation runs ahead of the cache.
- CLI — `--memory-storage={project,user,global}` (default `project`),
`--memory-project-root` override, rewritten `--memory` help text,
banner reports storage mode.
- Migration UX — if the legacy single-file DB has content while
project mode is active, an INFO log points users at
`--memory-storage=global`. Bridge currently only syncs the legacy
DB; a WARN fires when bridge + project mode are combined.
Backward-compatible: legacy `~/.headroom/memory.db` untouched and
reachable via `--memory-storage=global`. `request_context` is
keyword-only on entry points so existing tests/mocks keep working.
Tests: 24 new (resolver tiers, LRU eviction, two-cwd isolation,
user-mode partition, legacy fallback, provenance headers); full
suite 5260 passing, ci-precheck green.
The `qdrant-env-vars` change in d3c37d7 (PR #266) added `qdrant_url` and
`qdrant_api_key` keys to the kwargs that `MemoryHandler` passes into
`DirectMem0Adapter.__init__`. The corresponding assertion in
`test_ensure_initialized_fast_paths_and_qdrant_variants` was missed in that
PR and has been failing on `main` ever since. Surfacing here because it
fails on every PR's CI; not caused by the Rust tokenizer work this branch
adds.
The two new keys are both `None` when the corresponding `HEADROOM_QDRANT_*`
env vars are unset, which is the case in this test.
