Adds `HEADROOM_QDRANT_URL`, `_HOST`, `_PORT`, `_API_KEY`, `_HTTPS`,
`_PREFER_GRPC`, `_GRPC_PORT` support across the memory stack:
- `headroom/memory/qdrant_env.py`: shared resolver helper with
explicit-arg > env > default precedence (URL wins over host/port;
booleans parsed via standard truthy set).
- `memory/easy.py`, `backends/{mem0,direct_mem0}.py`,
`proxy/memory_handler.py`: call the resolver so
`Memory(backend="qdrant-neo4j")`, `Mem0Config`, and the proxy's
`MemoryConfig` all honor the same env keys.
- `proxy/models.py` + `proxy/server.py`: `ProxyConfig` picks up the
same keys so hosted Qdrant (e.g. Qdrant Cloud) works without code
changes.
- `cli/proxy.py`: adds `--memory-qdrant-{url,host,port,api-key}`
flags that override the env when present.
- `tests/test_memory/test_qdrant_env.py`: unit coverage for
precedence, URL-vs-host/port, boolean parsing, and unset defaults.
- `CHANGELOG.md`: documented under [Unreleased] / Added.
Explicit constructor arguments still win; unset env keeps the existing
localhost:6333 defaults, so this is backwards-compatible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but
74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings,
violating that contract. Every macOS/Linux clone reports these files as
"modified" on fresh checkout because git's diff engine sees the stored
bytes don't match the attribute contract, even though the working tree
and index match byte-for-byte.
Running `git add --renormalize .` rewrites each affected blob so the
stored form matches the attribute declaration. No semantic changes —
every affected file's diff is "N insertions, N deletions" with inserts
and deletes being the same lines modulo line endings.
Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub
blame skip this mechanical commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Learned: error recovery section was bloating with stale, near-duplicate,
and contradictory entries because the dedup key was the literal rendered
bullet text and there was no TTL or re-validation.
- Normalize the hash key for error_recovery patterns. Read recoveries key
on (basename(error_path), basename(success_path)); Bash recoveries strip
volatile suffixes (| tail -N, 2>&1, etc.) and hash only the primary
command before the first | or &&. Non-error-recovery categories keep
literal-content hashing.
- Stamp first_seen_at / last_seen_at on every pattern; bump both in
_bump_persisted_evidence via json_set. Stored in metadata JSON — no
schema change.
- Refine at render time (error_recovery only): drop rows not re-observed
in 21 days, re-validate Read success paths against the filesystem,
collapse same-error_path-with-multiple-targets into one "use Glob/Grep
first" bullet, rank by evidence_count * 0.5 ** (days/5), cap at 15
bullets.
15 new tests (TestNormalizedHash, TestRefineErrorRecovery). Full suite:
526 passed, 1 skipped. Ruff + mypy clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Make the ONNX + sqlite-vec memory path truly batched.
Batch ONNX embed_batch calls, batch sqlite-vec index/remove work under a single cached connection, and update MCP warm-up to use batch embed/save/index flows.
Add focused regression tests for ONNX batching, sqlite-vec single-connection batch behavior, and MCP warm-up batching.
Skip the MCP-specific test when optional MCP dependencies are not installed.
Refs #240
The test added in bb91cfe used ``CliRunner(mix_stderr=False)`` to keep
stderr separate from stdout for assertion purposes. That parameter was
removed in Click 8.2. The repo's pyproject.toml pins ``click>=8.1.0``,
so either Click 8.1 (needs mix_stderr) or Click 8.2+ (must omit it)
could appear in CI.
Switch to reading ``result.stderr`` when the attribute is populated,
falling back to ``result.output`` (combined stream) otherwise. This
covers every Click 8.x variant without branching on the installed
version.
Verified in the Docker e2e image (Click 8.3.3): all 45 tests in
tests/test_cli/test_init_cli.py pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When users hit an init regression it's opaque why: no visible state
about which agents were probed, which paths were written, which
subprocesses ran. Add a top-level flag to ``headroom init`` that routes
debug-level logging from the ``headroom.cli.init`` logger to stderr.
Instrumented decision points:
* detect_init_targets / _probe_init_targets — scope + per-target
shutil.which result
* _write_json, _ensure_claude_hooks, _ensure_copilot_hooks,
_ensure_codex_hooks, _ensure_codex_provider — file paths being written
* _apply_user_env — chosen scope (windows vs unix) and env-var keys
* _run_checked — each subprocess command + exit code + truncated
stdout/stderr (useful when ``claude plugin install`` fails)
* _run_init_targets — target dispatch order and resolved profile
* top-level init callback — all flag values and invoked_subcommand
Log output goes to stderr so stdout stays clean for pipes. The handler
attached by ``_enable_verbose_logging`` is idempotent - nested
subcommand invocations don't duplicate output. The logger does not
propagate to the root logger, so enabling ``headroom init -v`` does not
affect the rest of the process.
The flag is declared on the parent Click group. Subcommands (claude,
codex, copilot, openclaw) inherit the enabled logger automatically
because the group callback runs before dispatch.
Added tests cover:
* ``init -v`` emits the expected markers to stderr, including
``detect_init_targets``, ``global_scope=True``, and each agent name
* ``_enable_verbose_logging`` is safe to call repeatedly (handler
remains singular)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes#245.
Running ``headroom init -g`` with no supported agents on PATH previously
produced a single-line ClickException that read like the -g flag had
been removed:
Error: No supported user init targets were auto-detected. Specify one explicitly.
This left reporter #245 concluding the feature was gone. Replace that
message with a structured diagnostic that:
* states which scope (user / local) was tried
* lists every target probed (claude, codex, copilot, openclaw) and the
shutil.which() result for each
* explicitly confirms that -g / --global is still a supported flag
* shows the concrete per-target invocation for each agent
(``headroom init -g claude``, ...) so the user knows the escape hatch
The implementation factors ``detect_init_targets`` into a ``_probe_init_targets``
helper that returns ``[(name, which_result)]``. ``detect_init_targets``
keeps its existing signature so the test suite and external imports
aren't broken; the new helper backs both the auto-detection path and
the diagnostic error formatter.
Unit tests in tests/test_cli/test_init_cli.py cover:
* the end-to-end message shape (structural markers + every target name +
the example invocation)
* the local-scope variant omitting global-only agents (copilot / openclaw)
* that found binaries are surfaced with their absolute path so users can
debug cases where shutil.which returns an unexpected result
No behavior change when at least one target is detected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`headroom wrap codex` injects a `model_provider = "headroom"` block
plus a `[model_providers.headroom]` table into `~/.codex/config.toml`
so Codex routes both HTTP and WebSocket traffic through the proxy. The
matching `unwrap codex` subcommand did not exist, so the injected
block stayed in `config.toml` forever — the moment the proxy stopped,
Codex (CLI and macOS app) started erroring with
`Missing environment variable: OPENAI_API_KEY`, and users had to hand-
edit the file to recover.
Fix:
* `_inject_codex_provider_config` now snapshots the pre-wrap file to
`~/.codex/config.toml.headroom-backup` before the first modification
and leaves that snapshot untouched on subsequent wrap runs. The
injection is also rewritten to use two self-contained marker-
delimited blocks (top-level key and provider table) so stripping
them never consumes user content that sits between them.
* `_inject_memory_mcp_config` takes the same snapshot, so
`wrap codex --memory` without a full provider injection is still
fully reversible.
* New `_restore_codex_provider_config` helper and `unwrap codex`
click command:
* backup present → restore byte-for-byte and delete the backup;
* backup absent but Headroom block present → strip the block and
keep surrounding user content;
* config contained only Headroom content → remove the file so
Codex falls back to defaults;
* nothing to undo → safe no-op.
Codex is the only wrap target that modifies a persistent user config
file: claude/aider/cursor/copilot all go through env vars or project-
scoped files only, so this bug was unique to Codex.
Tests:
* `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the
strip/snapshot helpers directly, round-trip idempotency of
wrap → wrap → unwrap, handling of malformed prior configs, and
end-to-end CliRunner invocations of `headroom wrap codex
--prepare-only` / `headroom unwrap codex` against a temp `$HOME`.
* All 153 existing `tests/test_cli/` tests continue to pass.
Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2)
by the `sync-plugin-versions` pre-commit hook; the previous values
(0.10.3) had drifted.
Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on
current `main` (0.11.x).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compute_turn_id hashed the raw message dicts, which meant the same
user-text message produced a different hash on each call of one agent
loop because clients (notably Claude Code) move the cache_control
breakpoint to the newest message per call. The user-text block carries
cache_control on call 1 and not on call 2, so the serialized prefix
differs and the turn_id rolls over. Effect downstream: every API call
becomes its own "turn" and any prompt-level aggregation (e.g. the
Headroom desktop app's prompt all-time record) collapses to the
largest single call, not the sum across the prompt.
Add a small recursive normalization pass that strips cache_control from
the hashed prefix and from list-shaped system prompts before hashing.
Two new tests cover cache_control moving between calls on both the
messages array and the system prompt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When `headroom learn` re-surfaced a section heading that already existed
in CLAUDE.md / MEMORY.md, the writer replaced that section wholesale —
but the LLM never saw the prior block, so it emitted condensed bullets
like "X is *also* large — same rule as Y, Z" assuming Y and Z would
remain siblings. After replacement, Y and Z were gone and the "also"
dangled.
This threads the project's current `<!-- headroom:learn -->` block (from
both CLAUDE.md and MEMORY.md) into the digest as a "Prior Learned
Patterns" section, and extends the system prompt to make the re-emission
contract explicit: re-stating a section replaces it wholesale, so the
LLM must copy forward prior bullets it still agrees with. Prior sections
the LLM omits entirely are still carried forward by the writer (#231
behavior preserved as a safety net).
Changes:
- New `extract_marker_block(file_content)` helper in `learn.writer` that
returns the raw marker block (delimiters included) or None.
- New `_build_prior_patterns_section(project)` in `learn.analyzer` reads
`project.context_file` and `project.memory_file` via the new helper
and formats a labeled section ahead of the per-session event stream.
- `_build_digest` emits the prior-patterns section when present; char
budget accounting unchanged (prior blocks are small).
- `_SYSTEM_PROMPT` gains a "Prior Learned Patterns" rule block telling
the LLM how to integrate prior bullets (preserve / revise / drop-only-
if-contradicted) and warning against unresolved cross-references.
- Tests: 6 new `TestPriorPatternsInjection` cases (present/absent files,
no-marker-block, both-files, end-to-end via mocked `_call_llm`); 4 new
`TestExtractMarkerBlock` cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /livez-unaffected test flaked deterministically on the Python 3.10
matrix job of this PR while passing on 3.11/3.12/3.13 and on main. Root
cause: the 3-request warmup did not cover every lazy-init path the
restructured proxy triggers on first request, so one measured sample
(consistently index 2 of 20) came in at 336-356ms instead of <1ms.
Compounding this, the assertion called `statistics.quantiles(n=100)[98]`
"p99" on only 20 samples — which collapses to `max(latencies)` and fails
on any single stall.
Fix the test for real, not just for this PR:
- Bump warmup from 3 to 10 to clear all lazy-init paths exposed by the
upstream canonical-pipeline restructure. CI traces placed the rogue
sample at measured-index 2 (request #6 overall), so 10 is comfortably
past every observed lazy boundary.
- Stop mislabelling `max(latencies)` as p99. With 20 samples, drop the
single worst outlier and assert on the next-worst. A genuine regression
(semaphore actually blocking /livez) still fails hard because every
sample would cluster near the drained timeout; a single GC/scheduler
jitter no longer trips the assertion.
- Drop now-unused `statistics` import.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflicts:
- CHANGELOG.md: upstream landed Live flush + traffic-learner fix entries.
Placed turn_id entry at the top of the first [Unreleased] ### Added
section so both features coexist; preserved upstream ordering.
- headroom/proxy/server.py: upstream restructured the file substantially,
producing a whole-file conflict. Took upstream's version and
re-applied the single-line `"turn_id": log.get("turn_id")` addition
to the /transformations/feed response dict at its new location.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Patch coverage on helpers.py was 87% — 5 lines of compute_turn_id
were untested. Add cases for: non-dict / non-user messages in the
reverse scan, empty-string user content (should keep scanning),
mixed text+tool_result content (agent-loop continuation, not a
turn boundary), and system=None (hashes without the system segment).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to b2536e6: add the Unreleased changelog entry describing the
prompt-turn identifier, and pick up the ruff-fixed import layout in the
new test file (ruff --fix of I001).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds compute_turn_id() helper that hashes (model, system, messages prefix
up to the last user text message). An agent loop sends the same user-text
prefix across every iteration plus a growing tool chain, so this id is
stable across the turn but rolls over when the user sends a new prompt.
Stamps the id onto RequestLog at all three call sites (anthropic handler
bedrock + direct branches, and the streaming handler) and surfaces it as
turn_id in /transformations/feed so downstream consumers can aggregate
savings per user prompt rather than per API call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the codecov gap flagged on PR 232 (88.89% → near 100% on the patch):
- A file with no marker block returns no prior recommendations.
- A marker block with nothing between the markers yields an empty list
(the re.split fast-path with zero sections).
- A stray `### ` with no heading text inside the block is silently
skipped (the `if not heading: continue` branch, previously
unexercised in tests) — a real section after it still parses cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 17 targeted tests to close the coverage gap on the new traffic_learner
paths (codecov flagged ~51%). Exercises:
- `flush_to_file` end-to-end with a fake learn plugin + writer: verifies
anchored patterns are bucketed per project, recommendations are passed
to the writer, writer exceptions are swallowed, and each early-return
branch (no plugin, no patterns, discover_projects failure, un-anchored
patterns) is hit without raising.
- `_resolve_backend_db_path` on None backend, backend without
`_config`, and backend with empty `db_path`.
- `_collect_all_patterns` merging persisted + accumulator patterns by
content_hash with summed evidence_count, plus the missing-DB branch.
- `_hydrate_persisted_state` with backend=None and with a backend
pointing at a non-existent DB file (both no-ops).
- `_bump_persisted_evidence` with no backend, missing DB, and
unknown memory id (all silent no-ops so the proxy hot path never
blows up on malformed state).
- `stop()` cancelling the flush task cleanly.
All new tests use the existing `_FakeBackend` + `_init_db` helpers so
they exercise real SQLite paths, not mocks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The test drained the anthropic pre-upstream semaphore and asserted that 20
subsequent /livez calls stayed under 100 ms. With only 20 samples, the p99
computation falls through to max(latencies) — one cold-start outlier was
enough to fail the test. Observed on CI py3.10 runners where the first
TestClient request paid ~330 ms of one-time ASGI lifespan / import /
route-resolution cost while every subsequent request was sub-ms.
Add 3 warm-up requests before timing starts. Preserves the test's
signal (if /livez were actually blocked on the drained semaphore, all
post-warmup samples would still be slow) while removing the
runner-speed flake.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
