mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
18 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Chester
|
c471800e8e
|
fix(memory): keep vector metadata in sync (#2295)
## Description
Fixes #2296.
Metadata-only memory updates can leave the primary store, vector-index
metadata, and cache inconsistent. TrafficLearner also performs an atomic
SQLite evidence increment that bypasses normal secondary-index refresh.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Refresh vector metadata for HierarchicalMemory metadata-only,
importance, and entity-reference updates.
- Add a LocalBackend path that reloads a memory from the primary store
and refreshes vector metadata plus cache state.
- Preserve the atomic TrafficLearner SQL evidence increment, then
refresh secondary state only when a row was updated.
- Keep refresh failures fail-open and distinguish them from
primary-store increment failures in logs.
- Add backend-neutral contract tests instead of inspecting a specific
vector adapter private field.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`) — verified locally: mypy
1.20.2, no issues in 504 source files
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
141 passed, 1 skipped
ruff check: passed
ruff format --check: passed
```
The first CI run exposed one backend-specific test assertion against
HNSW private state while CI used SQLiteVectorIndex. Commit
|
||
|
Abhay Singh
|
1f5fefffd3
|
fix(memory): bound the TrafficLearner pending-pattern accumulator (memory leak) (#2579)
## Description
`TrafficLearner` (the memory/learning subsystem that accumulates
patterns from proxy traffic) has an unbounded in-memory accumulator.
`_pattern_counts` maps `content_hash -> (pattern, count)`. A pattern is
added on first sighting, its count is bumped on each re-sighting, and it
is **removed only when it reaches `min_evidence`** (default 5), at which
point it is promoted and its hash moves to `_saved_hashes`:
```python
if h in self._pattern_counts:
existing, count = self._pattern_counts[h]
count += 1
self._pattern_counts[h] = (existing, count)
else:
self._pattern_counts[h] = (pattern, 1)
return # first sighting — wait for more evidence
...
if count >= self._min_evidence:
del self._pattern_counts[h] # only removal path
self._saved_hashes.add(h)
if len(self._saved_hashes) > self._dedup_window: # sibling IS trimmed
self._saved_hashes.pop()
```
A pattern seen **once but never corroborated** — the common case for
one-off traffic (a unique error string, an ad-hoc shell command, a
distinct file path) — never reaches `min_evidence`, so it is **never
removed**. Over a long-lived proxy processing varied traffic,
`_pattern_counts` grows without bound and RSS climbs. The sibling
`_saved_hashes` is explicitly trimmed to `dedup_window` ("prevent
unbounded growth"); `_pattern_counts` was missed.
Reproduced directly: feeding 500 distinct one-off patterns leaves 500
entries in `_pattern_counts` (one per pattern, forever).
## Fix
Make `_pattern_counts` an LRU-ordered `OrderedDict` capped at a new
`max_pending_patterns` (default 2048):
- On each corroboration, `move_to_end(h)` so an actively-accumulating
pattern stays "fresh" and is never evicted before it can be promoted.
- On a first sighting when the accumulator is full, evict the
least-recently-corroborated pending entry (`popitem(last=False)`).
Evicting a stale one-off is safe: if it recurs it simply restarts
accumulation (delayed promotion at worst) — the same tradeoff
`_saved_hashes` already makes. Promotion at `min_evidence` is unchanged,
and the cap (2048) is generous enough that any pattern receiving repeat
sightings within a normal window reaches `min_evidence=5` long before
eviction.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring
## Changes Made
- `headroom/memory/traffic_learner.py`: `_pattern_counts` becomes a
capped LRU `OrderedDict`; add `max_pending_patterns` (default 2048);
`move_to_end` on corroboration and evict-oldest on overflow.
- `tests/test_memory/test_traffic_learner.py`: a regression that 500
one-off patterns keep the accumulator at its cap, and one that a
corroborated pattern still promotes into `_saved_hashes` (both sync via
`asyncio.run` so they run without the pytest-asyncio plugin).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_memory/test_traffic_learner.py -q
35 failed, 109 passed
# the 35 failures are pre-existing @pytest.mark.asyncio tests that need
# pytest-asyncio (not configured in this environment); they fail identically
# on clean main (35 failed, 107 passed) and pass in CI. My two new tests are
# synchronous and pass; they add +2 passing with no new failures.
# with the fix reverted, test_pending_accumulator_is_bounded fails
# (the accumulator holds all 500 one-off patterns)
$ uvx ruff@0.15.17 check headroom/memory/traffic_learner.py tests/test_memory/test_traffic_learner.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/memory/traffic_learner.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: built a `TrafficLearner(backend=None,
min_evidence=5, max_pending_patterns=8)` and drove `_accumulate` with
500 distinct one-off `ExtractedPattern`s; separately corroborated one
pattern to `min_evidence`; then reverted the source and re-ran.
- Observed result: with the fix `len(_pattern_counts)` stays at the cap
(8) after 500 one-offs, the corroborated pattern is removed from pending
and present in `_saved_hashes`, and an actively-bumped pattern survives
LRU eviction; with the fix reverted the accumulator holds all 500
one-off entries (the unbounded leak). Ran against the actual module.
- Not tested: a live multi-day proxy run measuring RSS (the leak is
inferred from the removed unbounded-growth path; the accumulator bound
is verified directly).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
gglucass
|
a70e5ff78d
|
fix(learn): run project discovery off the event loop (#2731)
## Description
`TrafficLearner.flush_to_file` is a coroutine, but it called
`plugin.discover_projects()` inline. That function walks the filesystem
to decode escaped project directory names — in
`learn/plugins/claude.py`, `_greedy_path_decode` recurses through
`iterdir()` at every level and tries each tokenization of each child,
backtracking on a miss — so on a large home tree it runs for minutes.
Doing that on the event loop freezes uvicorn for the whole window. The
port keeps accepting TCP, but `/readyz` never answers, so a supervisor
health-checking the proxy kills a process that is merely busy.
Field thread dumps show exactly that:
```
Current thread (most recent call first):
File "python3.12/pathlib.py", line 1056 in iterdir
File "headroom/learn/plugins/claude.py", line 454 in _greedy_path_decode
File "headroom/learn/plugins/claude.py", line 478 in _greedy_path_decode
File "headroom/learn/plugins/claude.py", line 478 in _greedy_path_decode
File "headroom/learn/plugins/claude.py", line 426 in _decode_project_path
File "headroom/learn/plugins/claude.py", line 71 in discover_projects
File "headroom/memory/traffic_learner.py", line 591 in flush_to_file
File "headroom/memory/traffic_learner.py", line 535 in _flush_worker
File "python3.12/asyncio/events.py", line 88 in _run
File "python3.12/asyncio/base_events.py", line 1999 in _run_once
File "python3.12/asyncio/base_events.py", line 645 in run_forever
File "uvicorn/server.py", line 75 in run
File "headroom/proxy/server.py", line 4992 in run_server
```
Accompanying signals from the same incidents: port accepts TCP,
`/readyz` times out, process CPU 2-13s across the window (I/O bound, not
spinning), proxy log silent 66-336s.
## Type of Change
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/memory/traffic_learner.py`: `flush_to_file` now awaits
`asyncio.to_thread(plugin.discover_projects)` instead of calling it
inline. `asyncio` was already imported. The result is cached per learner
(`_project_roots_cache`), so the steady-state flush path pays nothing
for the thread hop.
- `tests/test_memory/test_traffic_learner.py`: added
`test_discover_projects_does_not_block_the_event_loop`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_memory/test_traffic_learner.py -q
.................................................. [100%]
============================= 152 passed in 2.93s ==============================
$ uvx ruff check headroom/memory/traffic_learner.py tests/test_memory/test_traffic_learner.py
All checks passed!
$ uvx ruff format --check headroom/memory/traffic_learner.py tests/test_memory/test_traffic_learner.py
2 files already formatted
$ uv run --frozen --extra dev mypy headroom/memory/traffic_learner.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS 15.6 arm64, Python 3.10.18, pytest 9.0.3, branch
off `main` @ `
|
||
|
Chester
|
3eb0122068
|
fix(learn): filter ambient user-role scaffolding (#2275)
## Description Fixes #2274. Headroom Learn currently trusts `role=user` as sufficient preference provenance. Agent harnesses can transport ambient UI and orchestration context in user-role messages, and OpenAI Responses normalization also promotes missing roles to `user`. Correction-like text in those inputs can therefore become durable user preferences. This change keeps preference learning fail-closed for known non-user sources while preserving genuine user corrections. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Refactoring only ## Changes Made - Preserve missing OpenAI Responses roles as `unknown` instead of promoting them to `user`. - Canonicalize user-role text before preference extraction. - Remove proxy-appended `## Relevant Memories` suffixes from preference evidence. - Reject strict ambient-only harness prefixes such as heartbeat, environment, workspace-instruction, delegation, and app-context envelopes. - Apply the same guard in `on_messages` and `_extract_preferences` for defense in depth. - Add regression coverage for system/developer/unknown roles, ambient-only user messages, memory-only messages, and mixed genuine-user-plus-memory input. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text 149 passed, 1 warning ruff check: passed ruff format --check: passed git diff --check: passed ``` Focused test files: ```text tests/test_memory/test_traffic_learner.py tests/test_openai_responses_traffic_learner.py ``` ## Real Behavior Proof - Environment: macOS; Python 3.13; current Headroom main; direct invocation of the real `TrafficLearner` class, with no proxy or database mocks - Exact command / steps: create `TrafficLearner(backend=None, min_evidence=1)`; feed system, developer, heartbeat user-role, and memory-only user-role messages; read `patterns_extracted`; feed a genuine user correction followed by a `## Relevant Memories` suffix; read `patterns_extracted` again - Observed result: `ambient_patterns=0`, `after_user_patterns=1` — the ambient batch produced no preference evidence; the genuine correction produced one pattern, while the appended memory content did not become evidence - Not tested: live provider traffic against a remote OpenAI endpoint; every possible third-party harness envelope; migration or cleanup of already-persisted noisy memories ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] No new dependency - [x] Fail-open proxy behavior is unchanged - [x] Regression tests added - [x] Public examples contain no real user data - [x] CHANGELOG update, if requested (not requested — N/A) ## Additional Notes This extends the source filtering introduced by #466 rather than replacing it. The prefix checks are deliberately strict and anchored at the start of a canonicalized message. The intended failure mode is a missed preference, not durable storage of non-user instructions. Note: the strict prefix set was discussed and confirmed in JerrettDavis's review approvals. |
||
|
Abhay Singh
|
6cdfd3f64d
|
fix(proxy/openai): feed chat/completions traffic into the traffic learner (#2333)
## Description Addresses the chat/completions portion of #2060. The live traffic learner is wired into the Anthropic `/v1/messages` handler and, since then, the OpenAI Responses HTTP handler (`_observe_openai_responses_traffic`, called from `handle_openai_responses`). But `handle_openai_chat` has **no** ingestion call site: ```text headroom/proxy/handlers/openai.py handle_openai_responses -> _observe_openai_responses_traffic (wired) handle_openai_chat -> (no traffic_learner call) (gap) ``` So OpenAI-compatible clients that route through `/v1/chat/completions` — GitHub Copilot CLI, opencode, OpenAI SDKs — run through an apparently healthy proxy with Learn enabled while producing no learned patterns: the learner starts, but it never receives their tool results or user messages. ## Fix Observe the original client payload (before memory/compression mutates it) at the top of `handle_openai_chat`, mirroring the Responses and Anthropic ingestion paths: ```python await self._observe_openai_chat_traffic(original_client_messages, request_id=request_id) ``` `_observe_openai_chat_traffic` is the chat counterpart of `_observe_openai_responses_traffic`: same lazy backend wiring, same `on_tool_result` / `on_messages` lifecycle, same fail-soft `try/except`. The one format-specific piece is tool-result extraction. chat/completions encodes tool calls differently from Anthropic — the call is on an assistant message's `tool_calls` array (`id` -> function `name` + `arguments`) and each result is a separate `role: "tool"` message keyed by `tool_call_id`, so the existing `extract_tool_results_from_messages` (which scans for Anthropic `tool_use`/`tool_result` blocks) finds nothing. A new `TrafficLearner.extract_tool_results_from_openai_messages`: - builds the `tool_call_id -> function` map from assistant `tool_calls`; - for each `role: "tool"` message, resolves the tool name and joins string-or-list content; - parses the OpenAI `arguments` JSON string into a dict, so the downstream environment/recovery extractors (which call `input.get("command")`, `input.get("file_path")`, ...) see the same shape as an Anthropic `tool_use.input` instead of a raw string; - sniffs `is_error` from the output (chat tool messages carry no error flag). It returns the same `{tool_name, input, output, is_error}` shape as the Anthropic extractor, so `on_tool_result` stays format-agnostic. User-message preference extraction (`on_messages`) already reads plain `role`/`content`, so it consumes chat messages unchanged. Scope: this wires the **chat/completions** path. Codex WebSocket ingestion (`handle_openai_responses_ws`) additionally needs per-`response.create` evaluation plus transcript-replay baselining on reconnect, so it is intentionally left as a follow-up rather than half-implemented here. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/memory/traffic_learner.py`: add `extract_tool_results_from_openai_messages` (OpenAI chat tool-result extraction with `arguments` JSON parsed to a dict). - `headroom/proxy/handlers/openai.py`: add `_observe_openai_chat_traffic` and call it from `handle_openai_chat` on the original client payload. - `tests/test_memory/test_traffic_learner.py`: cover the OpenAI extractor (name resolution, arguments parsing, list content, error sniff, malformed/orphan handling, empty case). - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/memory/traffic_learner.py headroom/proxy/handlers/openai.py tests/test_memory/test_traffic_learner.py All checks passed! $ uvx ruff@0.15.17 format --check <same files> 3 files already formatted $ uvx mypy@1.20.2 --ignore-missing-imports headroom/memory/traffic_learner.py # clean for this file (the one reported error is a pre-existing # headroom/_subprocess.py:18 no-any-return, unrelated to this change and # present on main with these edits stashed) ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx mypy@1.20.2`. A full `pytest` OOMs this box (ML-stack import), so I reproduced the extractor with a dependency-free script and left the full pytest to CI. - Exact command / steps: replicated `extract_tool_results_from_openai_messages` and ran it over a typical chat round-trip (assistant `tool_calls` for `bash` + `read_file`, then two `role: "tool"` results, one erroring and one with list content), plus malformed-`arguments`, orphan-`tool_call_id`, and no-tool cases. - Observed result: tool names resolved from the call-id map; `arguments` parsed to a dict so `input.get("command")` works; list content joined; `is_error` sniffed from output; malformed arguments degrade to `{}` and an orphan id yields `unknown` without raising. The added unit tests assert the same through a real `TrafficLearner`. - Not tested: a live Copilot CLI session end to end; the added tests drive `TrafficLearner.extract_tool_results_from_openai_messages` directly, matching the existing `test_extract_tool_results_from_messages` pattern. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The "unit tests pass locally" box is unchecked because a local pytest run imports the ML stack and OOMs this box; the added tests reuse the existing `TrafficLearner(backend=None, ...)` harness in `test_traffic_learner.py` (no real backend) and run under the normal CI pytest job, and the extractor behavior is corroborated by the standalone proof above. This PR is deliberately scoped to `/v1/chat/completions`; I'm happy to follow up with the Codex WebSocket ingestion path (which needs the transcript-replay baselining discussed in the issue) as a separate change if useful. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Rudimar Ronsoni
|
b4571cc346
|
feat: headroom wrap opencode / unwrap opencode CLI (#1105)
## Summary This PR implements transparent `headroom wrap opencode` support without asking users to edit OpenCode provider URLs, choose an extra CLI flag, or maintain a static provider list. The wrapper now lives at the runtime transport boundary: OpenCode keeps its user/provider config, while Headroom intercepts outbound provider traffic in-process and routes it through the local Headroom proxy. ## What changed ### Transparent OpenCode wrapping - `headroom wrap opencode` injects the `headroom-opencode` plugin through `OPENCODE_CONFIG_CONTENT`. - Existing OpenCode provider URLs are preserved. We do not rewrite user config URLs to point at Headroom. - Existing `OPENAI_BASE_URL` and `ANTHROPIC_BASE_URL` env vars are preserved. - Local OpenCode traffic, localhost traffic, and Headroom proxy traffic bypass the shim to avoid loops. ### Runtime transport interception - Added an OpenCode plugin transport shim that wraps: - `globalThis.fetch` - `http.request` / `http.get` - `https.request` / `https.get` - External provider calls are routed to the local Headroom proxy. - The original upstream origin is passed through `x-headroom-base-url`, so the proxy can forward to the real provider without changing OpenCode config. - External `http2.connect` is blocked loudly instead of allowing direct provider traffic to leak outside Headroom. ### Live provider additions Provider coverage is no longer based on a static config scan. Because routing happens at outbound request time, providers added mid-session are routed through Headroom automatically as long as they use the covered Node transport paths. ### Subagent and child-process coverage - The parent OpenCode plugin sets a packaged Node preload shim through `NODE_OPTIONS=--import=.../hook-shim/handler.js`. - The transport shim patches `child_process.spawn`, `exec`, `execFile`, and `fork` so child Node processes receive the Headroom preload even when OpenCode passes a custom `env`. - The child-process shim fails closed if it loads without `HEADROOM_OPENCODE_TRANSPORT_PROXY_URL`. - This closes the subagent leak path where a child Node process could otherwise start without Headroom transport interception. ## Why this goes beyond PR #1089 PR #1089 improves OpenCode provider registration, but it still focuses on provider config shape. This PR moves the enforcement boundary to runtime transport interception. This PR goes further because: - No provider URL rewriting is required. - New providers added mid-session are covered automatically. - Subagents and child Node processes inherit the Headroom transport shim. - Direct external HTTP/2 paths fail loudly instead of leaking. - The wrap remains transparent to the user's OpenCode provider config. - The wrapper is fail-closed for unsupported child-process preload state. ## Additional robustness fixes While validating the change in Docker, the full Python suite exposed unrelated Linux/container robustness issues. These are fixed in this PR so the suite is green: - Binary cache handling now treats cache paths under a non-writable existing parent as unavailable, including when tests run as root in Docker. - `release_version.py` honors `MANUAL_VER` before git calls so direct script execution works outside a `.git` checkout. - Test logger isolation now resets relevant Headroom child loggers so proxy logging setup cannot poison later `caplog` tests. - The scanner missing-path test now uses a guaranteed missing `tmp_path` child instead of relying on `/nonexistent/path`. ## Validation All implementation validation was run inside Docker. - Full Python suite from a fresh Docker copy: `6605 passed, 523 skipped`. - Ruff on changed Python/OpenCode paths: passed. - OpenCode plugin typecheck: passed. - OpenCode plugin tests: `9 passed`. - OpenCode plugin build: passed. - Hook shim preload smoke test: passed. ## Notes This PR intentionally does not add a CLI option. `headroom wrap opencode` means full wrap. Either Headroom wraps OpenCode transparently, or the path fails loudly instead of silently leaking provider traffic. --------- Co-authored-by: Rudimar Ronsoni <6081613+rudironsoni@users.noreply.github.com> |
||
|
chopratejas
|
0be0eede9e |
fix(memory): traffic_learner indexes system-reminder fragments as user preferences (refs #464)
`TrafficLearner._extract_preferences` ran three regex patterns over raw
user-message text and saved any match as a `User preference: <captured>`
memory. Two compounding bugs made ~10% of the reporter's saved memories
(187 of 1796) garbage:
1. **System-reminder content was matched.** Claude Code injects
`<system-reminder>…</system-reminder>` blocks into user-role
messages — scaffolding ("don't mention this reminder", "use colgrep
instead of Grep", "never bypass signing") that hits every correction
trigger. The learner happily persisted scaffolding as authoritative
user preferences.
2. **Capture groups were fixed-length windows.** `(.{10,100})` grabbed
the next 10–100 chars with no boundary awareness, producing
mid-word truncations like `User preference: of Grep, Glob. When
spawning agents, mention colgrep features a`.
This change rewrites `_extract_preferences` to be **regex-free** and
adds two layered defences:
- `_strip_system_reminders` (literal `str.find` scan, no regex)
removes `<system-reminder>…</system-reminder>` blocks from user
text before any pattern matching. Unclosed reminders drop to
end-of-string. Case-insensitive on the tag name only. ~95% of the
reporter's noise sample comes from this single layer.
- A token-based correction scanner replaces the three `re.compile`
patterns. It tokenises on whitespace (lowercasing once, up front),
matches trigger sequences as ordered token lists (`don't`, `do not`,
`stop`, `never`, `avoid`, `no use`, `no try`, `no do`, `instead`),
and captures the trailing content until a sentence terminator
(`.!?\n`) or end-of-input. Captures shorter than 10 chars are
rejected (stray triggers), and captures that hit the 78/98-char cap
without finding a terminator are rejected (rambling fragments). The
former noise — `colgrep instead of Grep, Glob. When spawning…` —
fails this gate; short complete user utterances
(`don't use git push, I'll push manually`) still pass because
end-of-input counts as a boundary.
Net regex count in this file: -3, +0.
`_hydrate_persisted_state` already runs in `start()` and seeds
`_saved_hashes`/`_persisted_ids` from prior rows, so cross-restart
dedup is already wired up — the reporter's "doesn't survive restarts"
note was partially outdated. The narrow remaining edge (in-process
`_dedup_window=100` eviction within a single very-long-running
process) self-heals on next restart and is left as a separate
follow-up.
Tests: 17 new across `TestStripSystemReminders`,
`TestExtractPreferencesSystemReminderFiltering`,
`TestExtractPreferencesRealCorrections`, and
`TestExtractPreferencesSentenceBoundary`. Full traffic_learner suite:
139 passing. ci-precheck green.
|
||
|
Tejas Chopra
|
5ceca13c65 | fix: harden learn path handling across platforms | ||
|
Garm
|
4512a0626e |
test(traffic-learner): cover helper edge cases + apply ruff format
CI flagged two issues on the rebased branch: 1. ruff format --check failed on server.py and test_traffic_learner.py after the rebase; line-collapse / trailing-whitespace nits. 2. Codecov reported 80% patch coverage with 20 lines missing in the matcher helpers — mostly branches not exercised by the high-level tests (empty Levenshtein inputs, source-prefix Bash parsing, env-var skip, equal-string short-circuit in binary match, the substantive- token path that beats the edit-distance gate, error_recovery patterns with non-canonical content in _drop_contradictions). Adds 16 targeted unit tests for those branches and applies ruff format. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garm
|
606131451b |
fix(traffic-learner): tighten matchers and drop contradictions
The recovery matchers paired any failed and successful tool call within a 5-call window with no semantic check that the pair was actually a retry. This produced confidently-wrong rules like: File `state.rs` does not exist. The correct path is `lib.rs`. …where the user simply read two unrelated files in the same directory. Across sessions the same user can also typo in opposite directions, producing directly contradictory rules side by side. This commit adds three structural checks: 1. Read recovery: require the failed and successful basenames to be identical or close in Levenshtein distance. Rejects the "same dir, different file" case that was the most common noise source. 2. Bash recovery: require both commands to share a binary (allowing path-prefixed variants and short prefix-versions like `python` ↔ `python3`) AND either have low normalized edit distance or share a substantive non-flag token. Rejects pairs that share only the binary name but differ in every meaningful argument. 3. Contradiction filter on flush: detect A→B and B→A pairs in error_recovery patterns and drop both. They almost always indicate opposite-direction typos in different sessions, not stable advice. Also: stash failed_path in metadata so the contradiction filter and downstream consumers can reason about pairs without parsing content. Tests: 13 new tests covering the heuristics directly. Existing tests exercising legitimate recoveries (`python`→`python3`, `ruff`→`.venv/bin/ruff`, `pip install`→success) continue to pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garm
|
a8ebf9ac5e |
test(traffic-learner): regression test for shutdown evidence gate
Asserts that stop()'s final flush_to_file does not bypass the evidence threshold. Earlier behavior collapsed the gate to 1 at shutdown, persisting every singleton pattern. This guards against that change sneaking back in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garm
|
ac493cba1e |
test(memory): raise patch coverage from 83% to 98% on error_recovery fixes
26 new tests covering: - TestNormalizeBashForHash — empty string, no-suffix, head/tail strip, trailing context flags, stderr redirect, chain-boundary truncation - TestParseIsoTimestamp — None, empty, non-string, invalid format, naive (assumed UTC), tz-aware preserved - TestLoadPersistedPatternsTimestamps — reads first_seen_at/last_seen_at from metadata, falls back to created_at, collision-merges timestamps and bumps importance to max, handles malformed JSON and non-numeric importance cells gracefully - TestBumpPersistsLastSeenAt — verifies _bump_persisted_evidence writes $.last_seen_at into metadata JSON - TestHydrateLegacyRow — legacy rows without category, rows with unknown/invalid category, rows with empty content - TestCollectAllPatternsTimestamps — in-session re-sighting bumps last_seen_at past stale persisted timestamp - TestRefineErrorRecovery (additions) — refine-empties-section skips recommendation entirely, OSError during re-validation keeps the row, Read patterns without success_path skip re-validation cleanly Remaining uncovered lines in patch (4): defensive exception handlers in _hydrate_persisted_state (sqlite connect OperationalError, asyncio thread exception, JSONDecodeError on metadata) that require heavy mocking for marginal value. 91 tests pass, ruff + mypy clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garm
|
879064fea5 |
fix(memory): collapse and decay error_recovery patterns in MEMORY.md
The Learned: error recovery section was bloating with stale, near-duplicate, and contradictory entries because the dedup key was the literal rendered bullet text and there was no TTL or re-validation. - Normalize the hash key for error_recovery patterns. Read recoveries key on (basename(error_path), basename(success_path)); Bash recoveries strip volatile suffixes (| tail -N, 2>&1, etc.) and hash only the primary command before the first | or &&. Non-error-recovery categories keep literal-content hashing. - Stamp first_seen_at / last_seen_at on every pattern; bump both in _bump_persisted_evidence via json_set. Stored in metadata JSON — no schema change. - Refine at render time (error_recovery only): drop rows not re-observed in 21 days, re-validate Read success paths against the filesystem, collapse same-error_path-with-multiple-targets into one "use Glob/Grep first" bullet, rank by evidence_count * 0.5 ** (days/5), cap at 15 bullets. 15 new tests (TestNormalizedHash, TestRefineErrorRecovery). Full suite: 526 passed, 1 skipped. Ruff + mypy clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garm
|
b2536e602a |
test(learn): cover flush_to_file, backend edge cases, and hydrate/bump error paths
Adds 17 targeted tests to close the coverage gap on the new traffic_learner paths (codecov flagged ~51%). Exercises: - `flush_to_file` end-to-end with a fake learn plugin + writer: verifies anchored patterns are bucketed per project, recommendations are passed to the writer, writer exceptions are swallowed, and each early-return branch (no plugin, no patterns, discover_projects failure, un-anchored patterns) is hit without raising. - `_resolve_backend_db_path` on None backend, backend without `_config`, and backend with empty `db_path`. - `_collect_all_patterns` merging persisted + accumulator patterns by content_hash with summed evidence_count, plus the missing-DB branch. - `_hydrate_persisted_state` with backend=None and with a backend pointing at a non-existent DB file (both no-ops). - `_bump_persisted_evidence` with no backend, missing DB, and unknown memory id (all silent no-ops so the proxy hot path never blows up on malformed state). - `stop()` cancelling the flush task cleanly. All new tests use the existing `_FakeBackend` + `_init_db` helpers so they exercise real SQLite paths, not mocks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garm
|
3e290b734b |
fix(learn): persist real evidence_count and bump on re-sighting
Before this change, every persisted traffic_learner row in memory.db
landed with evidence_count=1, causing two user-visible problems:
1. The live flush gate (evidence_count >= 2) filtered out every row, so
CLAUDE.md / MEMORY.md never received the patterns the learner saw
repeatedly.
2. _saved_hashes is in-memory only and reset on each proxy restart, so
a pattern seen once in session A then twice in session B would insert
a *duplicate* DB row instead of bumping the existing one. Users
accumulated many rows stuck at 1 instead of a few rows with high
evidence.
Root cause chain:
- _accumulate tracks a running count in the _pattern_counts tuple but
enqueues the ExtractedPattern dataclass with its default
evidence_count=1 intact.
- _save_worker writes pattern.evidence_count into metadata verbatim.
- After save, the hash goes into _saved_hashes and further sightings
are early-returned — never bumped.
- Next process start has empty _saved_hashes, so the same content goes
through the accumulator as fresh and gets re-saved.
Fix:
- _accumulate now sets pattern.evidence_count = count before enqueuing,
so DB rows reflect the real number of sightings at save time.
- _save_worker captures the Memory.id returned by save_memory and
records content_hash → id in a new _persisted_ids map.
- _accumulate's saved-hash branch now awaits
_bump_persisted_evidence(memory_id), which runs an atomic
json_set('$.evidence_count', existing + 1) UPDATE via
asyncio.to_thread to keep the proxy hot path non-blocking.
- start() calls a new _hydrate_persisted_state() that reads existing
traffic_learner rows' (id, content) pairs from the DB and pre-seeds
_saved_hashes + _persisted_ids. Cross-session re-sightings bump the
seeded row instead of inserting a duplicate.
- _load_persisted_patterns_from_sqlite and _hydrate_persisted_state
query by json_extract(metadata, '$.source') = 'traffic_learner'
instead of the prior LIKE on raw JSON — the bump path uses json_set,
which rewrites the metadata string without the default ": " spacing,
which would otherwise make the LIKE blind to bumped rows.
Adds TestEvidencePersistence with three cases:
- save persists the actual accumulated count (not the default 1)
- re-sightings bump the persisted row instead of creating duplicates
- a fresh learner hydrates _saved_hashes from DB, so cross-session
re-sightings bump the pre-existing row
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Garm
|
d9138a3ed8 |
feat(learn): live flush of traffic patterns to agent-native context files
Replaces the previous shutdown-only flush with a debounced, near-real-time dirty-flag flush worker that writes patterns into the correct CLAUDE.md / MEMORY.md bucket as traffic accumulates. - New FLUSH_DEBOUNCE_SECONDS gate (10s) prevents context-file thrash on bursty traffic while keeping updates "live" from the user's perspective. - TrafficLearner.start() now spawns a _flush_worker alongside the save worker; _accumulate() sets a dirty flag; _flush_worker() calls flush_to_file() when dirty and past the debounce window. - flush_to_file() now reads *both* persisted rows (memory.db) and the in-memory accumulator via _load_persisted_patterns_from_sqlite and _collect_all_patterns, so patterns survive proxy restarts and the agent-native files converge toward the full learned set. - Patterns are bucketed per-project via the learn plugin registry (plugin.discover_projects()) and anchored to project roots through longest-matching-path on content or entity_refs (_project_for_pattern). Un-anchored patterns are dropped. - Patterns are routed by PatternCategory to either CONTEXT_FILE (CLAUDE.md) or MEMORY_FILE (MEMORY.md) via _patterns_to_recommendations + _CATEGORY_TO_TARGET. - Live flushes require evidence_count >= 2; shutdown flushes accept single-evidence rows to avoid losing last-session signal. Adds tests for project routing, persisted-pattern loading, category routing, and the debounced flush worker. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
chopratejas
|
d9cc4f3991 | Fix ruff lint errors in test files | ||
|
Tejas Chopra
|
0fd6dfcadb |
feat: add live traffic learning + cross-agent memory writers (--learn flag)
Live Traffic Learner extracts patterns from proxy traffic in real-time: - Error→recovery patterns (tool fails → next success teaches right approach) - Environment facts (working venv paths, test commands) - User preference signals (corrections, repeated choices) Agent-native memory writers export learned patterns to each agent's format: - Claude Code: MEMORY.md + per-topic files - Cursor: .cursor/rules/headroom-memory.mdc (YAML frontmatter) - Codex: AGENTS.md - Generic: plain markdown (Aider, Gemini, any agent) Memory Budget Manager handles token-optimized memory files: - Per-agent token budgets (2K Claude, 3K Cursor/Codex) - Temporal decay, staleness detection (git + filesystem) - Jaccard-similarity memory merging, dedup Opt-in via --learn flag on proxy/wrap commands: - headroom proxy --learn - headroom wrap claude --learn - --learn implies --memory; --no-learn overrides - compress() API completely unaffected (pure function) - Default behavior unchanged (no memory, no learning) |