mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
13 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
gglucass
|
c5563d3a7d
|
fix(learn): include stdout in CLI failure messages, not just stderr (#3080)
## Description `headroom learn` reports CLI backend failures using **stderr only**. `claude -p --output-format stream-json --verbose` writes *nothing* to stderr when the run fails at the API layer, so the failure a user actually sees is a message that stops at the colon: ```text LLM analysis failed: `claude -p --output-format stream-json --verbose` failed (exit 1): ``` The reason is not missing, it is discarded. Claude Code still emits a final `result` event on stdout whose `result` field is the human-readable cause, and the streaming path has already parsed it into `final_result` one line above the `raise`. This makes a whole class of failures undiagnosable for users and maintainers alike: a usage limit, an unreachable local proxy, and an expired login all render identically as an empty message. Reported by a desktop user who could only tell us "sometimes i have this LLM analysis failed" with nothing after the colon. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Add `_failure_detail(stderr, stdout, *, result_text=None)` in `headroom/learn/analyzer.py`. Prefers the already-parsed `result` text, falls back to the **tail** of stdout (CLI backends emit the error last, after their whole event log), keeps stderr when present, and returns `"(no output captured)"` so the message is never a dangling colon. - Use it in `_call_claude_cli_streaming` (streaming claude-cli path) and in `_call_cli_llm` (the `subprocess.run` backends, gemini-cli / codex-cli), so the same blind spot is closed for every CLI backend rather than only the one that was reported. - Existing truncation behaviour is unchanged: each stream is still capped at `_MAX_SNIPPET_LEN`. Complements #3016, which makes an analysis failure propagate instead of being swallowed as success; that PR fixes *whether* the user learns a failure happened, this one fixes *what* the failure says. No overlapping lines. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_learn/ -q 247 passed, 4 skipped in 27.08s $ uvx ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! $ uvx ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py 2 files already formatted $ uv run --frozen --extra dev mypy headroom/learn/analyzer.py Success: no issues found in 1 source file ``` New tests: `test_claude_cli_nonzero_exit_includes_api_error_from_stdout`, `test_claude_cli_nonzero_exit_with_no_output_says_so`, `test_claude_cli_nonzero_exit_keeps_stderr_when_present`, `test_codex_nonzero_exit_includes_stdout_when_stderr_empty`. ## Real Behavior Proof - Environment: macOS 15.6 (Darwin 24.6.0), Claude Code 2.1.228, Python 3.10.18, headroom on this branch. - Exact command / steps: forced an API-layer failure in the exact command the analyzer runs, capturing the streams separately: `echo "say hi" | claude -p --output-format stream-json --verbose --settings '{"env":{"ANTHROPIC_BASE_URL":"http://127.0.0.1:9"}}' > out.txt 2> err.txt; echo "EXIT=$?"; wc -c err.txt; tail -c 400 out.txt` - Observed result: `EXIT=1`, `err.txt` is **0 bytes**, and the reason appears only in the last stdout line: `"terminal_reason":"api_error", ..., "result":"API Error: Connection refused — a firewall or proxy may be blocking it (ConnectionRefused)"`. A second run with `--bare` produced the same shape with `"result":"Not logged in · Please run /login"`. Before this change both surface as `failed (exit 1):` with nothing after the colon; after it, the `result` text is in the message. The unit tests encode this exact stream shape (stdout `result` event, empty stderr, exit 1). - Not tested: real usage-limit and 429 responses, which I cannot provoke on demand. They travel the same code path as the reproduced `api_error` case (final `result` event on stdout, empty stderr), so they are covered by construction rather than by observation. Windows and the gemini-cli backend were not exercised manually; the shared helper is covered by unit tests for both the streaming and `subprocess.run` paths. ## Runtime Rollout Safety - Rollout-managed feature(s): None. This touches only the error text raised by `headroom learn`'s CLI backends; no rollout-gated feature, flag, or runtime component is involved. - Minimum rollout channel: N/A, not rollout-gated. Ships with the package like any other library fix. - Stable/default behavior changed: Yes, narrowly. The message text of an existing `RuntimeError` on a non-zero CLI exit now includes the stdout/`result` reason alongside stderr. No control flow, exit code, public API, or return value changes: the same exception is raised in the same cases. - Kill switch / disable path: None needed. Nothing is enabled or newly executed, so there is nothing to switch off; the only behavioral surface is the string inside an exception that was already being raised. - Unsafe override required: No. - Qualification impact: None. No qualification-gated path, model, or provider behavior is touched. Callers that pattern-match this message on `"failed (exit N)"` still match, since that prefix is unchanged. - Rollback path: Revert this commit. The previous stderr-only message returns with no migration, state, or config to undo. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
TenderDeve
|
85e8699451
|
fix(learn): keep traceback tail in tool-error digest preview (#2596)
## Description `_format_tool_call` in `headroom/learn/analyzer.py` built the error preview with a head-only slice — `tc.output[:200]`. For tracebacks the root cause (`ExceptionType: message`) is at the **tail**, so the digest showed only `Traceback (most recent call last):` plus the first frame and dropped the actual diagnosis. The issue reports 46% of 715 measured errors were truncated past the 200-char head. Closes #2590 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `_truncate_head_tail()` helper that collapses newlines and, when over budget, keeps both the head and the tail joined by `…`. - `_format_tool_call` now uses it for error output so the exception line survives truncation. Short errors are returned unchanged (no marker). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text $ uv run pytest tests/test_learn/test_analyzer.py::TestDigestBuilder -q 9 passed in 1.47s $ uv run ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! ``` ## Real Behavior Proof - Environment: headroom @ main, Python 3.14, uv - Exact command / steps: added a long synthetic traceback (`KeyError: 'the-actual-root-cause'` at the tail) as a failing tool call and built the digest. - Observed result: digest now contains both `Traceback` and `KeyError: 'the-actual-root-cause'`, separated by `…`; short errors have no `…`. - Not tested: mypy not run locally. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Truncation budget stays at 200 chars (now split head/tail). mypy not run locally; happy to adjust if CI flags anything. |
||
|
Fabien Culpo
|
d2170b1922
|
fix(learn): parse fenced JSON even with a prose preamble (#1988)
## Description `_strip_fenced_json` only stripped a markdown fence when the string *started with* ```` ``` ````. When the model prefixed prose before the fence (e.g. `Here is the JSON:\n\n```json ...`) despite being told to return JSON only, the guard was skipped and `json.loads` ran on the prose, raising `JSONDecodeError`. The claude-cli streaming path surfaced this as `returned unparseable output`, and `headroom learn` silently discarded the LLM analysis, degrading to "No actionable patterns found". This is the parsing-side cousin of the silent-degradation issue fixed in #373. Closes #1989. Related: #373. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/learn/analyzer.py`: rewrote `_strip_fenced_json` to locate the fenced block wherever it appears, then fall back to the whole text, then to a first-`{` / last-`}` slice, only re-raising `JSONDecodeError` if nothing parses as a JSON object. Preserves the prior "first opening / last closing fence" behaviour and triple-backtick content inside the payload. Fixes all three call sites (non-streaming CLI, claude-cli streaming, litellm). - `tests/test_learn/test_analyzer.py`: added regression cases to `TestStripFencedJson` for preamble-before-fence, prose around a bare object, and triple-backticks inside the payload. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) — scoped to the changed module (see Additional Notes) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_learn/test_analyzer.py -q ........................................................................ [ 86%] ........... [100%] 83 passed, 1 warning in 2.18s $ ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! $ ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py 2 files already formatted $ mypy --ignore-missing-imports --follow-imports=silent headroom/learn/analyzer.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS, Python 3.13.12, headroom-ai at this branch (runtime deps from an installed 0.30.0 env). - Exact command / steps: ran the old vs new `_strip_fenced_json` on the exact failing model output (a prose preamble followed by a ```json fence), then applied the fix over an installed 0.30.0 and re-ran the previously failing `headroom learn --apply`. Input sample: `'The JSON is my deliverable for this analysis task. Here it is:\n\n```json\n{"context_file_rules": [], "memory_file_rules": []}\n```'` - Observed result: OLD raised `JSONDecodeError: Expecting value: line 1 column 1 (char 0)`; NEW returned `{'context_file_rules': [], 'memory_file_rules': []}`. The real `headroom learn --apply` run that had been failing with `returned unparseable output` then completed and consumed the LLM analysis instead of dropping it. Full transcript: ```text OLD: JSONDecodeError -> Expecting value: line 1 column 1 (char 0) NEW: {'context_file_rules': [], 'memory_file_rules': []} ``` - Not tested: full end-to-end `headroom learn --apply` was not re-run inside CI here (it shells out to a live `claude` CLI); the parser is exercised deterministically by the added unit tests and the before/after repro above. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A — updated the function docstring only; no external docs affected) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (N/A — no CHANGELOG entry convention observed for this fix; happy to add if maintainers prefer) ## Additional Notes - `mypy` was run against the changed module in isolation (`--ignore-missing-imports --follow-imports=silent`) rather than the full project, because I validated in an ad-hoc environment; the change keeps the existing `-> dict` signature and annotations, so it is type-neutral. - Not addressed here (possible follow-up): the failure is swallowed as a warning in `analyze()`, so users only see "No actionable patterns found" with no signal the LLM pass produced nothing — the same silent-degradation class as #373, on the parsing side. |
||
|
Rod Boev
|
e3b45e402b
|
fix(learn): handle Windows UTF-8, drive-letter paths, and CLI shim fallback (#1895)
## Description `headroom learn --verbosity` is broken on Windows in three related ways: - Transcript/profile reads can use the platform default codec, so non-ASCII content can raise `UnicodeDecodeError` and collapse learning signals to empty output. - `--project <path>` can miss real Claude project directories because Windows profile junctions can raise `PermissionError` during directory walks, and escaped Claude project folder names cannot always distinguish `vibe-remote` from `vibe\remote`. - `headroom learn --agent codex` can fail with `` `claude` not found in PATH `` even when the npm-installed CLI exists, because Windows `.cmd` shims require `PATHEXT` resolution. Refs https://github.com/headroomlabs-ai/headroom/issues/1624 for the Windows learn failures. The dashboard-hint UX and third-party-provider-auth items in that issue are unrelated and out of scope for this PR. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/learn/verbosity.py`: read and write verbosity transcripts/profiles with `encoding="utf-8"` so non-ASCII content works regardless of the Windows locale codec. - `headroom/learn/plugins/claude.py`: skip inaccessible siblings one entry at a time during greedy project path decoding, so one Windows junction no longer hides valid project directories. - `headroom/learn/plugins/claude.py`: prefer a valid `cwd` found in Claude session JSONL when discovering project paths, which resolves ambiguous escaped folder names such as `vibe-remote` versus `vibe\remote`. - `headroom/learn/analyzer.py`: resolve Windows CLI shim paths through `shutil.which()` after `FileNotFoundError`, then retry once for streaming and non-streaming CLI calls. - `CHANGELOG.md`: document the Windows learn fixes under `Unreleased`. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_verbosity_learn.py::TestWindowsEncoding tests/test_learn/test_scanner.py::TestGreedyPathDecode::test_permission_denied_sibling_does_not_abort_the_walk tests/test_learn/test_analyzer.py::TestWindowsCliShimFallback tests/test_learn/test_scanner.py::TestDecodeProjectPath::test_discover_project_prefers_session_cwd_over_ambiguous_folder_name -q`) - [x] Linting passes (`uv run ruff check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py`) - [x] Formatting passes (`uv run ruff format --check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py`) - [ ] Type checking passes (`uv run mypy headroom`) not run; no new public type surface - [x] New tests added for the Windows `cwd` disambiguation regression - [x] Manual testing performed ### Test Output ```text uv run ruff format headroom/learn/plugins/claude.py 1 file reformatted uv run ruff check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py All checks passed! uv run ruff format --check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py 2 files already formatted uv run pytest tests/test_verbosity_learn.py::TestWindowsEncoding tests/test_learn/test_scanner.py::TestGreedyPathDecode::test_permission_denied_sibling_does_not_abort_the_walk tests/test_learn/test_analyzer.py::TestWindowsCliShimFallback tests/test_learn/test_scanner.py::TestDecodeProjectPath::test_discover_project_prefers_session_cwd_over_ambiguous_folder_name -q 9 passed in 0.25s ``` CI on current head ` |
||
|
gglucass
|
9bff5752bb
|
fix(learn): claude-cli streams output with idle timeout (#373)
## Description `headroom learn` with the claude-cli backend used `subprocess.run` with a hard 120s wall-clock cap and no liveness signal. A successful long analysis and a hung connection looked identical — exit 0 with "0 recommendations" was the only user-visible signal when the LLM call timed out, which silently hides genuine learnings. This PR makes the CLI backend timeout-aware, with progress detection for claude-cli and configurable wall-clock caps for every backend. Fixes #(issue number) ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Streaming claude-cli with idle timeout**: invoke `claude -p --output-format stream-json --verbose` and run a watchdog loop that drains stdout/stderr via reader threads. Each stream-json event resets an idle deadline. Kill the process if no output for `HEADROOM_LEARN_CLI_IDLE_TIMEOUT_SECS` (default 60s) or if total elapsed exceeds `HEADROOM_LEARN_CLI_TIMEOUT_SECS` (default 300s, was 120s). The final `type:"result"` event carries the assistant response, which is then parsed as JSON. Reader threads (rather than `select`) are used so the watchdog works on Windows where `select` does not support pipe handles. - **Bumped default `_CLI_TIMEOUT` from 120s to 300s** as the hard cap for all CLI backends. The previous 120s was too tight for large digests on slower networks. - **Env-var overrides** via new helper `_resolve_timeout_secs(env_var, default)`: - `HEADROOM_LEARN_CLI_TIMEOUT_SECS` — hard wall-clock cap (all CLI backends) - `HEADROOM_LEARN_CLI_IDLE_TIMEOUT_SECS` — idle cap (streaming claude-cli only) - Non-positive or non-integer values log a warning and fall back to defaults, so a typo can't disable the timeout. - **gemini-cli and codex-cli** keep `subprocess.run(timeout=hard_cap)` since they do not emit progress events. They benefit from the bumped default and the env-var override. - **CHANGELOG.md** updated under `[Unreleased]` → `### Fixed`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed (existing repro: 16k-call digest that previously timed out at 120s) New test coverage in `tests/test_learn/test_analyzer.py`: - `test_claude_cli_streams_and_parses_result_event` — happy path, fake Popen yields system/assistant/result events - `test_claude_cli_parses_fenced_result` — markdown fences in the result event still parse - `test_claude_cli_idle_timeout_kills_hang` — `HEADROOM_LEARN_CLI_IDLE_TIMEOUT_SECS=1` + a hanging stdout iterator triggers the idle watchdog - `test_claude_cli_hard_cap_kills_continuous_chatter` — continuous events with a low hard cap fire the wall-clock kill (proves idle reset alone can't keep a runaway alive) - `test_claude_cli_missing_result_event_raises` — graceful failure when no `result` event is emitted - `test_claude_cli_nonzero_exit_raises` / `test_claude_cli_unparseable_result_raises_with_context` / `test_claude_cli_not_installed_raises` — error paths - Parallel codex-cli error coverage (timeout-honors-env-override included) so the wall-clock path is exercised - `TestResolveTimeoutSecs` — unset / empty / non-integer / non-positive / valid override ## Test Output ``` $ uv run pytest tests/test_learn/test_analyzer.py ============================== 67 passed in 2.14s ============================== $ uv run ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! $ uv run ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py 2 files already formatted $ uv run mypy headroom/learn/analyzer.py Success: no issues found in 1 source file ``` ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The contract assumed for claude-cli stream-json output is: each line is a JSON object with a `type` field; the final event has `type:"result"` with a string `result` field carrying the assistant text. This matches the documented Anthropic CLI behavior. If the contract changes upstream, `_call_claude_cli_streaming` raises a clear "did not emit a final \`result\` event" error rather than silently succeeding. - Backwards-compatible for users without env-var configuration: behavior just becomes "longer hard cap, plus idle watchdog for claude-cli", neither of which can falsely succeed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Evan Alferez
|
d7973665f4 |
fix(learn): finish gemini-flash-latest default model sweep (#532)
Google deprecated gemini/gemini-2.0-flash; headroom learn silently fails when GEMINI_API_KEY is set. PR #532 updated the default in analyzer.py but left stale references in the CLI help text and unit test assertion. |
||
|
Garm
|
35073ccb23 |
style(learn): ruff format test_analyzer.py
Line-length wrapping only. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garm
|
6d2aba8741 |
fix(learn): show prior patterns block to LLM to prevent dangling refs
When `headroom learn` re-surfaced a section heading that already existed in CLAUDE.md / MEMORY.md, the writer replaced that section wholesale — but the LLM never saw the prior block, so it emitted condensed bullets like "X is *also* large — same rule as Y, Z" assuming Y and Z would remain siblings. After replacement, Y and Z were gone and the "also" dangled. This threads the project's current `<!-- headroom:learn -->` block (from both CLAUDE.md and MEMORY.md) into the digest as a "Prior Learned Patterns" section, and extends the system prompt to make the re-emission contract explicit: re-stating a section replaces it wholesale, so the LLM must copy forward prior bullets it still agrees with. Prior sections the LLM omits entirely are still carried forward by the writer (#231 behavior preserved as a safety net). Changes: - New `extract_marker_block(file_content)` helper in `learn.writer` that returns the raw marker block (delimiters included) or None. - New `_build_prior_patterns_section(project)` in `learn.analyzer` reads `project.context_file` and `project.memory_file` via the new helper and formats a labeled section ahead of the per-session event stream. - `_build_digest` emits the prior-patterns section when present; char budget accounting unchanged (prior blocks are small). - `_SYSTEM_PROMPT` gains a "Prior Learned Patterns" rule block telling the LLM how to integrate prior bullets (preserve / revise / drop-only- if-contradicted) and warning against unresolved cross-references. - Tests: 6 new `TestPriorPatternsInjection` cases (present/absent files, no-marker-block, both-files, end-to-end via mocked `_call_llm`); 4 new `TestExtractMarkerBlock` cases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Gyeonghun Park
|
c3cf022886 |
fix(learn): handle FileNotFoundError when CLI tool is not installed
When --model codex-cli is used but codex is not in PATH, subprocess.run raises FileNotFoundError. Catch it and raise a clear RuntimeError with guidance. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Gyeonghun Park
|
e98c975153 |
feat(learn): add CLI-based LLM backends for keyless headroom learn
Allow `headroom learn` to use locally installed coding agent CLIs (claude, gemini, codex) as LLM backends, so subscription users without raw API keys can run failure analysis. Priority: --model flag > API key > HEADROOM_LEARN_CLI env var > auto-detect - Pass prompts via stdin to avoid ARG_MAX limits - Handle TimeoutExpired, truncate stderr, enrich JSONDecodeError - Add 31 new tests (48 total), all passing - Update docs/learn.md with CLI backend documentation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Tejas Chopra
|
da481a359b |
fix(learn): pass explicit model in tests to avoid API key requirement
SessionAnalyzer() without a model calls _detect_default_model() which raises when no API keys are set (e.g., in CI). Pass model="test-model" in the three tests that mock _call_llm. |
||
|
Tejas Chopra
|
4d14012c2f |
feat: add headroom perf CLI and rewrite headroom learn to use LLM analysis
Proxy performance logging (`headroom perf`): - Add always-on RotatingFileHandler to ~/.headroom/logs/proxy.log (10MB x 5 backups) - Replace scattered log lines with structured PERF lines containing model, msgs, tok_before/after/saved, cache_read/write/hit_pct, opt_ms, and transforms - Emit PERF lines from all three response paths (streaming Anthropic, non-streaming Anthropic, Bedrock streaming) - Add `headroom perf` CLI that parses proxy logs and reports token savings, cache hit rates, prefix stability, transform effectiveness, routing breakdown, TOIN status, and actionable recommendations - Support --hours and --raw flags for time filtering and raw record output Learn module rewrite (LLM-based analysis): - Replace all regex/heuristic analyzers with a single LLM call via LiteLLM - New SessionAnalyzer builds compact digests and sends to any of 100+ models - Auto-detect best model from API keys (Anthropic → OpenAI → Gemini) - Add --model flag for explicit model selection - Enrich scanner with SessionEvent (user messages, interruptions, subagent summaries), token usage tracking, and timestamps - Simplify models: remove EnvironmentFact, StructureNote, Correction, CommandPattern, RetryPattern, AnalysisReport; add SessionEvent, AnalysisResult - Simplify writer: remove Recommender class (LLM now produces recommendations directly) - Update tests for new analyzer and models |
||
|
chopratejas
|
17442c2dcc |
Add headroom learn: offline failure learning for coding agents
Analyzes past conversation history to find tool call failure patterns, correlates each failure with what eventually succeeded, and writes specific project-level learnings to CLAUDE.md and MEMORY.md. Key design: - Success correlation: extracts the diff between failed and successful inputs as the learning (not generic advice) - Generic architecture: tool-agnostic ToolCall model with pluggable Scanner/Writer adapters (Claude Code first, extensible to Cursor/Codex) - 5 analyzers: Environment, Structure, Commands, Retries, Cross-Session - Dry-run by default, --apply to write, --all for all projects Also fixes mypy errors in litellm_callback, asgi, langchain chat_model, and anthropic provider (AsyncClient typing, ToolCall arg-type, int cast). |