diff --git a/README.md b/README.md index 19807d4f8..8ee4a38c1 100644 --- a/README.md +++ b/README.md @@ -130,15 +130,16 @@ Works with any language, any tool, any framework. **[Proxy docs](docs/proxy.md)* ### Coding agents — one command ```bash -headroom wrap claude # Starts proxy + launches Claude Code -headroom wrap codex # Starts proxy + launches OpenAI Codex CLI -headroom wrap aider # Starts proxy + launches Aider -headroom wrap cursor # Starts proxy + prints Cursor config -headroom wrap openclaw # Installs + configures OpenClaw plugin -headroom unwrap openclaw # Disables plugin + restores legacy engine +headroom wrap claude # Starts proxy + launches Claude Code +headroom wrap codex # Starts proxy + launches OpenAI Codex CLI +headroom wrap aider # Starts proxy + launches Aider +headroom wrap cursor # Starts proxy + prints Cursor config +headroom wrap openclaw # Installs + configures OpenClaw plugin +headroom wrap claude --memory # With persistent cross-agent memory +headroom wrap codex --memory # Shares the same memory store ``` -Headroom starts a proxy, points your tool at it, and compresses everything automatically. +Headroom starts a proxy, points your tool at it, and compresses everything automatically. Add `--memory` for persistent memory that's shared across agents. ### Multi-agent — SharedContext @@ -270,15 +271,24 @@ Auto-detects what's in your context — JSON arrays, code, logs, plain text — Stabilizes message prefixes so your provider's KV cache actually works. Claude offers a 90% read discount on cached prefixes — but almost no framework takes advantage of it. Headroom does. +### Cross-Agent Memory + +```bash +headroom wrap claude --memory # Claude with persistent memory +headroom wrap codex --memory # Codex shares the SAME memory store +``` + +Claude saves a fact, Codex reads it back. All agents sharing one proxy share one memory — project-scoped, user-isolated, with agent provenance tracking and automatic deduplication. No SDK changes needed. **[Memory docs](docs/memory.md)** + ### Failure Learning ```bash -headroom learn # Analyze past Claude Code sessions, show recommendations -headroom learn --apply # Write learnings to CLAUDE.md and MEMORY.md -headroom learn --all --apply # Learn across all your projects +headroom learn # Auto-detect agent (Claude, Codex, Gemini) +headroom learn --apply # Write learnings to agent-native files +headroom learn --agent codex --all # Analyze all Codex sessions ``` -Reads your conversation history, finds every failed tool call, correlates it with what eventually succeeded, and writes specific corrections into your project files. Next session starts smarter. **[Learn docs](docs/learn.md)** +Plugin-based: reads conversation history from Claude Code, Codex, or Gemini CLI. Finds failure patterns, correlates with successes, writes corrections to CLAUDE.md / AGENTS.md / GEMINI.md. External plugins via entry points. **[Learn docs](docs/learn.md)**

headroom learn demo @@ -302,10 +312,10 @@ Reads your conversation history, finds every failed tool call, correlates it wit | **CacheAligner** | Stabilizes prefixes for provider KV cache hits | | **IntelligentContext** | Score-based context management with learned importance | | **Image Compression** | 40-90% token reduction via trained ML router | -| **Memory** | Persistent memory across conversations | +| **Memory** | Cross-agent persistent memory — Claude saves, Codex reads it back. Agent provenance + auto-dedup | | **Compression Hooks** | Customize compression with pre/post hooks | | **Read Lifecycle** | Detects stale/superseded Read outputs, replaces with CCR markers | -| **`headroom learn`** | Analyzes past failures, writes project-specific learnings to CLAUDE.md/MEMORY.md | +| **`headroom learn`** | Plugin-based failure learning for Claude Code, Codex, Gemini CLI (extensible via entry points) | | **`headroom wrap`** | One-command setup for Claude Code, Codex, Aider, Cursor | | **SharedContext** | Compressed inter-agent context sharing for multi-agent workflows | | **MCP Tools** | headroom_compress, headroom_retrieve, headroom_stats for Claude Code/Cursor | @@ -480,11 +490,11 @@ Python 3.10+ | [Latency Benchmarks](docs/LATENCY_BENCHMARKS.md) | Compression overhead & cost-benefit analysis | | [Limitations](docs/LIMITATIONS.md) | When compression helps, when it doesn't | | [Evals Framework](headroom/evals/README.md) | Prove compression preserves accuracy | -| [Memory](docs/memory.md) | Persistent memory | +| [Memory](docs/memory.md) | Cross-agent persistent memory with provenance + dedup | | [Agno](docs/agno.md) | Agno agent framework | | [MCP](docs/mcp.md) | Context engineering toolkit (compress, retrieve, stats) | | [SharedContext](docs/shared-context.md) | Compressed inter-agent context sharing | -| [Learn](docs/learn.md) | Offline failure learning for coding agents | +| [Learn](docs/learn.md) | Plugin-based failure learning (Claude, Codex, Gemini, extensible) | | [Configuration](docs/configuration.md) | All options | --- diff --git a/docs/learn.md b/docs/learn.md index dce072a81..ce5b1b028 100644 --- a/docs/learn.md +++ b/docs/learn.md @@ -21,16 +21,18 @@ headroom learn --all --apply ## How It Works ``` -Past Sessions → Scanner → Analyzer → Writer → CLAUDE.md / MEMORY.md +Past Sessions → Plugin → Analyzer → Writer → Agent-native context file │ │ │ │ │ └─ Writes marker-delimited sections │ │ (replaced on re-run, not duplicated) │ │ - │ └─ Success Correlation: for each failure, - │ finds what succeeded and extracts the diff + │ └─ LLM-based analysis: finds failure patterns, + │ success correlations, and actionable rules │ - └─ Reads ~/.claude/projects/*.jsonl - (extensible to Cursor, Codex, etc.) + └─ Plugin reads agent-specific logs: + • Claude Code: ~/.claude/projects/*.jsonl + • Codex: ~/.codex/sessions/*.json + • Gemini CLI: ~/.gemini/tmp/*/chats/session-*.json ``` ### Success Correlation @@ -95,12 +97,12 @@ Commands repeatedly rejected — model should suggest them to the user instead. ## Where Learnings Go -| Pattern | Destination | Why | -|---------|-------------|-----| -| Environment, paths, search scope, commands, large files | **CLAUDE.md** | Stable project facts, version-controllable | -| Missing paths, retry patterns, permissions | **MEMORY.md** | May change, agent-specific | +| Pattern | Claude Code | Codex | Gemini CLI | +|---------|-------------|-------|-----------| +| Environment, paths, commands | **CLAUDE.md** | **AGENTS.md** | **GEMINI.md** | +| Retry patterns, permissions | **MEMORY.md** | **instructions.md** | **GEMINI.md** | -CLAUDE.md lives in your project directory. MEMORY.md lives in `~/.claude/projects/*/memory/`. +Output files are agent-native: Claude Code uses CLAUDE.md/MEMORY.md, Codex uses AGENTS.md, Gemini uses GEMINI.md. The same learnings, written to the format each agent reads. ## Marker-Based Updates @@ -116,25 +118,30 @@ Headroom manages a clearly-delimited section in each file: On re-run, only the content between markers is replaced. Your existing file content is preserved. -## Architecture +## Architecture (Plugin System) + +Headroom Learn uses a plugin architecture where each agent is a self-contained plugin: ``` -Scanner (adapter) → Analyzer (generic) → Writer (adapter) -├── ClaudeCodeScanner ├── EnvironmentAnalyzer ├── ClaudeCodeWriter -├── (CursorScanner) ├── StructureAnalyzer ├── (CursorWriter) -└── (GenericScanner) ├── CommandAnalyzer └── (GenericWriter) - ├── RetryAnalyzer - └── CrossSessionAnalyzer +Plugin Registry (auto-discovered) +├── ClaudeCodePlugin → Analyzer (LLM) → ClaudeCodeWriter → CLAUDE.md / MEMORY.md +├── CodexPlugin → Analyzer (LLM) → CodexWriter → AGENTS.md / instructions.md +├── GeminiPlugin → Analyzer (LLM) → GeminiWriter → GEMINI.md +└── (your plugin) → Analyzer (LLM) → (your writer) → (your file) ``` -**Scanners** read tool-specific log formats and produce normalized `ToolCall` sequences. -**Analyzers** work on `ToolCall` — same analysis for any agent system. -**Writers** output to tool-specific context injection mechanisms. +**Plugins** bundle scanning, detection, and writing for one agent. Built-in plugins are auto-discovered from `headroom.learn.plugins.*`. External plugins register via the `headroom.learn_plugin` entry point. -To add support for a new agent (e.g., Cursor): -1. Write `CursorScanner(ConversationScanner)` — reads Cursor's log format -2. Write `CursorWriter(ContextWriter)` — writes to `.cursorrules` -3. Same analyzers, same models, same recommendations +**The Analyzer** is shared — it uses an LLM (Sonnet, GPT-4o, or Gemini Flash) to find patterns. Same analysis for any agent. + +### Adding Support for a New Agent + +1. Create `headroom/learn/plugins/myagent.py` +2. Implement `LearnPlugin` + `ConversationScanner` (scanner + writer + detection) +3. Add `plugin = MyAgentPlugin()` at module scope +4. Done — `headroom learn --agent myagent` works automatically + +Or install an external plugin: `pip install headroom-learn-cursor` (registers via entry point). ## CLI Reference @@ -142,12 +149,22 @@ To add support for a new agent (e.g., Cursor): headroom learn [OPTIONS] Options: - --project PATH Project directory to analyze (default: current directory) - --all Analyze all discovered projects - --apply Write recommendations (default: dry-run) - --claude-dir PATH Path to .claude directory (default: ~/.claude) + --project PATH Project directory (default: current directory) + --all Analyze all discovered projects + --apply Write recommendations (default: dry-run) + --agent [auto|claude|codex|gemini] + Which agent to analyze (default: auto-detect) + --model TEXT LLM for analysis (default: auto from API keys) ``` +### Supported Agents + +| Agent | Scanner | Writer | Output Files | +|-------|---------|--------|-------------| +| **Claude Code** | Reads `~/.claude/projects/*.jsonl` | ClaudeCodeWriter | CLAUDE.md, MEMORY.md | +| **OpenAI Codex** | Reads `~/.codex/sessions/*.json` | CodexWriter | AGENTS.md, instructions.md | +| **Gemini CLI** | Reads `~/.gemini/tmp/*/chats/session-*.json` | GeminiWriter | GEMINI.md | + ## Real-World Results Tested on 67,583 tool calls across 23 projects: diff --git a/docs/memory.md b/docs/memory.md index 717cb0b84..ce440a726 100644 --- a/docs/memory.md +++ b/docs/memory.md @@ -18,6 +18,10 @@ This is *temporal compression* - instead of carrying 10,000 tokens of conversati | Feature | Headroom | Letta (MemGPT) | Mem0 | |---------|----------|----------------|------| +| **Cross-Agent Memory** | Any agent shares one DB via proxy | Per-agent only | Per-user, no cross-agent | +| **Agent Provenance** | Tracks which agent saved/updated each memory | No | No | +| **LLM-Mediated Dedup** | Piggybacks on user's own LLM for merge decisions | No | Separate LLM call ($) | +| **Transparent Proxy** | Zero code changes — just route through proxy | Requires agent framework | Requires SDK integration | | **Hierarchical Scoping** | User → Session → Agent → Turn | Flat (per-agent) | Flat (per-user) | | **Temporal Versioning** | Full supersession chains | No | No | | **Zero-Latency Extraction** | Inline (Letta-style) | Inline | Separate call | @@ -29,6 +33,112 @@ This is *temporal compression* - instead of carrying 10,000 tokens of conversati --- +## Cross-Agent Memory (Proxy) + +The most powerful way to use memory: **any agent that routes through the proxy shares the same memory store.** Claude saves a fact, Codex reads it back. Zero configuration needed. + +```bash +# Start the proxy with memory enabled +headroom proxy --memory + +# Or use wrap (auto-starts proxy) +headroom wrap claude --memory # Claude Code with persistent memory +headroom wrap codex --memory # Codex with the SAME memory store +headroom wrap aider --memory # Aider shares it too +``` + +### How It Works + +``` +Claude Code Codex CLI Gemini CLI + │ │ │ + └── /v1/messages ──┐ └── /v1/chat/completions ──┤ └── /generateContent ──┐ + │ │ │ + ▼ ▼ ▼ + ┌──────────────────────────────────────────────────────────────────┐ + │ Headroom Proxy (--memory) │ + │ │ + │ 1. Search memory DB for relevant context │ + │ 2. Inject memories as system context (provider-native format) │ + │ 3. Add memory_save/search/update/delete tools │ + │ 4. Forward to upstream LLM │ + │ 5. Handle memory tool calls in response │ + │ 6. Async background dedup (>92% cosine → auto-remove) │ + │ │ + └──────────────────────┬───────────────────────────────────────────┘ + │ + ▼ + .headroom/memory.db + (project-scoped SQLite) +``` + +### Project-Scoped Database + +Memory is stored per-project at `{cwd}/.headroom/memory.db`. Each project has its own memory — no cross-project contamination. Override with `--memory-db-path` for a custom location. + +### User Identity + +User ID is auto-detected from `$USER` (your OS username). Override per-request with the `x-headroom-user-id` header. All memories are scoped to the user — multiple developers on the same project have separate memory stores. + +### Agent Provenance + +Every memory tracks which agent created or updated it: + +```json +{ + "content": "Project uses alembic for migrations", + "metadata": { + "source_agent": "claude", + "source_provider": "anthropic", + "created_via": "tool_call", + "created_at_utc": "2026-04-10T17:30:00Z" + } +} +``` + +When an agent updates a memory, the update is tracked: + +```json +{ + "reason": "Updated by codex via openai: Added version info" +} +``` + +### Intelligent Deduplication + +When the LLM calls `memory_save`, headroom: + +1. **Saves immediately** (zero latency) +2. **Searches for similar existing memories** (cosine similarity) +3. **Returns an enriched hint** if duplicates found: + +```json +{ + "status": "saved", + "memory_id": "abc123", + "note": "Similar memory exists (id: def456, 89% match, saved by codex): + 'DB migration tool is alembic'. Call memory_update('def456', + '') to consolidate." +} +``` + +The LLM then decides whether to merge — using the user's own LLM, not a separate model. No extra cost to headroom. + +4. **Background auto-dedup**: If similarity >92%, the older duplicate is automatically removed (async, non-blocking). + +### Supported Providers + +Memory works with ALL providers routing through the proxy: + +| Provider | Context Injection | Memory Tools | Format | +|----------|-------------------|--------------|--------| +| **Anthropic** (Claude) | System parameter | Anthropic tool_use | Native | +| **OpenAI** (Codex, GPT) | System message | OpenAI function calling | Native | +| **Gemini** | systemInstruction | functionDeclarations | Native | +| **Any OpenAI-compatible** | System message | Function calling | OpenAI format | + +--- + ## Quick Start ```python @@ -443,6 +553,10 @@ Headroom Memory uses **Protocol interfaces** (ports) for all components, enablin | Feature | Headroom | Letta | Mem0 | |---------|:--------:|:-----:|:----:| +| Cross-agent sharing (proxy) | ✅ | ❌ | ❌ | +| Agent provenance tracking | ✅ | ❌ | ❌ | +| LLM-mediated dedup (no extra cost) | ✅ | ❌ | ❌ (uses separate LLM) | +| Transparent proxy (zero code) | ✅ | ❌ | ❌ | | Hierarchical scoping | ✅ | ❌ | ❌ | | Temporal versioning | ✅ | ❌ | ❌ | | Zero-latency extraction | ✅ | ✅ | ❌ |