- headroom wrap codex: injects rtk instructions into AGENTS.md
- headroom wrap cursor: injects into .cursorrules, prints config steps
- headroom wrap aider: injects into CONVENTIONS.md, sets both env vars
- All with --no-rtk flag to skip rtk setup
Unified savings metrics across three layers:
- cli_filtering: tokens avoided by rtk before reaching context
- compression: tokens removed by proxy (SmartCrusher, etc.)
- prefix_cache: provider cache discount with honest attribution
Fix flaky test_process_stats_collected when psutil not installed.
Fix OpenAI streaming with backends and /v1 double-path bug
Add stream_openai_message() to LiteLLM and any-llm backends so
/v1/chat/completions with stream:true returns SSE events instead
of a JSON blob. Clients (Kilo Code, Cursor, etc.) were hanging
because the proxy ignored the stream flag when routing through
a backend.
Also strip trailing /v1 from OPENAI_TARGET_API_URL to prevent
double-path URLs like /v1/v1/models.
Move heavy deps to optional extras: sentence-transformers, torch,
numpy, pillow, datasets, accelerate out of core. Remove unused deps
entirely (semantic-router, protobuf, sentencepiece).
New extras: [ml] for Kompress, [image] for image compression,
[langchain] for LangChain integration. Guard memory/image imports
so core install works without numpy/torch.
Core (tiktoken, pydantic, litellm, click, rich) gives full
compression: SmartCrusher, ContentRouter, CCR, TOIN, CLI.
LiteLLM removed claude-3-5-sonnet-20241022 from its cost database.
Add alias fallback map so retired model names resolve to current
equivalents for pricing lookups.
- CLI now reads OPENAI_TARGET_API_URL, GEMINI_TARGET_API_URL, and
HEADROOM_ANYLLM_PROVIDER environment variables
- Add --openai-api-url and --gemini-api-url CLI flags
- Remove --backend choices restriction so litellm-* backends work
- Forward tools/tool_choice through LiteLLM and any-llm backends
- Parse tool call arguments from JSON string to dict (Anthropic format)
- Forward top_p, stop_sequences, tools in streaming paths
- Update Vertex AI model map with Claude 3 through 4.6 (from official docs)
- Bump version to 0.4.1
- headroom wrap claude is the simplest way to start up claude
- It will also install rtk-ai locally
- rtk-ai is a cli wrapper that can save ~90% tokens for CLI calls made by Claude Code
Adds kompress_compressor.py — a self-contained ModernBERT-based token
compressor that auto-downloads from chopratejas/kompress-base on HuggingFace.
Trained on 330K structured tool outputs (JSON, diffs, logs, code, SQL,
agentic traces), achieving 82% entity preservation vs LLMLingua-2's 36%.
Changes:
- New: kompress_compressor.py — dual-head ModernBERT (token + span CNN)
with HuggingFace auto-download, no extra pip install needed
- ContentRouter: Kompress is primary ML compressor, LLMLingua-2 is fallback
- fallback_strategy changed from PASSTHROUGH to KOMPRESS — unknown/mixed
content now gets compressed instead of ignored
- No hardcoded compression ratios — model decides per-token importance,
optional target_ratio only when user explicitly sets it via API
- Version bump: 0.3.8 → 0.4.0
Dashboard was showing wildly incorrect metrics (99.5% savings, 3ms overhead)
due to using Anthropic API's non-cached input_tokens instead of optimized_tokens,
and dividing overhead by total request count instead of optimized-only count.
Key fixes:
- Use optimized_tokens (what we sent) for dashboard aggregation, not API's
input_tokens which excludes cached portion
- Track overhead_count separately from latency_count for correct averages
- Add TTFB (time to first byte) measurement, replace full stream latency in UI
- Eager-load LLMLingua model at proxy startup (eliminates 5.9s first-request delay)
- Simplify CostTracker to token-based accounting with counterfactual cost display
- Add two-tier compression cache to ContentRouter (skip set + result cache)
- Fix compression pinning to detect both CCR and ReadLifecycle markers
- Clamp tokens_saved to max(0, ...) across all provider paths
- Add per-transform timing instrumentation to pipeline
- Guard against over-aggressive code compression (<5% ratio)
- Fix ReadLifecycle partial read supersede logic (_read_covers range check)
- Disable CacheAligner and compress_superseded by default
- Fix all pre-existing mypy errors (CompressionCache return types)
- Fix test mocks to accept **kwargs for cache token parameters
SessionAnalyzer() without a model calls _detect_default_model() which
raises when no API keys are set (e.g., in CI). Pass model="test-model"
in the three tests that mock _call_llm.
Proxy performance logging (`headroom perf`):
- Add always-on RotatingFileHandler to ~/.headroom/logs/proxy.log (10MB x 5 backups)
- Replace scattered log lines with structured PERF lines containing model, msgs,
tok_before/after/saved, cache_read/write/hit_pct, opt_ms, and transforms
- Emit PERF lines from all three response paths (streaming Anthropic, non-streaming
Anthropic, Bedrock streaming)
- Add `headroom perf` CLI that parses proxy logs and reports token savings, cache
hit rates, prefix stability, transform effectiveness, routing breakdown, TOIN
status, and actionable recommendations
- Support --hours and --raw flags for time filtering and raw record output
Learn module rewrite (LLM-based analysis):
- Replace all regex/heuristic analyzers with a single LLM call via LiteLLM
- New SessionAnalyzer builds compact digests and sends to any of 100+ models
- Auto-detect best model from API keys (Anthropic → OpenAI → Gemini)
- Add --model flag for explicit model selection
- Enrich scanner with SessionEvent (user messages, interruptions, subagent summaries),
token usage tracking, and timestamps
- Simplify models: remove EnvironmentFact, StructureNote, Correction, CommandPattern,
RetryPattern, AnalysisReport; add SessionEvent, AnalysisResult
- Simplify writer: remove Recommender class (LLM now produces recommendations directly)
- Update tests for new analyzer and models
Address code review feedback on the initial MCP fix:
- mcp_uninstall: now also calls `claude mcp remove -s user` when the claude
CLI is available, mirroring mcp_install. Removes from mcp.json fallback
config as well if present. This fixes the broken uninstall->reinstall
roundtrip on machines using Claude Code CLI >=2.x.
- cli/mcp.py: move `import subprocess` to module level (was deferred inside
mcp_install function body, inconsistent with other stdlib imports and
harder to mock).
- tests: add TestMCPInstallWithClaudeCLI and TestMCPUninstallWithClaudeCLI
covering the previously-untested `claude mcp add` code path, including
force-overwrite ordering, -e flag for env vars, and fallback on failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two fixes to get the headroom MCP server working with Claude Code CLI:
1. mcp_server.py: define CCR_TOOL_NAME inline instead of importing from
tool_injection, which triggered headroom/__init__.py -> LiteLLM -> HTTP
requests to GitHub, adding 4-5 seconds to startup time.
2. cli/mcp.py: prefer `claude mcp add -s user` when the claude CLI is
available (Claude Code CLI ≥2.x stores servers in ~/.claude/.claude.json,
not ~/.claude/mcp.json). Falls back to writing mcp.json for older versions
and the claude.ai desktop app.
3. tests/test_cli/test_mcp.py: update mock_claude_config_path fixture to
also stub out the claude CLI so install tests exercise the mcp.json
fallback path, matching the fixture's intent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Enable ReadLifecycle by default so stale/superseded Read outputs are
automatically replaced with compact CCR markers — these are provably safe
to compress (file was edited or re-read).
Replace static compression thresholds with adaptive parameters that scale
with conversation length and context pressure:
- protect_recent_reads_fraction: protects the most-recent 50% of messages
from Read exclusion. Old Reads beyond this window become compressible,
preventing the "28 excluded Read/Glob, 0 tokens saved" problem.
- min_ratio_relaxed / min_ratio_aggressive: compression acceptance
threshold interpolates linearly with context pressure (tokens / model
limit). Low pressure → 0.85 (picky), high pressure → 0.65 (accept
anything helpful). Eliminates the fixed 0.9 gate that was rejecting
20+ messages per request.
Also adds --no-read-lifecycle CLI flag, and fixes a missing
pytest.importorskip guard for sentence-transformers in memory tests.
CodeAwareCompressor now analyzes intra-file symbol relationships before
compression, using tree-sitter AST walks to count references, map call
graphs, and detect public/private visibility. This replaces uniform
"keep first N body lines" compression with budget-based allocation driven
by the existing target_compression_rate config.
Key design decisions:
- Distribution-based scoring (min-max normalized within each file) so it
adapts to any file structure: utility libs, test files, orchestrators
- Budget allocation: target_compression_rate determines total body line
budget, distributed proportionally to importance × body size
- max_body_lines respected as a hard cap over budget allocation
- Context-aware: the existing `context` parameter now boosts symbols
matching the user's task (word-boundary matching, not substring)
- Qualified names (ClassName.method) internally to avoid collisions
between identically-named methods in different classes
- Omitted comments include call graph info from AST analysis
- Zero new dependencies — uses tree-sitter already in headroom[code]
- semantic_analysis=True by default, fully backward-compatible when False
- Codex adapter: CodexScanner reads ~/.codex/sessions/*.json, CodexWriter
writes to AGENTS.md + instructions.md. Tested on 328 real sessions.
- Gemini writer: GeminiWriter writes to GEMINI.md (scanner deferred,
sessions stored in protobuf).
- CLI --agent flag: auto-detect available agents or specify claude/codex/gemini.
- Quality gates: min_evidence, min_confidence, min_total_evidence thresholds
prevent weak signals from writing noise to project files.
- Integration tests against real Claude Code and Codex session data on disk.
Tests skip gracefully if data directories don't exist.
- Bash path extraction for Codex (reads files via sed/cat, not Read tool).
- Idempotency, false positive filtering, and skip-write-on-empty tests.
Analyzes past conversation history to find tool call failure patterns,
correlates each failure with what eventually succeeded, and writes
specific project-level learnings to CLAUDE.md and MEMORY.md.
Key design:
- Success correlation: extracts the diff between failed and successful
inputs as the learning (not generic advice)
- Generic architecture: tool-agnostic ToolCall model with pluggable
Scanner/Writer adapters (Claude Code first, extensible to Cursor/Codex)
- 5 analyzers: Environment, Structure, Commands, Retries, Cross-Session
- Dry-run by default, --apply to write, --all for all projects
Also fixes mypy errors in litellm_callback, asgi, langchain chat_model,
and anthropic provider (AsyncClient typing, ToolCall arg-type, int cast).
LangChain provides tool_call args in different shapes (dict args, str
arguments, nested function.arguments) depending on the source. Add
_tool_call_args_to_json() helper to normalize all formats to JSON strings.
Use .get() instead of [] to handle missing keys gracefully.
Detects Read tool outputs that became stale (file was later edited) or
superseded (file was later re-Read) and replaces them with compact markers
+ CCR hashes. Fresh Reads are never touched.
Adds ReadLifecycleConfig to config.py and integrates ReadLifecycleManager
as a pre-processing pass in ContentRouter. Opt-in via config flag to
preserve backward-compatible behavior.
Foundational models (Claude Code, ChatGPT) store memory in flat .md files while
Headroom uses a semantic vector store. This bridge connects the two worlds —
importing .md files into Headroom for semantic search, exporting Headroom memories
back to .md, and keeping them in sync with hash-based change detection.
Three new integration paths — no proxy needed:
1. headroom.compress(messages, model) → CompressResult
One function, auto-detects tokenizer per model, works with any client.
2. headroom.integrations.asgi.CompressionMiddleware
Drop-in ASGI middleware for LiteLLM proxy, FastAPI, or any ASGI app.
3. headroom.integrations.litellm_callback.HeadroomCallback
LiteLLM callback: litellm.callbacks = [HeadroomCallback()]
Fix: TransformPipeline._get_tokenizer() no longer requires a Provider.
Falls back to tokenizer registry which auto-detects per model:
- OpenAI → tiktoken (exact)
- Anthropic → calibrated estimation (3.5 chars/token)
- Open models → HuggingFace (if installed)
15 tests covering compress(), ASGI middleware, LiteLLM callback.
Query Echo addresses attention decay in compressed contexts. After
SmartCrusher compresses tool outputs, the user's question may be
thousands of tokens away. Echo appends a brief reminder after the
last compressed block.
- New: headroom/transforms/query_echo.py
- Compression-ratio-proportional: only triggers when >30% compressed
- Cache-safe: appended at end (after KV cache boundary)
- Provider-agnostic: Anthropic, OpenAI, Gemini
- 18 tests (15 unit + 3 integration with real API)
- Fix _crush_array 4-tuple unpack in test_critical_fixes.py
- Mock MCP SDK availability for install/status/lifecycle tests
- Skip MCP server initialization tests when SDK not available
- Skip negative test (test without SDK) when SDK is installed
Tests now pass both with and without MCP SDK installed.
- Add `headroom mcp install` to configure ~/.claude/mcp.json
- Add `headroom mcp uninstall` to remove configuration
- Add `headroom mcp status` to check setup
- Add `headroom mcp serve` for MCP server (called by Claude Code)
- Add `mcp` optional dependency in pyproject.toml
- Add docs/mcp.md with full documentation
- Add 22 integration tests for MCP CLI
- Fix mypy errors in trained_router.py (None check, type annotation)
- Update README with subscription user instructions
This enables CCR (Compress-Cache-Retrieve) for subscription users who
don't have API access. MCP is Claude's official extension mechanism
that works with subscriptions.
Usage:
pip install "headroom-ai[mcp]"
headroom mcp install
headroom proxy # Terminal 1
claude # Terminal 2
The default model was changed from xlm-roberta-large (~1GB) to
bert-base-multilingual (~350MB) to reduce memory usage. Updated
the test to reflect this change.
Fixes failing CI test: test_llmlingua_compressor.py::test_default_values
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Create headroom/models/config.py as single source of truth for all ML model defaults
- Support environment variable overrides (HEADROOM_SENTENCE_TRANSFORMER, etc.)
- Update all components to use ML_MODEL_DEFAULTS instead of hardcoded values
- Switch LLMLingua default to smaller bert-base model (~350MB vs 1GB)
- Total memory footprint reduced from ~1.6GB to ~980MB
Updated files:
- MLModelRegistry now resolves defaults from config
- EmbeddingScorer, LocalEmbedder, TrainedRouter use config
- All dataclass configs use field(default_factory=...) for consistency
- Tests updated to handle auto-selected vector backends
Remove unnecessary module-level patches for AutoTokenizer and
AutoModelForSequenceClassification since models are now loaded via
MLModelRegistry. The _load_models mock is sufficient for this test.
- Add VectorBackend.AUTO that prefers SQLITE_VEC if available, else HNSW
- Add SQLiteVectorIndex configuration options (vector_db_path, vector_cache_size_kb)
- Add update_embedding() method to SQLiteVectorIndex for VectorIndex protocol
- Update factory to create SQLiteVectorIndex when appropriate
- Skip HNSW-specific tests when hnswlib is not available (fixes CI on Python 3.12)
HNSWVectorIndex now supports optional memory bounding:
- New max_entries parameter sets soft limit on number of entries
- When limit reached, lowest importance entries are evicted
- Eviction uses importance (ascending) then age (oldest first) ordering
- eviction_batch_size controls how many entries evicted at once
Changes:
- Add max_entries and eviction_batch_size parameters
- Add _evict_entries() method for importance-based eviction
- Update get_memory_stats() to report budget_bytes and evictions
- Update stats() to include eviction metrics
- Update save_index/load_index to persist eviction settings
- Add 5 new tests for eviction behavior
- Add MemoryToolAdapter for unified memory across providers
- Anthropic: Uses native memory tool (memory_20250818) for subscription safety
- OpenAI/Gemini/Others: Uses function calling format
- All providers share the same semantic vector store backend
- Simplify CLI to single --memory flag with auto-detection
- Add proper resource cleanup (close methods) to fix test isolation
- Update README with memory documentation
Follows up on PR #19 which fixed RollingWindow but missed IntelligentContextManager,
the default context manager used by the proxy.
Changes:
1. intelligent_context.py: Extended `_get_protected_indices()` to handle Anthropic format:
- Scan assistant.content for type="tool_use" blocks
- Protect user messages containing type="tool_result" blocks with matching tool_use_id
2. test_intelligent_context.py: Added TestAnthropicFormatToolProtection class with 5 tests:
- test_anthropic_tool_result_protected_when_tool_use_protected
- test_anthropic_tool_units_dropped_atomically
- test_anthropic_multiple_tools_same_message_atomic
- test_anthropic_format_no_api_error_scenario (verifies bug fix)
- test_mixed_openai_and_anthropic_formats
3. test_rolling_window.py: Added matching TestAnthropicFormatToolProtection class with 5 tests
- Added skip decorator for CI/CD when OPENAI_API_KEY is not set
This ensures both context managers (RollingWindow and IntelligentContextManager) correctly
handle Anthropic's native tool_use/tool_result format, preventing the
"unexpected tool_use_id found in tool_result blocks" API error.
Root Cause:
The `find_tool_units()` function in `parser.py` only detected OpenAI
format tool calls (assistant.tool_calls + role="tool" messages), not
Anthropic format (assistant.content[type=tool_use] + user.content[type=tool_result]).
This caused RollingWindow and IntelligentContext transforms to treat
Anthropic tool_use and tool_result as separate, independently droppable
messages. When context needed to be trimmed, the assistant message with
tool_use could be dropped while keeping the user message with tool_result,
creating orphaned tool_result blocks.
When sent to the Anthropic API, this produces the error:
"unexpected tool_use_id found in tool_result blocks"
Changes:
1. parser.py: Extended `find_tool_units()` to detect Anthropic format:
- Scan user messages for content blocks with type="tool_result"
- Scan assistant messages for content blocks with type="tool_use"
- Map tool_use_id to corresponding response message indices
2. rolling_window.py: Extended `_get_protected_indices()` to protect
Anthropic format tool pairs:
- Detect tool_use blocks in assistant.content
- Find and protect matching user messages with tool_result blocks
3. tests/test_parser.py: Added 4 new tests for Anthropic format:
- test_anthropic_format_tool_use_and_result
- test_anthropic_format_multiple_tool_uses
- test_anthropic_format_orphaned_tool_result
- test_mixed_openai_and_anthropic_formats
Test Results: 82 passed (including 4 new Anthropic format tests)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
## What this PR fixes
1. **CI Python 3.12 failure**: Added skip decorator to `TestLocalBackend`
in `test_memory_system.py` - these tests require hnswlib which is not
available on all CI runners.
2. **Missing test coverage**: Added 6 tests for the `exclude_tools` feature
in `test_content_router.py`. Tests use existing helper functions
`generate_python_code()`, `generate_json_data()`, and
`generate_search_results()` defined at lines 57-95 of the same file.
3. **Anthropic/OpenAI inconsistency**: Fixed `_process_content_blocks()`
to add `router:excluded:tool` marker for Anthropic format, matching
the OpenAI format behavior at line 1157.
4. **Dead code removal**: Removed unused `exclude_tools` field from
`SmartCrusherConfig` - the actual implementation uses
`ContentRouterConfig.exclude_tools` in content_router.py.
AI review: code-reviewer (2 iterations), adversarial-reviewer (2 iterations)
Issues fixed: missing test coverage, format inconsistency, dead code
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add pytestmark skip conditions to memory test modules that depend on
hnswlib (core_operations, factory, easy). The subprocess probe for
hnswlib correctly detects unavailability on some platforms (like
Python 3.13 CI runners), but these tests were still trying to run
and failing with ImportError.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add pytest.importorskip("trafilatura") to HTML extractor test modules
to skip tests gracefully when the optional trafilatura dependency is
not installed. This fixes CI failures in the base test matrix that
doesn't include the html extras.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HTMLExtractor uses trafilatura to extract main content from HTML pages,
removing scripts, styles, navigation, and ads. This achieves 94.9%
compression while preserving 98.2% recall on the Scrapinghub benchmark.
Key features:
- Automatic HTML detection in content router
- Configurable output format (markdown or text)
- Metadata extraction (title, author, date, description)
- Batch extraction support
Evaluation framework:
- OSS benchmark integration (Scrapinghub Article Extraction Benchmark)
- LLM-as-judge evaluation for QA accuracy preservation
- F1 score: 0.919 on 181-sample benchmark (baseline: 0.958)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
DiffCompressor:
- Parse unified diff format and compress by reducing context lines
- Preserve file headers and all +/- change lines
- Score hunks by relevance (error keywords, query matches)
- Add summary line: [N files, +X -Y lines]
- Expected 30-50% savings on typical git diffs
- Wire into content router for CompressionStrategy.DIFF
- 30 tests covering parsing, compression, edge cases
hnswlib SIGILL fix:
- Move hnswlib import from module level to lazy loading
- hnswlib crashes with SIGILL (Illegal Instruction) on CPUs
without AVX support, before Python can catch the error
- Now imports only when HNSWVectorIndex is actually used
- HNSW_AVAILABLE is checked lazily via __getattr__
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
