- Cache LiteLLM model resolution to avoid synchronous cost_per_token()
calls blocking the async event loop on every request (3.28ms → 0.12ms)
- Add proper error handling in streaming generate() for httpx connection
errors, timeouts, and pool exhaustion — previously crashed ASGI app
- Bump version to 0.3.3
- Fix LiteLLM pricing for claude-opus-4-6 by adding provider prefix
fallback (_resolve_litellm_model) when model name is unrecognized
- Fix cost savings double-counting: remove cache_read/cache_write
tokens from original_cost calculation at all 3 call sites, which
was inflating reported savings ~425x
- Add information saturation engine (adaptive_sizer.py) using Kneedle
algorithm on unique bigram coverage curves to statistically determine
optimal compression K instead of hardcoded thresholds
- Add CompressionProfile with per-tool bias multipliers (conservative,
moderate, aggressive) configurable via --tool-profile CLI flag and
HEADROOM_TOOL_PROFILES env var
- Integrate adaptive K into SmartCrusher, SearchCompressor, and
LogCompressor, threading bias through ContentRouter
- Remove Grep and Bash from DEFAULT_EXCLUDE_TOOLS so their outputs
are now compressed (only Read and Glob excluded)
- Route /v1/chat/completions through configured LiteLLM backend (Bedrock, Azure,
Databricks, etc.) instead of hardcoding to OpenAI API
- Add send_openai_message() method to Backend base class and LiteLLMBackend
- Add Databricks provider to PROVIDER_REGISTRY
- Add /serving-endpoints/{model}/invocations endpoint for Databricks CLI compatibility
- Integrate Magika ML-based content detection in ContentRouter for improved accuracy
- Fall back to regex-based detection when Magika is unavailable
The server was using tokenizer.count_text() on message content only,
while the pipeline uses tokenizer.count_messages() for full message
structure. This caused the "saved X tokens" logs to show different
values (e.g., 2.5M vs 6.7M for the same compression).
Now the server uses result.tokens_before and result.tokens_after from
the pipeline when optimization succeeds, ensuring consistent metrics
between pipeline logs and server logs.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Cache hits were re-reporting the original compression savings to metrics,
causing massive inflation. With 400 cache hits each re-reporting 471K
"saved" tokens, stats showed 188M tokens saved instead of actual ~300K.
Token savings should only be counted once when compression is first
applied, not on every subsequent cache hit.
Instead of hardcoding model mappings, now fetches available inference
profiles from AWS Bedrock API using list_inference_profiles(). This:
- Automatically gets correct region-specific profiles (us., eu., apac.)
- Handles model name normalization from various input formats
- Caches results per region to avoid repeated API calls
- Supports all Claude models available in the user's region
Requires boto3 and valid AWS credentials.
- Mock MCP SDK availability for install/status/lifecycle tests
- Skip MCP server initialization tests when SDK not available
- Skip negative test (test without SDK) when SDK is installed
Tests now pass both with and without MCP SDK installed.
- Add `headroom mcp install` to configure ~/.claude/mcp.json
- Add `headroom mcp uninstall` to remove configuration
- Add `headroom mcp status` to check setup
- Add `headroom mcp serve` for MCP server (called by Claude Code)
- Add `mcp` optional dependency in pyproject.toml
- Add docs/mcp.md with full documentation
- Add 22 integration tests for MCP CLI
- Fix mypy errors in trained_router.py (None check, type annotation)
- Update README with subscription user instructions
This enables CCR (Compress-Cache-Retrieve) for subscription users who
don't have API access. MCP is Claude's official extension mechanism
that works with subscriptions.
Usage:
pip install "headroom-ai[mcp]"
headroom mcp install
headroom proxy # Terminal 1
claude # Terminal 2
Some versions of transformers return BaseModelOutputWithPooling objects
instead of raw tensors from get_image_features() and get_text_features().
This caused an AttributeError when trying to call .norm() on the output.
Added _extract_tensor() helper that handles both cases:
- Returns tensor unchanged if already a tensor (backward compatible)
- Extracts pooler_output from BaseModelOutputWithPooling objects
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The default model was changed from xlm-roberta-large (~1GB) to
bert-base-multilingual (~350MB) to reduce memory usage. Updated
the test to reflect this change.
Fixes failing CI test: test_llmlingua_compressor.py::test_default_values
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixes issue where Claude Code requests with images (e.g., from Playwright
screenshots) were being rejected with "Request too large" error.
Root cause: Hardcoded 10MB limit was too small for base64-encoded images.
Solution: Increased limit to 100MB to accommodate image-heavy payloads.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Create headroom/models/config.py as single source of truth for all ML model defaults
- Support environment variable overrides (HEADROOM_SENTENCE_TRANSFORMER, etc.)
- Update all components to use ML_MODEL_DEFAULTS instead of hardcoded values
- Switch LLMLingua default to smaller bert-base model (~350MB vs 1GB)
- Total memory footprint reduced from ~1.6GB to ~980MB
Updated files:
- MLModelRegistry now resolves defaults from config
- EmbeddingScorer, LocalEmbedder, TrainedRouter use config
- All dataclass configs use field(default_factory=...) for consistency
- Tests updated to handle auto-selected vector backends
Remove unnecessary module-level patches for AutoTokenizer and
AutoModelForSequenceClassification since models are now loaded via
MLModelRegistry. The _load_models mock is sufficient for this test.
Previously, SentenceTransformer was loaded up to 5 times in different
components, wasting ~1.5GB of memory. Now all ML models are shared via
MLModelRegistry:
- SentenceTransformer (text embeddings)
- SIGLIP (image embeddings)
- spaCy (NER)
- Technique router (image optimization)
Updated components to use the registry:
- headroom/relevance/embedding.py
- headroom/memory/adapters/embedders.py
- headroom/cache/dynamic_detector.py
- headroom/prediction/feature_extractor.py
- headroom/evals/metrics.py
- headroom/image/trained_router.py
- Add VectorBackend.AUTO that prefers SQLITE_VEC if available, else HNSW
- Add SQLiteVectorIndex configuration options (vector_db_path, vector_cache_size_kb)
- Add update_embedding() method to SQLiteVectorIndex for VectorIndex protocol
- Update factory to create SQLiteVectorIndex when appropriate
- Skip HNSW-specific tests when hnswlib is not available (fixes CI on Python 3.12)
HNSWVectorIndex now supports optional memory bounding:
- New max_entries parameter sets soft limit on number of entries
- When limit reached, lowest importance entries are evicted
- Eviction uses importance (ascending) then age (oldest first) ordering
- eviction_batch_size controls how many entries evicted at once
Changes:
- Add max_entries and eviction_batch_size parameters
- Add _evict_entries() method for importance-based eviction
- Update get_memory_stats() to report budget_bytes and evictions
- Update stats() to include eviction metrics
- Update save_index/load_index to persist eviction settings
- Add 5 new tests for eviction behavior
- Add MemoryToolAdapter for unified memory across providers
- Anthropic: Uses native memory tool (memory_20250818) for subscription safety
- OpenAI/Gemini/Others: Uses function calling format
- All providers share the same semantic vector store backend
- Simplify CLI to single --memory flag with auto-detection
- Add proper resource cleanup (close methods) to fix test isolation
- Update README with memory documentation
Follows up on PR #19 which fixed RollingWindow but missed IntelligentContextManager,
the default context manager used by the proxy.
Changes:
1. intelligent_context.py: Extended `_get_protected_indices()` to handle Anthropic format:
- Scan assistant.content for type="tool_use" blocks
- Protect user messages containing type="tool_result" blocks with matching tool_use_id
2. test_intelligent_context.py: Added TestAnthropicFormatToolProtection class with 5 tests:
- test_anthropic_tool_result_protected_when_tool_use_protected
- test_anthropic_tool_units_dropped_atomically
- test_anthropic_multiple_tools_same_message_atomic
- test_anthropic_format_no_api_error_scenario (verifies bug fix)
- test_mixed_openai_and_anthropic_formats
3. test_rolling_window.py: Added matching TestAnthropicFormatToolProtection class with 5 tests
- Added skip decorator for CI/CD when OPENAI_API_KEY is not set
This ensures both context managers (RollingWindow and IntelligentContextManager) correctly
handle Anthropic's native tool_use/tool_result format, preventing the
"unexpected tool_use_id found in tool_result blocks" API error.
Root Cause:
The `find_tool_units()` function in `parser.py` only detected OpenAI
format tool calls (assistant.tool_calls + role="tool" messages), not
Anthropic format (assistant.content[type=tool_use] + user.content[type=tool_result]).
This caused RollingWindow and IntelligentContext transforms to treat
Anthropic tool_use and tool_result as separate, independently droppable
messages. When context needed to be trimmed, the assistant message with
tool_use could be dropped while keeping the user message with tool_result,
creating orphaned tool_result blocks.
When sent to the Anthropic API, this produces the error:
"unexpected tool_use_id found in tool_result blocks"
Changes:
1. parser.py: Extended `find_tool_units()` to detect Anthropic format:
- Scan user messages for content blocks with type="tool_result"
- Scan assistant messages for content blocks with type="tool_use"
- Map tool_use_id to corresponding response message indices
2. rolling_window.py: Extended `_get_protected_indices()` to protect
Anthropic format tool pairs:
- Detect tool_use blocks in assistant.content
- Find and protect matching user messages with tool_result blocks
3. tests/test_parser.py: Added 4 new tests for Anthropic format:
- test_anthropic_format_tool_use_and_result
- test_anthropic_format_multiple_tool_uses
- test_anthropic_format_orphaned_tool_result
- test_mixed_openai_and_anthropic_formats
Test Results: 82 passed (including 4 new Anthropic format tests)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add `question` parameter to LLMLinguaCompressor.compress() for QA-aware
token selection (passes to LLMLingua-2's compress_prompt)
- Flow `question` parameter through ContentRouter compression pipeline
- Enable ContentRouter in default pipeline (was missing, causing 0% compression)
- Add `content_router_enabled` config option to HeadroomConfig
This improves compression accuracy for QA tasks by allowing LLMLingua-2 to
preserve tokens relevant to answering the given question.
WebFetch and WebSearch should NOT be excluded by default because:
1. Web content is Headroom's sweet spot - lots of noise (nav, ads, boilerplate)
2. CCR allows retrieval if LLM needs original content
3. Excluding them undermines the core value proposition
DEFAULT_EXCLUDE_TOOLS now only contains local file/code tools:
- Read, Glob, Grep, Bash (and lowercase variants)
These local tools return precise content (line numbers, paths, code)
where exact fidelity matters immediately. Web tools benefit from
compression and can use CCR for on-demand retrieval.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
## What this PR fixes
1. **CI Python 3.12 failure**: Added skip decorator to `TestLocalBackend`
in `test_memory_system.py` - these tests require hnswlib which is not
available on all CI runners.
2. **Missing test coverage**: Added 6 tests for the `exclude_tools` feature
in `test_content_router.py`. Tests use existing helper functions
`generate_python_code()`, `generate_json_data()`, and
`generate_search_results()` defined at lines 57-95 of the same file.
3. **Anthropic/OpenAI inconsistency**: Fixed `_process_content_blocks()`
to add `router:excluded:tool` marker for Anthropic format, matching
the OpenAI format behavior at line 1157.
4. **Dead code removal**: Removed unused `exclude_tools` field from
`SmartCrusherConfig` - the actual implementation uses
`ContentRouterConfig.exclude_tools` in content_router.py.
AI review: code-reviewer (2 iterations), adversarial-reviewer (2 iterations)
Issues fixed: missing test coverage, format inconsistency, dead code
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add pytestmark skip conditions to memory test modules that depend on
hnswlib (core_operations, factory, easy). The subprocess probe for
hnswlib correctly detects unavailability on some platforms (like
Python 3.13 CI runners), but these tests were still trying to run
and failing with ImportError.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The previous fix attempted to lazily import hnswlib but calling
_check_hnswlib_available() still triggered the import, which crashed
with SIGILL on CPUs without AVX support before Python could catch it.
Fix by using subprocess to safely probe for hnswlib availability:
- Import AND create an Index in a subprocess to catch SIGILL at both
import time and first use of AVX instructions
- If subprocess succeeds, then import in main process
- Add debug logging for all failure modes (timeout, crash, etc.)
- Isolates any crash to the subprocess, keeping test process alive
AI review: code-reviewer (1 iteration)
Adversarial review: code-critic (addressed logging, more robust probe)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add pytest.importorskip("trafilatura") to HTML extractor test modules
to skip tests gracefully when the optional trafilatura dependency is
not installed. This fixes CI failures in the base test matrix that
doesn't include the html extras.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add ability to exclude specific tools from compression, useful for CLI tools
like Claude Code where file/search output should be passed through unmodified.
Changes:
- Add DEFAULT_EXCLUDE_TOOLS constant with Read, Grep, Glob, Bash, WebFetch, WebSearch
- Add exclude_tools field to SmartCrusherConfig and ContentRouterConfig
- Add _build_tool_name_map() to ContentRouter for tool_call_id -> name mapping
- Skip compression for tool_result blocks from excluded tools
- Support both Anthropic (tool_use/tool_result) and OpenAI (tool_calls/tool) formats
This prevents Headroom from compressing output from tools where the user
expects to see the full, unmodified content (e.g., file reads, search results).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add native support for OpenRouter API via LiteLLM backend
- Introduce PROVIDER_REGISTRY pattern to eliminate scattered if/else blocks
- New providers can now be added with a single registry entry
Features:
- `headroom proxy --backend openrouter` routes requests to OpenRouter
- Pass-through model naming (anthropic/claude-3.5-sonnet, openai/gpt-4o, etc.)
- CLI shows provider-specific setup instructions from registry
Usage:
export OPENROUTER_API_KEY="sk-or-v1-..."
headroom proxy --backend openrouter
Also fixes mypy type errors in mcp_server.py
Rename client/response variables to be unique per provider branch
to avoid type inference conflicts. Use getattr for Anthropic content
block text access to handle union types.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
