Documents what the integration optimizes (messages, tool calls, streaming)
and what operates outside the optimization boundary (agent memory, knowledge
bases, agent teams). Includes best practices and future improvement plans.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- HeadroomAgnoModel: Drop-in wrapper for any Agno model with automatic
context optimization
- HeadroomPreHook/HeadroomPostHook: Agent-level hooks for tracking
optimization metrics across tool calls
- Provider detection for Agno models (OpenAI, Anthropic, Google, etc.)
- Full test coverage for model wrapper and hooks
- Add litellm as a core dependency for accessing its community-maintained
model pricing database (2,425+ models across all major providers)
- Create headroom/pricing/litellm_pricing.py with simple wrapper functions
- Update ModelRegistry.estimate_cost() to fetch pricing from LiteLLM
- Remove hardcoded pricing fields from ModelInfo dataclass
- Update tests to reflect new pricing source
The passthrough route was incorrectly routing requests based on
User-Agent string, which caused requests from Anthropic SDK to be
forwarded to OpenAI's API instead of Anthropic's API.
Now checks for anthropic-version header or x-api-key header to
reliably detect Anthropic SDK requests and route them correctly.
The LLMLingua PromptCompressor.compress_prompt() expects context as a list,
not original_prompt as a string. Also use empty list [] for force_tokens
instead of None.
Handle multimodal messages (images) where content is a list instead of string.
Skip non-string content in LLMLingua and code-aware compression transforms.
- Add --no-llmlingua flag (compression ON by default, opt-out)
- Add --llmlingua-device and --llmlingua-rate flags
- Add --no-code-aware flag (AST compression ON by default, opt-out)
- Add code_aware_enabled config and _setup_code_aware() method
- Update banner to show Code-Aware compression status
- Better UX: zero-config proxy starts with all optimizations enabled
- Fix compress_with_llmlingua call: use compression_rate parameter
- Fix CompressionStore.store() call: use positional args for original/compressed
- Fix extract_json_schema return type to support arrays
- Add proper type casts for Any returns
Features:
- with_fast_memory(): Zero-latency inline extraction (Letta-style)
- Memory extracted as part of LLM response, no extra API calls
- Semantic retrieval with local embeddings (sub-50ms)
- with_memory(): Background extraction for non-blocking memory
- SQLite + FTS5 storage with vector similarity search
- Multi-user isolation by user_id
Memory enables temporal compression - extract key facts instead of
carrying full conversation history (4000 tokens → 50 tokens).
Includes:
- Comprehensive test suite (71 new tests)
- Documentation (docs/memory.md)
- Benchmark examples comparing approaches
- E2E test with LLM-as-judge evaluation
Build artifacts with Python 3.10 to ensure maximum compatibility
and avoid accidentally using newer features in build tooling.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Adds Python 3.13 to the test matrix. Optimizes CI by running linting
and type checking only on Python 3.12 instead of all versions.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Test cases verify:
- Content-Encoding and Content-Length headers are correctly removed
- Response bodies are already decompressed by httpx
- Keeping compression headers causes length mismatch issues
- The fix doesn't break uncompressed responses
These tests will catch regressions of the ZlibError bug.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
httpx automatically decompresses responses when reading response.content,
but the proxy was forwarding the original Content-Encoding headers. This
caused clients to attempt double-decompression, resulting in ZlibError.
Fixes compression header handling in:
- Non-streaming Anthropic/OpenAI responses
- Cached responses
- Passthrough handler
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
CodeAwareCompressor:
- Tree-sitter based AST parsing for Python, JS, TS, Go, Rust, Java, C, C++
- Preserves imports, signatures, type annotations, error handlers
- Guarantees syntactically valid output
- Uses tree-sitter-language-pack for broad language support
ContentRouter:
- Intelligent compression orchestrator
- Auto-routes content to optimal compressor based on type detection
- Source hint support for high-confidence routing
Custom Model Configuration:
- HEADROOM_MODEL_LIMITS env var and ~/.headroom/models.json support
- Pattern-based inference for unknown models (opus/sonnet/haiku tiers)
- Support for Claude 4.5, Claude 4, o3, o3-mini
- Graceful fallback - never crashes on unknown models
Integrate Microsoft's LLMLingua-2 ML-based compression as an opt-in
feature for the proxy server, with excellent developer experience.
Features:
- New CLI flags: --llmlingua, --llmlingua-device, --llmlingua-rate
- ProxyConfig options: llmlingua_enabled, llmlingua_device, llmlingua_target_rate
- Smart startup hints when llmlingua is available but not enabled
- Helpful error messages when enabled but not installed
- LLMLinguaCompressor inserted before RollingWindow in pipeline
Why opt-in:
- Heavy dependencies (~2GB torch, transformers)
- 10-30s cold start for model loading
- ~1GB RAM when loaded
- Default proxy stays lightweight (<5ms overhead)
Tests:
- 26 new tests in test_proxy_llmlingua.py covering config, setup,
banner status, CLI args, DevEx messages, and edge cases
Documentation:
- Updated README.md with proxy integration section
- Updated docs/proxy.md with LLMLingua CLI options
- Updated docs/transforms.md with LLMLinguaCompressor reference
- Updated docs/ARCHITECTURE.md with pipeline and file structure
- Updated CHANGELOG.md with new feature
Add standalone text compression utilities that applications can use
explicitly for non-JSON content:
- SearchCompressor: for grep/ripgrep output (file:line:content format)
- LogCompressor: for build/test logs (pytest, npm, cargo output)
- TextCompressor: for generic plain text with anchor preservation
- detect_content_type: content type detection for routing decisions
Design decision: Text compression is OPT-IN, not automatic. SmartCrusher
continues to compress JSON automatically (structure-preserving, safe),
but passes non-JSON through unchanged. Applications decide when and how
to compress text content based on their specific needs.
This prevents lossy text compression from being applied automatically,
which could lose important context in coding tasks (e.g., root cause
errors in logs, critical matches in search results).
Includes 22 tests covering content detection, compression utilities,
and SmartCrusher pass-through behavior.
Separates deterministic anchor matching from probabilistic relevance
scoring to reliably preserve needle records in crushing:
- Add extract_query_anchors() calls before relevance scoring
- Items matching UUIDs, quoted names, IDs are now always preserved
- Works across all 4 compression plans: time_series, cluster_sample,
top_n, and smart_sample
This fixes the test_preserves_needle_by_name failure where "Alice" at
index 50 was dropped because hybrid relevance scoring gave it only
0.17 (below 0.25 threshold). Now anchor matching catches exact
matches deterministically.
- Fix 34 mypy errors across 17 files with type annotations and casts
- Add type: ignore comments for legitimate dynamic patterns
- Handle None operands with (value or 0) pattern
- Cast return values to proper types (int, float, str, bool)
- Add EstimatingTokenCounter imports where needed
- Use getattr() for potentially missing attributes
- Fix flaky test_paraphrase_match with more distinct semantic examples
- Add mlx to mypy ignore list (broken third-party stubs)
- Fix 34 mypy errors across 17 files with type annotations and casts
- Add type: ignore comments for legitimate dynamic patterns
- Handle None operands with (value or 0) pattern
- Cast return values to proper types (int, float, str, bool)
- Add EstimatingTokenCounter imports where needed
- Use getattr() for potentially missing attributes
- Fix flaky test_paraphrase_match with more distinct semantic examples
- Add mlx to mypy ignore list (broken third-party stubs)
- Renamed package from 'headroom' to 'headroom-ai' (PyPI name conflict)
- Fixed numpy/jinja2 imports to be lazy (core install no longer crashes)
- Fixed SQLite default path (now uses temp directory)
- Fixed f-string {tool} crash in proxy server
- Updated README with correct package name and examples
- Added quickstart and troubleshooting docs
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
