IntelligentContext is a message-level compressor that drops low-value
messages. This change adds bidirectional TOIN integration:
- Dropped messages stored in CCR for potential retrieval
- Drops recorded to TOIN for cross-user learning
- Retrieval feedback improves future importance scoring
When messages are dropped and users retrieve them via CCR, TOIN learns
to score those patterns higher next time. This creates a feedback loop
that improves drop decisions across all users.
Changes:
- Add _create_message_signature() for TOIN pattern tracking
- Add _get_compression_store() for CCR integration
- Add _store_dropped_in_ccr() to store dropped messages
- Add _record_drops_to_toin() to record drops for learning
- Update marker to include CCR reference when available
- Update docs with TOIN + CCR integration section
- Update tests to accept both marker formats
ContentRouter now routes purely based on content analysis instead of
relying on hardcoded tool name mappings. This makes the router work
with any MCP tool regardless of naming convention.
Changes:
- Remove generate_source_hint() function and _strategy_from_hint() method
- Remove source_hint parameter from compress() method
- Remove _get_tool_source_hint() from IntelligentContextManager
- Update tests to remove source hint test cases
- Update docs to document content detection approach
- Add headroom.evals module with 12+ dataset loaders (HotpotQA, SQuAD,
Natural Questions, TriviaQA, MS MARCO, LongBench, NarrativeQA, BFCL,
ToolBench, CodeSearchNet, HumanEval, built-in tool outputs)
- Add before/after evaluation runner that compares LLM responses with
original vs compressed context
- Add metrics: F1 score, semantic similarity, exact match, ground truth
- Add CLI: python -m headroom.evals quick|benchmark|list|report
- Add [evals] extra to pyproject.toml for pip install headroom-ai[evals]
Fix ContentRouter to use LLMLingua for plain text compression:
- Route TEXT strategy through LLMLingua instead of heuristic TextCompressor
- Adjust LLMLingua compression rates for better accuracy (0.5 vs 0.25)
- HotpotQA now achieves 95% accuracy with 44% compression
Update documentation with evaluation framework section
Fix test isolation in test_toin.py (TOIN singleton persistence)
Replace static "first 3 + last 2" preservation with intelligent anchor
selection that adapts to data patterns and array size.
Key changes:
- Add AnchorSelector class for dynamic position-based preservation
- Add AnchorConfig for configurable anchor allocation (budget ratio,
strategy weights, information density scoring)
- Add content-based deduplication to prevent wasting slots on identical
items using SHA256 hashing
- Add _fill_remaining_slots() to maximize output when dedup reduces items
- Support data pattern detection (TIME_SERIES, SEARCH_RESULTS, LOGS, GENERIC)
- Support query-aware anchor adjustment for back-heavy patterns
Enterprise hardening:
- Thread-safe: No shared state modified
- O(n) performance for dedup and slot filling
- Fault-tolerant serialization with fallbacks
- Configurable via dedup_identical_items flag
52 tests covering adversarial positions, size adaptation, pattern-aware
anchoring, query-aware selection, information density, coverage metrics,
edge cases, and preservation guarantees.
- Add quality_retention_eval.py for needle-in-haystack testing to verify
intelligent compression retains critical information (100% retention achieved)
- Add intelligent_context_integration_test.py for comprehensive pipeline testing
- Add test_progressive_summarizer.py with 36 tests for ProgressiveSummarizer
- Add HeadroomConfig parameter to HeadroomClient for direct config injection
- Update pipeline.py with IntelligentContextManager wiring and logging
- Fix all ruff linting issues and format for Python 3.12 compatibility
- Add comprehensive_eval.py benchmark for multi-scenario evaluation
- Add real_data_demo.py for production-scale volume testing
- Add reasoning agent test examples (groq, debug)
Phase 2 - Progressive Summarization:
- Add ProgressiveSummarizer with callback pattern for external summarization
- Add AnchoredSummary for tracking which message positions were summarized
- Add SummarizationResult for tracking summarization operations
- Add extractive_summarizer fallback when no LLM callback provided
- Integrate CCR for storing originals and enabling retrieval
- Add SUMMARIZE strategy to IntelligentContextManager
- Add comprehensive tests (59 total for intelligent context)
Agno Integration Fix:
- Add _ensure_message_objects() to convert dicts to Agno Message objects
- Fix response(), response_stream(), aresponse(), aresponse_stream() to
ensure messages are Message objects before calling super()
- Update test mocks to use proper ModelResponse and Metrics objects
- All 66 Agno tests now pass
When context is <10% over budget, try deeper compression of tool messages
before dropping. Uses ContentRouter integration for intelligent routing to
SmartCrusher, CodeAwareCompressor, SearchCompressor, or LogCompressor.
- Add _get_content_router() with lazy loading and aggressive config
- Add _apply_compress_first() to compress tool messages via ContentRouter
- Add _get_tool_source_hint() to extract hints from tool calls
- Add _compress_content_blocks() for Anthropic-style content blocks
- Falls back to DROP_BY_SCORE if compression isn't enough
Adds 14 comprehensive integration tests (no mocks):
- TestCompressFirstStrategy: core functionality (8 tests)
- TestCompressFirstWithContentBlocks: Anthropic format
- TestCompressFirstIntegrationWithTOIN: TOIN integration
- TestCompressFirstEdgeCases: edge cases (4 tests)
Features:
- with_fast_memory(): Zero-latency inline extraction (Letta-style)
- Memory extracted as part of LLM response, no extra API calls
- Semantic retrieval with local embeddings (sub-50ms)
- with_memory(): Background extraction for non-blocking memory
- SQLite + FTS5 storage with vector similarity search
- Multi-user isolation by user_id
Memory enables temporal compression - extract key facts instead of
carrying full conversation history (4000 tokens → 50 tokens).
Includes:
- Comprehensive test suite (71 new tests)
- Documentation (docs/memory.md)
- Benchmark examples comparing approaches
- E2E test with LLM-as-judge evaluation
CodeAwareCompressor:
- Tree-sitter based AST parsing for Python, JS, TS, Go, Rust, Java, C, C++
- Preserves imports, signatures, type annotations, error handlers
- Guarantees syntactically valid output
- Uses tree-sitter-language-pack for broad language support
ContentRouter:
- Intelligent compression orchestrator
- Auto-routes content to optimal compressor based on type detection
- Source hint support for high-confidence routing
Custom Model Configuration:
- HEADROOM_MODEL_LIMITS env var and ~/.headroom/models.json support
- Pattern-based inference for unknown models (opus/sonnet/haiku tiers)
- Support for Claude 4.5, Claude 4, o3, o3-mini
- Graceful fallback - never crashes on unknown models
Integrate Microsoft's LLMLingua-2 ML-based compression as an opt-in
feature for the proxy server, with excellent developer experience.
Features:
- New CLI flags: --llmlingua, --llmlingua-device, --llmlingua-rate
- ProxyConfig options: llmlingua_enabled, llmlingua_device, llmlingua_target_rate
- Smart startup hints when llmlingua is available but not enabled
- Helpful error messages when enabled but not installed
- LLMLinguaCompressor inserted before RollingWindow in pipeline
Why opt-in:
- Heavy dependencies (~2GB torch, transformers)
- 10-30s cold start for model loading
- ~1GB RAM when loaded
- Default proxy stays lightweight (<5ms overhead)
Tests:
- 26 new tests in test_proxy_llmlingua.py covering config, setup,
banner status, CLI args, DevEx messages, and edge cases
Documentation:
- Updated README.md with proxy integration section
- Updated docs/proxy.md with LLMLingua CLI options
- Updated docs/transforms.md with LLMLinguaCompressor reference
- Updated docs/ARCHITECTURE.md with pipeline and file structure
- Updated CHANGELOG.md with new feature
