chopratejas
|
c1feb60595
|
feat: Add CCR architecture, TOIN telemetry, and DevEx improvements
## Core Features
### Compress-Cache-Retrieve (CCR) Architecture
- Implement reversible compression with automatic retrieval support
- Add CompressionStore for caching original content with TTL-based eviction
- Add CompressionFeedback for learning from retrieval patterns
- Implement tool injection for LLM retrieval capability
- Add MCP server support for CCR operations
- Track retrieval rates to dynamically adjust compression aggressiveness
### Tool Output Intelligence Network (TOIN)
- Implement cross-session pattern learning for tool compression
- Add ToolSignature for structural hashing of tool outputs
- Track compression success rates per strategy (top_n, sample, truncate, etc.)
- Implement privacy-preserving telemetry with SHA256 hashing
- Add persistent storage with JSON file backend
- Support network-effect learning across tool types
### SmartCrusher Enhancements
- Add crushability analysis with variance/uniqueness detection
- Implement statistical anomaly detection for outlier preservation
- Add relevance-based item prioritization using BM25 scoring
- Support multiple compression strategies with quality retention
- Add change point detection for time-series data
- Implement constant factoring for homogeneous datasets
## Developer Experience Improvements
### Exception Hierarchy
- Add HeadroomError base class for all custom exceptions
- Add specific exceptions: ConfigurationError, ProviderError,
StorageError, CompressionError, TokenizationError, CacheError,
ValidationError, TransformError
### Client Enhancements
- Add validate_setup() for configuration verification
- Add get_stats() for in-memory session metrics without DB query
- Track session statistics (requests, tokens saved, cache hits)
### Logging Infrastructure
- Add structured logging to TransformPipeline with token savings
- Add logging to RollingWindow for dropped message tracking
- Add logging to ToolCrusher for compression events
- Add logging to CacheAligner for cache hit/miss detection
- Add logging to SmartCrusher for strategy selection
## Bug Fixes (from deep analysis)
### Critical Fixes
- Fix eviction heap memory leak with stale entry tracking
- Fix hash collision detection in compression store
- Fix strategy truncation desync in TOIN
- Fix non-deterministic set truncation with sorted iteration
- Fix race conditions in lazy initialization with proper locking
- Fix user count double-counting in TOIN metrics
### High Priority Fixes
- Fix unbounded strategy_success_rates growth with LRU eviction
- Fix mutable pattern references with defensive copying
- Fix lock held during file I/O with copy-then-write pattern
- Fix state divergence on eviction with success event recording
- Fix TOIN skip check order for CPU efficiency
- Fix preserve_fields type mismatch (set vs list)
- Fix prioritize_indices exceeding max_items limit
- Fix instance ID collision risk (32-bit to 64-bit hash)
## Testing
- Add comprehensive test suites for CCR, TOIN, and telemetry
- Add crushability detection tests
- Add quality retention tests for compression
- Add integration tests for cross-component data flow
- All 902 tests passing
|
2026-01-10 10:12:13 -08:00 |
|
chopratejas
|
9c7d4512d6
|
Initial commit: Headroom SDK - LLM context optimization toolkit
A comprehensive SDK for optimizing LLM context windows, reducing token
usage while preserving critical information for AI agents.
Core Features:
- SmartCrusher: Statistical compression of tool outputs (70-85% reduction)
- CacheAligner: Prefix optimization for prompt cache hits
- RollingWindow: Intelligent context window management
- BM25/Hybrid relevance scoring for smart item selection
Integrations:
- OpenAI and Anthropic provider support
- LangChain integration (ChatModel, Callbacks, Runnable)
- MCP (Model Context Protocol) integration for tool compression
Test Coverage:
- 372 tests passing across all modules
- 35 performance benchmarks
- Real-world agent evaluations with 88% token savings
Key Components:
- headroom/transforms/: Core compression transforms
- headroom/providers/: OpenAI and Anthropic support
- headroom/integrations/: LangChain and MCP integrations
- headroom/relevance/: BM25 and hybrid scoring
- headroom/pricing/: Model pricing registry
- benchmarks/: Performance benchmark suite
- examples/: Usage examples and demos
|
2026-01-06 23:16:58 -08:00 |
|