chopratejas
83c0334ccd
docs: update documentation for IntelligentContext TOIN + CCR integration
...
Updates across multiple docs to reflect the new message-level compression
with TOIN + CCR integration:
- docs/ccr.md: Add CCR-enabled components table, message-level CCR section
- docs/ARCHITECTURE.md: Expand Transform 6 with TOIN + CCR integration details
- docs/configuration.md: Add CCR integration config and marker format
- docs/proxy.md: Add CCR integration note for context management
- docs/README.md: Update to reference IntelligentContextManager as default
Also adds examples/test_intelligent_context_toin_ccr.py for scale testing
the TOIN + CCR integration with real API calls.
2026-01-27 16:08:36 -08:00
Tejas Chopra
2fd9552102
Add image token compression with trained ML router
...
Introduces automatic image compression for LLM requests, reducing token
usage by 40-90% while maintaining answer accuracy.
Key features:
- Trained MiniLM classifier (93.7% accuracy) hosted on HuggingFace
- SigLIP-based image analysis for content-aware routing
- Provider-specific compression:
- OpenAI: detail="low" parameter
- Anthropic: PIL resize to 512px
- Google: PIL resize to 768px (tile-optimized)
- Four compression techniques: full_low, preserve, crop, transcode
- Integration in both Headroom proxy and SDK (ContentRouter)
New files:
- headroom/image/ module with ImageCompressor API
- docs/image-compression.md user documentation
- tests/test_image_compressor.py (51 tests)
Model: chopratejas/technique-router on HuggingFace (~128MB)
2026-01-25 22:40:43 -08:00
chopratejas
cd5ea2ea1d
Add architecture diagrams to README and docs
2026-01-20 00:44:55 -08:00
chopratejas
14ecab6bfa
Add IntelligentContextManager for semantic-aware context management
...
- Add multi-factor importance scoring (recency, semantic similarity,
TOIN importance, error indicators, forward references, token density)
- No hardcoded patterns - all signals learned from TOIN or computed
- Add ScoringWeights and IntelligentContextConfig dataclasses
- Add MessageScorer for scoring individual messages
- Add strategy selection: NONE, COMPRESS_FIRST, DROP_BY_SCORE
- Preserve tool call/response atomicity when dropping
- Add comprehensive tests (62 tests total)
- Update documentation (transforms, configuration, api, architecture)
2026-01-18 22:22:48 -08:00
chopratejas
d724f14022
v0.2.2: Add CCR Response Handler, Context Tracker, and restructure docs
...
Features:
- CCR Response Handler: Automatically intercepts and handles headroom_retrieve tool calls
- CCR Context Tracker: Multi-turn awareness with proactive expansion of relevant compressed content
- New CCR demo script showing before/after flow
Documentation:
- Restructured README from 885 lines to 190 lines for better DevEx
- Split detailed docs into focused guides: ccr.md, sdk.md, configuration.md,
text-compression.md, llmlingua.md, metrics.md, errors.md
- Updated docs/README.md index with all new documentation
Tests:
- Added comprehensive tests for Response Handler (32 tests)
- Added comprehensive tests for Context Tracker (32 tests)
- All 977 tests passing
2026-01-14 13:03:41 -08:00
chopratejas
45633b69ab
Add LLMLingua-2 opt-in support to proxy server
...
Integrate Microsoft's LLMLingua-2 ML-based compression as an opt-in
feature for the proxy server, with excellent developer experience.
Features:
- New CLI flags: --llmlingua, --llmlingua-device, --llmlingua-rate
- ProxyConfig options: llmlingua_enabled, llmlingua_device, llmlingua_target_rate
- Smart startup hints when llmlingua is available but not enabled
- Helpful error messages when enabled but not installed
- LLMLinguaCompressor inserted before RollingWindow in pipeline
Why opt-in:
- Heavy dependencies (~2GB torch, transformers)
- 10-30s cold start for model loading
- ~1GB RAM when loaded
- Default proxy stays lightweight (<5ms overhead)
Tests:
- 26 new tests in test_proxy_llmlingua.py covering config, setup,
banner status, CLI args, DevEx messages, and edge cases
Documentation:
- Updated README.md with proxy integration section
- Updated docs/proxy.md with LLMLingua CLI options
- Updated docs/transforms.md with LLMLinguaCompressor reference
- Updated docs/ARCHITECTURE.md with pipeline and file structure
- Updated CHANGELOG.md with new feature
2026-01-14 12:21:51 -08:00
chopratejas
c1feb60595
feat: Add CCR architecture, TOIN telemetry, and DevEx improvements
...
## Core Features
### Compress-Cache-Retrieve (CCR) Architecture
- Implement reversible compression with automatic retrieval support
- Add CompressionStore for caching original content with TTL-based eviction
- Add CompressionFeedback for learning from retrieval patterns
- Implement tool injection for LLM retrieval capability
- Add MCP server support for CCR operations
- Track retrieval rates to dynamically adjust compression aggressiveness
### Tool Output Intelligence Network (TOIN)
- Implement cross-session pattern learning for tool compression
- Add ToolSignature for structural hashing of tool outputs
- Track compression success rates per strategy (top_n, sample, truncate, etc.)
- Implement privacy-preserving telemetry with SHA256 hashing
- Add persistent storage with JSON file backend
- Support network-effect learning across tool types
### SmartCrusher Enhancements
- Add crushability analysis with variance/uniqueness detection
- Implement statistical anomaly detection for outlier preservation
- Add relevance-based item prioritization using BM25 scoring
- Support multiple compression strategies with quality retention
- Add change point detection for time-series data
- Implement constant factoring for homogeneous datasets
## Developer Experience Improvements
### Exception Hierarchy
- Add HeadroomError base class for all custom exceptions
- Add specific exceptions: ConfigurationError, ProviderError,
StorageError, CompressionError, TokenizationError, CacheError,
ValidationError, TransformError
### Client Enhancements
- Add validate_setup() for configuration verification
- Add get_stats() for in-memory session metrics without DB query
- Track session statistics (requests, tokens saved, cache hits)
### Logging Infrastructure
- Add structured logging to TransformPipeline with token savings
- Add logging to RollingWindow for dropped message tracking
- Add logging to ToolCrusher for compression events
- Add logging to CacheAligner for cache hit/miss detection
- Add logging to SmartCrusher for strategy selection
## Bug Fixes (from deep analysis)
### Critical Fixes
- Fix eviction heap memory leak with stale entry tracking
- Fix hash collision detection in compression store
- Fix strategy truncation desync in TOIN
- Fix non-deterministic set truncation with sorted iteration
- Fix race conditions in lazy initialization with proper locking
- Fix user count double-counting in TOIN metrics
### High Priority Fixes
- Fix unbounded strategy_success_rates growth with LRU eviction
- Fix mutable pattern references with defensive copying
- Fix lock held during file I/O with copy-then-write pattern
- Fix state divergence on eviction with success event recording
- Fix TOIN skip check order for CPU efficiency
- Fix preserve_fields type mismatch (set vs list)
- Fix prioritize_indices exceeding max_items limit
- Fix instance ID collision risk (32-bit to 64-bit hash)
## Testing
- Add comprehensive test suites for CCR, TOIN, and telemetry
- Add crushability detection tests
- Add quality retention tests for compression
- Add integration tests for cross-component data flow
- All 902 tests passing
2026-01-10 10:12:13 -08:00
chopratejas
175746cc26
Prepare for OSS release v0.2.0
...
This commit prepares Headroom for public open source release with
comprehensive documentation, licensing, and community infrastructure.
License & Legal:
- Add Apache 2.0 LICENSE file
- Add NOTICE file with third-party attributions
- Add SECURITY.md for vulnerability reporting
Community:
- Add CONTRIBUTING.md with contribution guidelines
- Add CODE_OF_CONDUCT.md (Contributor Covenant)
- Add GitHub issue templates (bug report, feature request)
- Add pull request template
Documentation:
- Update README.md with compelling value proposition
- Add docs/getting-started.md
- Add docs/proxy.md for proxy server documentation
- Add docs/transforms.md for transform reference
- Add docs/api.md for API reference
- Add examples/README.md
Package Infrastructure:
- Add headroom/py.typed for PEP 561 compliance
- Add headroom/cli.py for CLI entry point
- Add .github/workflows/ci.yml for CI pipeline
- Add .github/workflows/publish.yml for PyPI publishing
- Update pyproject.toml with proper metadata
New Features:
- Add multi-provider support (Google, Cohere, LiteLLM, OpenAI-compatible)
- Add universal tokenizer registry with multiple backends
- Add model registry with pricing and context limits
- Add production proxy server with caching and rate limiting
Code Quality:
- Fix 83 lint issues via ruff auto-fix
- Fix version consistency (benchmarks 0.1.0 → 0.2.0)
- Add skip decorators for optional dependency tests
2026-01-07 11:36:44 -08:00
