mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
9c7d4512d6
A comprehensive SDK for optimizing LLM context windows, reducing token usage while preserving critical information for AI agents. Core Features: - SmartCrusher: Statistical compression of tool outputs (70-85% reduction) - CacheAligner: Prefix optimization for prompt cache hits - RollingWindow: Intelligent context window management - BM25/Hybrid relevance scoring for smart item selection Integrations: - OpenAI and Anthropic provider support - LangChain integration (ChatModel, Callbacks, Runnable) - MCP (Model Context Protocol) integration for tool compression Test Coverage: - 372 tests passing across all modules - 35 performance benchmarks - Real-world agent evaluations with 88% token savings Key Components: - headroom/transforms/: Core compression transforms - headroom/providers/: OpenAI and Anthropic support - headroom/integrations/: LangChain and MCP integrations - headroom/relevance/: BM25 and hybrid scoring - headroom/pricing/: Model pricing registry - benchmarks/: Performance benchmark suite - examples/: Usage examples and demos
74 lines
2.1 KiB
Markdown
74 lines
2.1 KiB
Markdown
# LangChain + Headroom Demo
|
|
|
|
Real-world demonstration of Headroom optimization on LangChain agents.
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# Show compression in action (no API key needed)
|
|
PYTHONPATH=. python -m examples.langchain_demo.show_compression
|
|
|
|
# Verify 100% ERROR preservation
|
|
PYTHONPATH=. python -m examples.langchain_demo.verify_errors_kept
|
|
|
|
# Run full agent comparison (requires OPENAI_API_KEY)
|
|
export OPENAI_API_KEY='your-key-here'
|
|
PYTHONPATH=. python -m examples.langchain_demo.run_comparison
|
|
```
|
|
|
|
## Results
|
|
|
|
### Token Savings (with 100% ERROR preservation)
|
|
|
|
| Tool | Before | After | Saved |
|
|
|------|--------|-------|-------|
|
|
| search_users (100 items) | 15,453 | 2,014 | **87%** |
|
|
| search_logs (200 items) | 25,679 | 3,213 | **87%** |
|
|
| get_metrics (100 items) | 11,517 | 8,425 | **27%** |
|
|
| search_docs (50 items) | 6,912 | 2,127 | **69%** |
|
|
| fetch_api_data (75 items) | 15,786 | 3,622 | **77%** |
|
|
| **TOTAL** | **75,347** | **19,401** | **74%** |
|
|
|
|
### Critical Data Preservation
|
|
|
|
- **100% ERROR entries preserved** (27/27 in test runs)
|
|
- **100% anomaly detection** (CPU spikes, high error rates)
|
|
- **First/last items always kept** (context preservation)
|
|
|
|
### Cost Impact (at gpt-4o $2.50/1M)
|
|
|
|
- Per request: $0.19 → $0.05
|
|
- At 1000 req/day: **$4,196/month saved**
|
|
|
|
## What Headroom Does
|
|
|
|
SmartCrusher intelligently compresses tool outputs by:
|
|
|
|
1. **100% ERROR preservation** - NEVER drops error items (bug fix v1.1)
|
|
2. **Keeping first/last items** - Context for pagination
|
|
3. **Keeping anomalies** - High CPU, memory spikes (statistical detection)
|
|
4. **Relevance scoring** - Items matching user's query
|
|
5. **Change points** - Significant transitions in data
|
|
|
|
## Files
|
|
|
|
- `mock_tools.py` - Realistic tool output generators
|
|
- `show_compression.py` - Standalone compression demo
|
|
- `verify_errors_kept.py` - Verify 100% ERROR preservation
|
|
- `run_comparison.py` - Full agent before/after comparison
|
|
|
|
## Eval Tests
|
|
|
|
Run the comprehensive eval suite:
|
|
|
|
```bash
|
|
PYTHONPATH=. pytest tests/test_integrations/test_langchain_evals.py -v
|
|
```
|
|
|
|
12 evals covering:
|
|
- Error preservation (100%)
|
|
- Anomaly detection
|
|
- Relevance matching
|
|
- Compression efficiency
|
|
- Schema preservation
|
|
- Edge cases
|