mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
- Add quality_retention_eval.py for needle-in-haystack testing to verify intelligent compression retains critical information (100% retention achieved) - Add intelligent_context_integration_test.py for comprehensive pipeline testing - Add test_progressive_summarizer.py with 36 tests for ProgressiveSummarizer - Add HeadroomConfig parameter to HeadroomClient for direct config injection - Update pipeline.py with IntelligentContextManager wiring and logging - Fix all ruff linting issues and format for Python 3.12 compatibility - Add comprehensive_eval.py benchmark for multi-scenario evaluation - Add real_data_demo.py for production-scale volume testing - Add reasoning agent test examples (groq, debug) |
||
|---|---|---|
| .. | ||
| deployment/macos-launchagent | ||
| langchain_demo | ||
| mcp_demo | ||
| anthropic_example.py | ||
| basic_usage.py | ||
| capture_before_after_json.py | ||
| ccr_demo.py | ||
| fast_memory_e2e.py | ||
| groq_reasoning_test.py | ||
| inline_memory_demo.py | ||
| intelligent_context_integration_test.py | ||
| langchain_before_after.py | ||
| memory_e2e_test.py | ||
| memory_latency_benchmark.py | ||
| multi_tool_agent_test.py | ||
| multi_tool_compression_test.py | ||
| needle_in_haystack_test.py | ||
| quality_retention_eval.py | ||
| README.md | ||
| real_before_after_test.py | ||
| real_data_demo.py | ||
| real_world_eval.py | ||
| real_world_openai_eval.py | ||
| reasoning_agent_debug_test.py | ||
| show_actual_compression.py | ||
| smart_vs_naive_eval.py | ||
| streaming_example.py | ||
Headroom Examples
This directory contains examples demonstrating Headroom's capabilities.
Quick Start Examples
basic_usage.py
Basic integration with OpenAI client:
export OPENAI_API_KEY='your-key'
python examples/basic_usage.py
anthropic_example.py
Integration with Anthropic Claude:
export ANTHROPIC_API_KEY='your-key'
python examples/anthropic_example.py
streaming_example.py
Streaming responses with optimization:
export OPENAI_API_KEY='your-key'
python examples/streaming_example.py
Evaluation Examples
smart_vs_naive_eval.py
Compare SmartCrusher against naive truncation:
export OPENAI_API_KEY='your-key'
python examples/smart_vs_naive_eval.py
real_world_eval.py
Comprehensive evaluation with Anthropic models:
export ANTHROPIC_API_KEY='your-key'
python examples/real_world_eval.py
real_world_openai_eval.py
Comprehensive evaluation with OpenAI models:
export OPENAI_API_KEY='your-key'
python examples/real_world_openai_eval.py
Demo Directories
langchain_demo/
Full LangChain agent integration demo:
# No API key needed for compression demo
PYTHONPATH=. python -m examples.langchain_demo.show_compression
# Full comparison (requires API key)
export OPENAI_API_KEY='your-key'
PYTHONPATH=. python -m examples.langchain_demo.run_comparison
See langchain_demo/README.md for details.
mcp_demo/
MCP (Model Context Protocol) integration demo:
export OPENAI_API_KEY='your-key'
PYTHONPATH=. python -m examples.mcp_demo.run_agent_eval
Running Examples
All examples can be run from the repository root:
# Install dependencies
pip install -e ".[dev]"
# Run any example
python examples/<example_name>.py
Expected Results
| Example | Token Savings | Notes |
|---|---|---|
| basic_usage | 50-70% | Simple tool output compression |
| langchain_demo | 70-85% | Real agent with multiple tools |
| mcp_demo | 60-80% | MCP tool outputs |
| real_world_eval | 50-90% | Varies by scenario |
Troubleshooting
ModuleNotFoundError: No module named 'headroom'
Run from the repository root with PYTHONPATH:
PYTHONPATH=. python examples/basic_usage.py
Or install in development mode:
pip install -e .
API Key Errors
Ensure your API keys are set:
export OPENAI_API_KEY='sk-...'
export ANTHROPIC_API_KEY='sk-ant-...'