The streaming code path in _stream_response was missing cost calculation
and recording. While non-streaming requests properly calculated costs via
CostTracker.estimate_cost() and passed them to metrics.record_request(),
streaming requests always passed the default value of 0.
This caused session summaries to show $0.00 for total cost and savings
even when using expensive models like Claude Opus.
Added cost calculation logic to the streaming finally block, matching
the non-streaming implementation.
- Remove hardcoded fallback pricing dictionary
- Fix litellm API usage: use cost_per_token() instead of broken
completion_cost() call (API changed, old kwargs no longer work)
- Add proper logging when pricing lookup fails
- Handle cached tokens at 10% of input price
The previous code silently failed because litellm.completion_cost()
no longer accepts prompt_tokens/completion_tokens kwargs, causing
all cost calculations to return $0.00.
- Add quality_retention_eval.py for needle-in-haystack testing to verify
intelligent compression retains critical information (100% retention achieved)
- Add intelligent_context_integration_test.py for comprehensive pipeline testing
- Add test_progressive_summarizer.py with 36 tests for ProgressiveSummarizer
- Add HeadroomConfig parameter to HeadroomClient for direct config injection
- Update pipeline.py with IntelligentContextManager wiring and logging
- Fix all ruff linting issues and format for Python 3.12 compatibility
- Add comprehensive_eval.py benchmark for multi-scenario evaluation
- Add real_data_demo.py for production-scale volume testing
- Add reasoning agent test examples (groq, debug)
- Add .dockerignore to exclude unnecessary files from Docker builds
- Add Dockerfile to containerize Python application with health checks
- Add docker-compose.yml for easy local development and deployment of headroom proxy service
Add support for a custom OpenAI API URL via new configuration field, CLI argument, and environment variable. This allows overriding the default OpenAI API endpoint in the proxy, for users needing to target specific API instances or custom endpoints. Includes null safety and conditional application in the proxy initialization.
The plist template was using ${HEADROOM_PROXY_PORT} in ProgramArguments,
but LaunchAgent doesn't expand environment variables in that context.
Changed to use __PORT__ placeholder which install.sh replaces via sed.
This fixes the "invalid int value" error when starting the proxy service.
AI review: Clean (via pre-commit hook)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add comprehensive macOS deployment support for running headroom proxy as a
persistent background service using LaunchAgent. This enables automatic startup,
crash recovery, and proper lifecycle management for local development environments.
Files added:
- examples/deployment/macos-launchagent/com.headroom.proxy.plist.template
- examples/deployment/macos-launchagent/install.sh (shellcheck-clean)
- examples/deployment/macos-launchagent/uninstall.sh (shellcheck-clean)
- examples/deployment/macos-launchagent/shell-integration.sh (bash + zsh)
- examples/deployment/macos-launchagent/README.md
- docs/macos-deployment.md
Key features:
- Configurable port via HEADROOM_PROXY_PORT environment variable (default: 8787)
- Automated installation and uninstallation scripts
- Shell integration supporting both bash and zsh
- Comprehensive documentation with troubleshooting guide
- All shell scripts are shellcheck-clean (zero errors, warnings, or info messages)
Files modified:
- .gitignore: Added CLAUDE.md to prevent committing local config
- docs/README.md: Added Deployment & Operations section with navigation entry
AI review: Pending (will be run by pre-commit hook)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Phase 2 - Progressive Summarization:
- Add ProgressiveSummarizer with callback pattern for external summarization
- Add AnchoredSummary for tracking which message positions were summarized
- Add SummarizationResult for tracking summarization operations
- Add extractive_summarizer fallback when no LLM callback provided
- Integrate CCR for storing originals and enabling retrieval
- Add SUMMARIZE strategy to IntelligentContextManager
- Add comprehensive tests (59 total for intelligent context)
Agno Integration Fix:
- Add _ensure_message_objects() to convert dicts to Agno Message objects
- Fix response(), response_stream(), aresponse(), aresponse_stream() to
ensure messages are Message objects before calling super()
- Update test mocks to use proper ModelResponse and Metrics objects
- All 66 Agno tests now pass
Previously, HeadroomAgnoModel called wrapped_model.response() which ran the
tool execution loop internally. This meant tool outputs (often 60k+ chars)
were never optimized - only the initial messages were compressed.
The fix delegates response() to the inherited Model.response(), which calls
self.invoke() for each API call. Our invoke() override optimizes messages
before delegating to wrapped_model.invoke(), ensuring tool outputs are
compressed on every API request.
Results from multi_tool_agent_test.py with Claude Sonnet:
- Tokens before optimization: 25,713
- Tokens after optimization: 6,100
- Tokens saved: 19,613 (76.3%)
- Both baseline and optimized found all critical information
Also adds:
- multi_tool_agent_test.py: Real function calling test with 4 tools
- multi_tool_compression_test.py: Direct compression test
- README update with multi-tool agent test results
- Replace marketing claims with actual API test results
- Add needle-in-haystack test: critical error at position 67/100
- Show real JSON compression: 100 entries → 6 entries (93.9% reduction)
- Verified with Claude Sonnet: 87.6% fewer tokens, 4/4 correct answers
- Add example scripts for reproducing the tests
When context is <10% over budget, try deeper compression of tool messages
before dropping. Uses ContentRouter integration for intelligent routing to
SmartCrusher, CodeAwareCompressor, SearchCompressor, or LogCompressor.
- Add _get_content_router() with lazy loading and aggressive config
- Add _apply_compress_first() to compress tool messages via ContentRouter
- Add _get_tool_source_hint() to extract hints from tool calls
- Add _compress_content_blocks() for Anthropic-style content blocks
- Falls back to DROP_BY_SCORE if compression isn't enough
Adds 14 comprehensive integration tests (no mocks):
- TestCompressFirstStrategy: core functionality (8 tests)
- TestCompressFirstWithContentBlocks: Anthropic format
- TestCompressFirstIntegrationWithTOIN: TOIN integration
- TestCompressFirstEdgeCases: edge cases (4 tests)
- Fix HeadroomAgnoModel to properly extend agno.models.base.Model as a dataclass
- Implement required abstract methods (invoke, ainvoke, invoke_stream, ainvoke_stream)
- Add type: ignore comments for method signature overrides
- Fix mypy errors in telemetry/models.py and telemetry/toin.py
- Add real Ollama integration tests for both Agno and LangChain (no API keys needed)
- Add ollama and langchain-ollama to dev dependencies for local testing
- Update existing tests to use new HeadroomAgnoModel API
- Add detailed breakdowns by provider and model to /stats
- Include compression, telemetry, and feedback loop statistics
- Add latency.average_ms metric
- Add real-world agent benchmark with MCP tool patterns
- Add worst-case and adversarial benchmarks for edge cases
- Bump version to 0.2.12
Features:
- Add field-level learning to TOIN from retrieved items
- CompressionStore now passes retrieved_items to TOIN for learning
- Add FieldSemantics class for tracking field usage patterns
Test improvements:
- Add TestCacheOptimizerInvocation to verify optimizer is actually invoked
- Add TestSemanticCacheIntegration to verify cache hit returns without API call
- Add TestSessionStatsTracking to verify session stats are tracked
- Add TestEndToEndTOINIntegration for full CCR cycle with TOIN
- Add critical field_semantics assertions to catch feedback loop bugs
Fixes:
- Remove unused imports and variables (ruff linting)
Bump version to 0.2.11
Documents what the integration optimizes (messages, tool calls, streaming)
and what operates outside the optimization boundary (agent memory, knowledge
bases, agent teams). Includes best practices and future improvement plans.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- HeadroomAgnoModel: Drop-in wrapper for any Agno model with automatic
context optimization
- HeadroomPreHook/HeadroomPostHook: Agent-level hooks for tracking
optimization metrics across tool calls
- Provider detection for Agno models (OpenAI, Anthropic, Google, etc.)
- Full test coverage for model wrapper and hooks
- Add litellm as a core dependency for accessing its community-maintained
model pricing database (2,425+ models across all major providers)
- Create headroom/pricing/litellm_pricing.py with simple wrapper functions
- Update ModelRegistry.estimate_cost() to fetch pricing from LiteLLM
- Remove hardcoded pricing fields from ModelInfo dataclass
- Update tests to reflect new pricing source
The passthrough route was incorrectly routing requests based on
User-Agent string, which caused requests from Anthropic SDK to be
forwarded to OpenAI's API instead of Anthropic's API.
Now checks for anthropic-version header or x-api-key header to
reliably detect Anthropic SDK requests and route them correctly.
The LLMLingua PromptCompressor.compress_prompt() expects context as a list,
not original_prompt as a string. Also use empty list [] for force_tokens
instead of None.
Handle multimodal messages (images) where content is a list instead of string.
Skip non-string content in LLMLingua and code-aware compression transforms.
- Add --no-llmlingua flag (compression ON by default, opt-out)
- Add --llmlingua-device and --llmlingua-rate flags
- Add --no-code-aware flag (AST compression ON by default, opt-out)
- Add code_aware_enabled config and _setup_code_aware() method
- Update banner to show Code-Aware compression status
- Better UX: zero-config proxy starts with all optimizations enabled
- Fix compress_with_llmlingua call: use compression_rate parameter
- Fix CompressionStore.store() call: use positional args for original/compressed
- Fix extract_json_schema return type to support arrays
- Add proper type casts for Any returns
