diff --git a/README.md b/README.md index cf927e9c6..3d0724925 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Every tool call, DB query, file read, and RAG retrieval your agent makes is 70-95% boilerplate.
Headroom compresses it away before it hits the model.

Works with any agent — coding agents (Claude Code, Codex, Cursor, Aider), custom agents
- (LangChain, LangGraph, CrewAI, Agno, OpenAI Agents SDK), or your own Python code. + (LangChain, LangGraph, Agno, Strands), or your own Python code.

@@ -83,7 +83,7 @@ response = client.messages.create(model="claude-sonnet-4-5-20250929", messages=r print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})") ``` -Works with any Python LLM client — Anthropic, OpenAI, LiteLLM, Bedrock, httpx, anything. Works with any agent framework — LangChain, LangGraph, CrewAI, Agno, OpenAI Agents SDK, or your own code. +Works with any Python LLM client — Anthropic, OpenAI, LiteLLM, Bedrock, httpx, anything. Works with any agent framework — LangChain, LangGraph, Agno, Strands, or your own code. ### Any agent — proxy (zero code changes) @@ -140,7 +140,7 @@ Gives your AI tool three MCP tools: `headroom_compress`, `headroom_retrieve`, `h | **LiteLLM** | Callback | `litellm.callbacks = [HeadroomCallback()]` | | **Any Python proxy** | ASGI Middleware | `app.add_middleware(CompressionMiddleware)` | | **Agno agents** | Wrap model | `HeadroomAgnoModel(your_model)` | -| **LangChain** | Wrap model | `HeadroomChatModel(your_llm)` *(experimental)* | +| **LangChain** | Wrap model | `HeadroomChatModel(your_llm)` | | **Claude Code** | Wrap | `headroom wrap claude` | | **Codex / Aider** | Wrap | `headroom wrap codex` or `headroom wrap aider` | @@ -285,7 +285,7 @@ Context compression is a new space. Here's how the approaches differ: | | Approach | Scope | Deploy as | Framework integrations | Data stays local? | Reversible | |---|---|---|---|---|---|---| -| **Headroom** | Multi-algorithm compression | All context (tool outputs, DB reads, RAG, files, logs, history) | Proxy, Python library, ASGI middleware, or callback | LangChain, Agno, LiteLLM, Strands, MCP | Yes (OSS) | Yes (CCR) | +| **Headroom** | Multi-algorithm compression | All context (tool outputs, DB reads, RAG, files, logs, history) | Proxy, Python library, ASGI middleware, or callback | LangChain, LangGraph, Agno, Strands, LiteLLM, MCP | Yes (OSS) | Yes (CCR) | | **[RTK](https://github.com/rtk-ai/rtk)** | CLI command rewriter | Shell command outputs | CLI wrapper | None | Yes (OSS) | No | | **[Compresr](https://compresr.ai)** | Cloud compression API | Text sent to their API | API call | None | No | No | | **[Token Company](https://thetokencompany.ai)** | Cloud compression API | Text sent to their API | API call | None | No | No | @@ -338,7 +338,7 @@ Context compression is a new space. Here's how the approaches differ: | Agno | **Stable** | [Agno Guide](docs/agno.md) | | MCP (Claude Code, Cursor, etc.) | **Stable** | [MCP Guide](docs/mcp.md) | | Strands | **Stable** | [Strands Guide](docs/strands.md) | -| LangChain | **Experimental** | [LangChain Guide](docs/langchain.md) | +| LangChain | **Stable** | [LangChain Guide](docs/langchain.md) | --- diff --git a/docs/LATENCY_BENCHMARKS.md b/docs/LATENCY_BENCHMARKS.md index 7ac916395..66560edcb 100644 --- a/docs/LATENCY_BENCHMARKS.md +++ b/docs/LATENCY_BENCHMARKS.md @@ -11,6 +11,8 @@ Generated: 2026-02-24 01:11 UTC - **Python**: 3.11.11 - **Headroom**: v0.3.7 +> **Note:** These benchmarks were captured on v0.3.7. Since then, v0.5.6 added parallel message compression, eliminated redundant token counting, and optimized hot-path hashing. Expect lower latency on current versions. Re-benchmarking is planned. + ## TL;DR - Average compression: **93%** token reduction diff --git a/docs/index.md b/docs/index.md index a444c2ef7..0f51a9cce 100644 --- a/docs/index.md +++ b/docs/index.md @@ -33,15 +33,16 @@ That's it. Your existing code works unchanged, with 40-90% fewer tokens. ### Option 2: Python SDK ```python -from headroom import Headroom +from headroom import compress -hr = Headroom() +# Compress messages before sending to LLM +result = compress(messages, model="claude-sonnet-4-5-20250929") +response = client.messages.create( + model="claude-sonnet-4-5-20250929", + messages=result.messages, +) -# Compress tool output before sending to LLM -compressed = hr.compress(large_tool_output) - -# If LLM needs the full data, retrieve it -original = hr.retrieve(compressed) +print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})") ``` --- @@ -112,6 +113,19 @@ Your App → Headroom → LLM Provider agent = Agent(model=model) ``` +=== "Strands" + + ```python + from strands import Agent + from strands.models.bedrock import BedrockModel + from headroom.integrations.strands import HeadroomStrandsModel + + model = HeadroomStrandsModel(wrapped_model=BedrockModel( + model_id="us.anthropic.claude-sonnet-4-20250514-v1:0" + )) + agent = Agent(model=model) + ``` + === "AWS Bedrock" ```bash diff --git a/docs/integration-guide.md b/docs/integration-guide.md index 659649c30..fe3c8838d 100644 --- a/docs/integration-guide.md +++ b/docs/integration-guide.md @@ -235,7 +235,7 @@ See [Agno Guide](agno.md) for hooks, multi-provider, and streaming. ## LangChain -> **Experimental.** Core compression works. Streaming callbacks and async chains are still being tested. +Full integration with LangChain — chat models, memory, retrievers, tool wrappers, and streaming. ```python from langchain_openai import ChatOpenAI @@ -289,4 +289,4 @@ Headroom stores originals in CCR (Compress-Cache-Retrieve). The LLM can call `he Yes. Compression happens before the request is sent. Streaming responses are unaffected. **Q: How much latency does it add?** -1-5ms for compression. The token savings typically save more time on the LLM side than compression adds. +15-200ms depending on content size and type. Small JSON arrays take ~15ms, large tool outputs take 100-200ms. The token savings typically save far more time on the LLM side than compression adds — a 50% token reduction on a Sonnet call saves seconds of generation time. See [Latency Benchmarks](LATENCY_BENCHMARKS.md) for real numbers. diff --git a/docs/strands.md b/docs/strands.md new file mode 100644 index 000000000..171852a39 --- /dev/null +++ b/docs/strands.md @@ -0,0 +1,175 @@ +# Strands Integration + +Headroom integrates with [Strands Agents](https://github.com/strands-agents/sdk-python) to provide automatic context optimization. Two integration patterns: wrap the model, or hook into tool calls. + +--- + +## Installation + +```bash +pip install headroom-ai strands-agents +``` + +--- + +## Quick Start + +```python +from strands import Agent +from strands.models.bedrock import BedrockModel +from headroom.integrations.strands import HeadroomStrandsModel + +# Wrap your model +model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0") +optimized = HeadroomStrandsModel(wrapped_model=model) + +# Create agent as usual +agent = Agent(model=optimized) +response = agent("Investigate the production incident") + +# Check savings +print(f"Tokens saved: {optimized.total_tokens_saved}") +``` + +Every API call the agent makes — including tool result round-trips — gets compressed automatically. + +--- + +## Integration Patterns + +### 1. Model Wrapping + +Wraps the Strands `Model` interface. Every call to `stream()` compresses the messages before they hit the provider. + +```python +from strands.models.bedrock import BedrockModel +from headroom.integrations.strands import HeadroomStrandsModel + +model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0") +optimized = HeadroomStrandsModel(wrapped_model=model) + +# Streaming works identically +agent = Agent(model=optimized) +response = agent("Analyze these logs") +``` + +With custom config: + +```python +from headroom import HeadroomConfig + +config = HeadroomConfig() +optimized = HeadroomStrandsModel(wrapped_model=model, config=config) +``` + +### 2. Hook Provider (Tool Output Compression) + +Compresses tool call results via Strands' hook system. Uses SmartCrusher on JSON arrays returned by tools. + +```python +from strands import Agent +from strands.models.bedrock import BedrockModel +from headroom.integrations.strands import HeadroomHookProvider + +model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0") +hooks = HeadroomHookProvider( + compress_tool_outputs=True, + min_tokens_to_compress=200, + preserve_errors=True, +) + +agent = Agent(model=model, hooks=[hooks]) +response = agent("Search the database for recent failures") + +# Check tool compression savings +print(f"Tokens saved by hooks: {hooks.total_tokens_saved}") +``` + +The hook preserves: + +- Error items (error indicators, exceptions) +- Anomalous values (statistical outliers) +- Items matching the user's query context +- First/last items for boundary context + +### 3. Both Together + +Model wrapping compresses conversation history. Hooks compress individual tool results. Use both for maximum savings. + +```python +from headroom.integrations.strands import HeadroomStrandsModel, HeadroomHookProvider + +optimized = HeadroomStrandsModel(wrapped_model=model) +hooks = HeadroomHookProvider(compress_tool_outputs=True) + +agent = Agent(model=optimized, hooks=[hooks]) +``` + +--- + +## Structured Output + +HeadroomStrandsModel supports Strands' structured output feature: + +```python +from pydantic import BaseModel + +class Analysis(BaseModel): + severity: str + root_cause: str + recommendation: str + +result = optimized.structured_output(Analysis, messages) +``` + +--- + +## Metrics + +```python +# Per-request metrics +for m in optimized.metrics_history: + print(f" {m.tokens_before} → {m.tokens_after} ({m.tokens_saved} saved)") + +# Running total +print(f"Total saved: {optimized.total_tokens_saved}") +``` + +--- + +## How It Works + +``` +Agent decides to call tool + │ + ▼ +Tool executes, returns result + │ + ▼ +HeadroomHookProvider (optional) + compresses tool result JSON + │ + ▼ +Agent builds next API request + │ + ▼ +HeadroomStrandsModel.stream() + compresses full message list + │ + ▼ +Provider API (Bedrock, etc.) +``` + +The model wrapper uses Headroom's full pipeline (CacheAligner → ContentRouter → IntelligentContext). The hook provider uses SmartCrusher directly for fast JSON compression of individual tool results. + +--- + +## Supported Providers + +HeadroomStrandsModel auto-detects the provider from the wrapped model: + +| Strands Model | Provider Detected | +|--------------|-------------------| +| `BedrockModel` | Anthropic (via Bedrock) | +| `OllamaModel` | OpenAI-compatible | +| Custom `Model` | Falls back to estimation | diff --git a/mkdocs.yml b/mkdocs.yml index 9d286cf1f..717a723a8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -68,8 +68,11 @@ nav: - CCR (Compress-Cache-Retrieve): ccr.md - Configuration: configuration.md - Integrations: + - Integration Guide: integration-guide.md - LangChain: langchain.md - Agno: agno.md + - Strands: strands.md + - MCP Tools: mcp.md - Advanced: - LLMLingua: llmlingua.md - Text Compression: text-compression.md