mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
Fix docs to match implementation: remove false claims, add Strands guide
- Remove CrewAI and OpenAI Agents SDK claims (not implemented) - Upgrade LangChain from "Experimental" to "Stable" (fully implemented) - Fix latency FAQ: "1-5ms" → accurate "15-200ms" with cost-benefit context - Create docs/strands.md (README linked to it but file didn't exist) - Align docs/index.md with compress() function API (was showing stale class API) - Add Strands, MCP, Integration Guide to mkdocs nav - Note stale v0.3.7 benchmarks in LATENCY_BENCHMARKS.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
cfd44b3f6a
commit
c61ba522a3
6 changed files with 208 additions and 14 deletions
10
README.md
10
README.md
|
|
@ -7,7 +7,7 @@
|
||||||
Every tool call, DB query, file read, and RAG retrieval your agent makes is 70-95% boilerplate.<br>
|
Every tool call, DB query, file read, and RAG retrieval your agent makes is 70-95% boilerplate.<br>
|
||||||
Headroom compresses it away before it hits the model.<br><br>
|
Headroom compresses it away before it hits the model.<br><br>
|
||||||
Works with <b>any agent</b> — coding agents (Claude Code, Codex, Cursor, Aider), custom agents<br>
|
Works with <b>any agent</b> — coding agents (Claude Code, Codex, Cursor, Aider), custom agents<br>
|
||||||
(LangChain, LangGraph, CrewAI, Agno, OpenAI Agents SDK), or your own Python code.
|
(LangChain, LangGraph, Agno, Strands), or your own Python code.
|
||||||
</p>
|
</p>
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
|
|
@ -83,7 +83,7 @@ response = client.messages.create(model="claude-sonnet-4-5-20250929", messages=r
|
||||||
print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")
|
print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")
|
||||||
```
|
```
|
||||||
|
|
||||||
Works with any Python LLM client — Anthropic, OpenAI, LiteLLM, Bedrock, httpx, anything. Works with any agent framework — LangChain, LangGraph, CrewAI, Agno, OpenAI Agents SDK, or your own code.
|
Works with any Python LLM client — Anthropic, OpenAI, LiteLLM, Bedrock, httpx, anything. Works with any agent framework — LangChain, LangGraph, Agno, Strands, or your own code.
|
||||||
|
|
||||||
### Any agent — proxy (zero code changes)
|
### Any agent — proxy (zero code changes)
|
||||||
|
|
||||||
|
|
@ -140,7 +140,7 @@ Gives your AI tool three MCP tools: `headroom_compress`, `headroom_retrieve`, `h
|
||||||
| **LiteLLM** | Callback | `litellm.callbacks = [HeadroomCallback()]` |
|
| **LiteLLM** | Callback | `litellm.callbacks = [HeadroomCallback()]` |
|
||||||
| **Any Python proxy** | ASGI Middleware | `app.add_middleware(CompressionMiddleware)` |
|
| **Any Python proxy** | ASGI Middleware | `app.add_middleware(CompressionMiddleware)` |
|
||||||
| **Agno agents** | Wrap model | `HeadroomAgnoModel(your_model)` |
|
| **Agno agents** | Wrap model | `HeadroomAgnoModel(your_model)` |
|
||||||
| **LangChain** | Wrap model | `HeadroomChatModel(your_llm)` *(experimental)* |
|
| **LangChain** | Wrap model | `HeadroomChatModel(your_llm)` |
|
||||||
| **Claude Code** | Wrap | `headroom wrap claude` |
|
| **Claude Code** | Wrap | `headroom wrap claude` |
|
||||||
| **Codex / Aider** | Wrap | `headroom wrap codex` or `headroom wrap aider` |
|
| **Codex / Aider** | Wrap | `headroom wrap codex` or `headroom wrap aider` |
|
||||||
|
|
||||||
|
|
@ -285,7 +285,7 @@ Context compression is a new space. Here's how the approaches differ:
|
||||||
|
|
||||||
| | Approach | Scope | Deploy as | Framework integrations | Data stays local? | Reversible |
|
| | Approach | Scope | Deploy as | Framework integrations | Data stays local? | Reversible |
|
||||||
|---|---|---|---|---|---|---|
|
|---|---|---|---|---|---|---|
|
||||||
| **Headroom** | Multi-algorithm compression | All context (tool outputs, DB reads, RAG, files, logs, history) | Proxy, Python library, ASGI middleware, or callback | LangChain, Agno, LiteLLM, Strands, MCP | Yes (OSS) | Yes (CCR) |
|
| **Headroom** | Multi-algorithm compression | All context (tool outputs, DB reads, RAG, files, logs, history) | Proxy, Python library, ASGI middleware, or callback | LangChain, LangGraph, Agno, Strands, LiteLLM, MCP | Yes (OSS) | Yes (CCR) |
|
||||||
| **[RTK](https://github.com/rtk-ai/rtk)** | CLI command rewriter | Shell command outputs | CLI wrapper | None | Yes (OSS) | No |
|
| **[RTK](https://github.com/rtk-ai/rtk)** | CLI command rewriter | Shell command outputs | CLI wrapper | None | Yes (OSS) | No |
|
||||||
| **[Compresr](https://compresr.ai)** | Cloud compression API | Text sent to their API | API call | None | No | No |
|
| **[Compresr](https://compresr.ai)** | Cloud compression API | Text sent to their API | API call | None | No | No |
|
||||||
| **[Token Company](https://thetokencompany.ai)** | Cloud compression API | Text sent to their API | API call | None | No | No |
|
| **[Token Company](https://thetokencompany.ai)** | Cloud compression API | Text sent to their API | API call | None | No | No |
|
||||||
|
|
@ -338,7 +338,7 @@ Context compression is a new space. Here's how the approaches differ:
|
||||||
| Agno | **Stable** | [Agno Guide](docs/agno.md) |
|
| Agno | **Stable** | [Agno Guide](docs/agno.md) |
|
||||||
| MCP (Claude Code, Cursor, etc.) | **Stable** | [MCP Guide](docs/mcp.md) |
|
| MCP (Claude Code, Cursor, etc.) | **Stable** | [MCP Guide](docs/mcp.md) |
|
||||||
| Strands | **Stable** | [Strands Guide](docs/strands.md) |
|
| Strands | **Stable** | [Strands Guide](docs/strands.md) |
|
||||||
| LangChain | **Experimental** | [LangChain Guide](docs/langchain.md) |
|
| LangChain | **Stable** | [LangChain Guide](docs/langchain.md) |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,8 @@ Generated: 2026-02-24 01:11 UTC
|
||||||
- **Python**: 3.11.11
|
- **Python**: 3.11.11
|
||||||
- **Headroom**: v0.3.7
|
- **Headroom**: v0.3.7
|
||||||
|
|
||||||
|
> **Note:** These benchmarks were captured on v0.3.7. Since then, v0.5.6 added parallel message compression, eliminated redundant token counting, and optimized hot-path hashing. Expect lower latency on current versions. Re-benchmarking is planned.
|
||||||
|
|
||||||
## TL;DR
|
## TL;DR
|
||||||
|
|
||||||
- Average compression: **93%** token reduction
|
- Average compression: **93%** token reduction
|
||||||
|
|
|
||||||
|
|
@ -33,15 +33,16 @@ That's it. Your existing code works unchanged, with 40-90% fewer tokens.
|
||||||
### Option 2: Python SDK
|
### Option 2: Python SDK
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from headroom import Headroom
|
from headroom import compress
|
||||||
|
|
||||||
hr = Headroom()
|
# Compress messages before sending to LLM
|
||||||
|
result = compress(messages, model="claude-sonnet-4-5-20250929")
|
||||||
|
response = client.messages.create(
|
||||||
|
model="claude-sonnet-4-5-20250929",
|
||||||
|
messages=result.messages,
|
||||||
|
)
|
||||||
|
|
||||||
# Compress tool output before sending to LLM
|
print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")
|
||||||
compressed = hr.compress(large_tool_output)
|
|
||||||
|
|
||||||
# If LLM needs the full data, retrieve it
|
|
||||||
original = hr.retrieve(compressed)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
@ -112,6 +113,19 @@ Your App → Headroom → LLM Provider
|
||||||
agent = Agent(model=model)
|
agent = Agent(model=model)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
=== "Strands"
|
||||||
|
|
||||||
|
```python
|
||||||
|
from strands import Agent
|
||||||
|
from strands.models.bedrock import BedrockModel
|
||||||
|
from headroom.integrations.strands import HeadroomStrandsModel
|
||||||
|
|
||||||
|
model = HeadroomStrandsModel(wrapped_model=BedrockModel(
|
||||||
|
model_id="us.anthropic.claude-sonnet-4-20250514-v1:0"
|
||||||
|
))
|
||||||
|
agent = Agent(model=model)
|
||||||
|
```
|
||||||
|
|
||||||
=== "AWS Bedrock"
|
=== "AWS Bedrock"
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
|
||||||
|
|
@ -235,7 +235,7 @@ See [Agno Guide](agno.md) for hooks, multi-provider, and streaming.
|
||||||
|
|
||||||
## LangChain
|
## LangChain
|
||||||
|
|
||||||
> **Experimental.** Core compression works. Streaming callbacks and async chains are still being tested.
|
Full integration with LangChain — chat models, memory, retrievers, tool wrappers, and streaming.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from langchain_openai import ChatOpenAI
|
from langchain_openai import ChatOpenAI
|
||||||
|
|
@ -289,4 +289,4 @@ Headroom stores originals in CCR (Compress-Cache-Retrieve). The LLM can call `he
|
||||||
Yes. Compression happens before the request is sent. Streaming responses are unaffected.
|
Yes. Compression happens before the request is sent. Streaming responses are unaffected.
|
||||||
|
|
||||||
**Q: How much latency does it add?**
|
**Q: How much latency does it add?**
|
||||||
1-5ms for compression. The token savings typically save more time on the LLM side than compression adds.
|
15-200ms depending on content size and type. Small JSON arrays take ~15ms, large tool outputs take 100-200ms. The token savings typically save far more time on the LLM side than compression adds — a 50% token reduction on a Sonnet call saves seconds of generation time. See [Latency Benchmarks](LATENCY_BENCHMARKS.md) for real numbers.
|
||||||
|
|
|
||||||
175
docs/strands.md
Normal file
175
docs/strands.md
Normal file
|
|
@ -0,0 +1,175 @@
|
||||||
|
# Strands Integration
|
||||||
|
|
||||||
|
Headroom integrates with [Strands Agents](https://github.com/strands-agents/sdk-python) to provide automatic context optimization. Two integration patterns: wrap the model, or hook into tool calls.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install headroom-ai strands-agents
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
```python
|
||||||
|
from strands import Agent
|
||||||
|
from strands.models.bedrock import BedrockModel
|
||||||
|
from headroom.integrations.strands import HeadroomStrandsModel
|
||||||
|
|
||||||
|
# Wrap your model
|
||||||
|
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
|
||||||
|
optimized = HeadroomStrandsModel(wrapped_model=model)
|
||||||
|
|
||||||
|
# Create agent as usual
|
||||||
|
agent = Agent(model=optimized)
|
||||||
|
response = agent("Investigate the production incident")
|
||||||
|
|
||||||
|
# Check savings
|
||||||
|
print(f"Tokens saved: {optimized.total_tokens_saved}")
|
||||||
|
```
|
||||||
|
|
||||||
|
Every API call the agent makes — including tool result round-trips — gets compressed automatically.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Patterns
|
||||||
|
|
||||||
|
### 1. Model Wrapping
|
||||||
|
|
||||||
|
Wraps the Strands `Model` interface. Every call to `stream()` compresses the messages before they hit the provider.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from strands.models.bedrock import BedrockModel
|
||||||
|
from headroom.integrations.strands import HeadroomStrandsModel
|
||||||
|
|
||||||
|
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
|
||||||
|
optimized = HeadroomStrandsModel(wrapped_model=model)
|
||||||
|
|
||||||
|
# Streaming works identically
|
||||||
|
agent = Agent(model=optimized)
|
||||||
|
response = agent("Analyze these logs")
|
||||||
|
```
|
||||||
|
|
||||||
|
With custom config:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from headroom import HeadroomConfig
|
||||||
|
|
||||||
|
config = HeadroomConfig()
|
||||||
|
optimized = HeadroomStrandsModel(wrapped_model=model, config=config)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Hook Provider (Tool Output Compression)
|
||||||
|
|
||||||
|
Compresses tool call results via Strands' hook system. Uses SmartCrusher on JSON arrays returned by tools.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from strands import Agent
|
||||||
|
from strands.models.bedrock import BedrockModel
|
||||||
|
from headroom.integrations.strands import HeadroomHookProvider
|
||||||
|
|
||||||
|
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
|
||||||
|
hooks = HeadroomHookProvider(
|
||||||
|
compress_tool_outputs=True,
|
||||||
|
min_tokens_to_compress=200,
|
||||||
|
preserve_errors=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
agent = Agent(model=model, hooks=[hooks])
|
||||||
|
response = agent("Search the database for recent failures")
|
||||||
|
|
||||||
|
# Check tool compression savings
|
||||||
|
print(f"Tokens saved by hooks: {hooks.total_tokens_saved}")
|
||||||
|
```
|
||||||
|
|
||||||
|
The hook preserves:
|
||||||
|
|
||||||
|
- Error items (error indicators, exceptions)
|
||||||
|
- Anomalous values (statistical outliers)
|
||||||
|
- Items matching the user's query context
|
||||||
|
- First/last items for boundary context
|
||||||
|
|
||||||
|
### 3. Both Together
|
||||||
|
|
||||||
|
Model wrapping compresses conversation history. Hooks compress individual tool results. Use both for maximum savings.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from headroom.integrations.strands import HeadroomStrandsModel, HeadroomHookProvider
|
||||||
|
|
||||||
|
optimized = HeadroomStrandsModel(wrapped_model=model)
|
||||||
|
hooks = HeadroomHookProvider(compress_tool_outputs=True)
|
||||||
|
|
||||||
|
agent = Agent(model=optimized, hooks=[hooks])
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Structured Output
|
||||||
|
|
||||||
|
HeadroomStrandsModel supports Strands' structured output feature:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
class Analysis(BaseModel):
|
||||||
|
severity: str
|
||||||
|
root_cause: str
|
||||||
|
recommendation: str
|
||||||
|
|
||||||
|
result = optimized.structured_output(Analysis, messages)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Metrics
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Per-request metrics
|
||||||
|
for m in optimized.metrics_history:
|
||||||
|
print(f" {m.tokens_before} → {m.tokens_after} ({m.tokens_saved} saved)")
|
||||||
|
|
||||||
|
# Running total
|
||||||
|
print(f"Total saved: {optimized.total_tokens_saved}")
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How It Works
|
||||||
|
|
||||||
|
```
|
||||||
|
Agent decides to call tool
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Tool executes, returns result
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
HeadroomHookProvider (optional)
|
||||||
|
compresses tool result JSON
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Agent builds next API request
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
HeadroomStrandsModel.stream()
|
||||||
|
compresses full message list
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Provider API (Bedrock, etc.)
|
||||||
|
```
|
||||||
|
|
||||||
|
The model wrapper uses Headroom's full pipeline (CacheAligner → ContentRouter → IntelligentContext). The hook provider uses SmartCrusher directly for fast JSON compression of individual tool results.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Supported Providers
|
||||||
|
|
||||||
|
HeadroomStrandsModel auto-detects the provider from the wrapped model:
|
||||||
|
|
||||||
|
| Strands Model | Provider Detected |
|
||||||
|
|--------------|-------------------|
|
||||||
|
| `BedrockModel` | Anthropic (via Bedrock) |
|
||||||
|
| `OllamaModel` | OpenAI-compatible |
|
||||||
|
| Custom `Model` | Falls back to estimation |
|
||||||
|
|
@ -68,8 +68,11 @@ nav:
|
||||||
- CCR (Compress-Cache-Retrieve): ccr.md
|
- CCR (Compress-Cache-Retrieve): ccr.md
|
||||||
- Configuration: configuration.md
|
- Configuration: configuration.md
|
||||||
- Integrations:
|
- Integrations:
|
||||||
|
- Integration Guide: integration-guide.md
|
||||||
- LangChain: langchain.md
|
- LangChain: langchain.md
|
||||||
- Agno: agno.md
|
- Agno: agno.md
|
||||||
|
- Strands: strands.md
|
||||||
|
- MCP Tools: mcp.md
|
||||||
- Advanced:
|
- Advanced:
|
||||||
- LLMLingua: llmlingua.md
|
- LLMLingua: llmlingua.md
|
||||||
- Text Compression: text-compression.md
|
- Text Compression: text-compression.md
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue