mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
Fix docs to match implementation: remove false claims, add Strands guide
- Remove CrewAI and OpenAI Agents SDK claims (not implemented) - Upgrade LangChain from "Experimental" to "Stable" (fully implemented) - Fix latency FAQ: "1-5ms" → accurate "15-200ms" with cost-benefit context - Create docs/strands.md (README linked to it but file didn't exist) - Align docs/index.md with compress() function API (was showing stale class API) - Add Strands, MCP, Integration Guide to mkdocs nav - Note stale v0.3.7 benchmarks in LATENCY_BENCHMARKS.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
chopratejas
parent
cfd44b3f6a
commit
c61ba522a3
6 changed files with 208 additions and 14 deletions
10
README.md
10
README.md
|
|
@ -7,7 +7,7 @@
|
|||
Every tool call, DB query, file read, and RAG retrieval your agent makes is 70-95% boilerplate.<br>
|
||||
Headroom compresses it away before it hits the model.<br><br>
|
||||
Works with <b>any agent</b> — coding agents (Claude Code, Codex, Cursor, Aider), custom agents<br>
|
||||
(LangChain, LangGraph, CrewAI, Agno, OpenAI Agents SDK), or your own Python code.
|
||||
(LangChain, LangGraph, Agno, Strands), or your own Python code.
|
||||
</p>
|
||||
</p>
|
||||
|
||||
|
|
@ -83,7 +83,7 @@ response = client.messages.create(model="claude-sonnet-4-5-20250929", messages=r
|
|||
print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")
|
||||
```
|
||||
|
||||
Works with any Python LLM client — Anthropic, OpenAI, LiteLLM, Bedrock, httpx, anything. Works with any agent framework — LangChain, LangGraph, CrewAI, Agno, OpenAI Agents SDK, or your own code.
|
||||
Works with any Python LLM client — Anthropic, OpenAI, LiteLLM, Bedrock, httpx, anything. Works with any agent framework — LangChain, LangGraph, Agno, Strands, or your own code.
|
||||
|
||||
### Any agent — proxy (zero code changes)
|
||||
|
||||
|
|
@ -140,7 +140,7 @@ Gives your AI tool three MCP tools: `headroom_compress`, `headroom_retrieve`, `h
|
|||
| **LiteLLM** | Callback | `litellm.callbacks = [HeadroomCallback()]` |
|
||||
| **Any Python proxy** | ASGI Middleware | `app.add_middleware(CompressionMiddleware)` |
|
||||
| **Agno agents** | Wrap model | `HeadroomAgnoModel(your_model)` |
|
||||
| **LangChain** | Wrap model | `HeadroomChatModel(your_llm)` *(experimental)* |
|
||||
| **LangChain** | Wrap model | `HeadroomChatModel(your_llm)` |
|
||||
| **Claude Code** | Wrap | `headroom wrap claude` |
|
||||
| **Codex / Aider** | Wrap | `headroom wrap codex` or `headroom wrap aider` |
|
||||
|
||||
|
|
@ -285,7 +285,7 @@ Context compression is a new space. Here's how the approaches differ:
|
|||
|
||||
| | Approach | Scope | Deploy as | Framework integrations | Data stays local? | Reversible |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **Headroom** | Multi-algorithm compression | All context (tool outputs, DB reads, RAG, files, logs, history) | Proxy, Python library, ASGI middleware, or callback | LangChain, Agno, LiteLLM, Strands, MCP | Yes (OSS) | Yes (CCR) |
|
||||
| **Headroom** | Multi-algorithm compression | All context (tool outputs, DB reads, RAG, files, logs, history) | Proxy, Python library, ASGI middleware, or callback | LangChain, LangGraph, Agno, Strands, LiteLLM, MCP | Yes (OSS) | Yes (CCR) |
|
||||
| **[RTK](https://github.com/rtk-ai/rtk)** | CLI command rewriter | Shell command outputs | CLI wrapper | None | Yes (OSS) | No |
|
||||
| **[Compresr](https://compresr.ai)** | Cloud compression API | Text sent to their API | API call | None | No | No |
|
||||
| **[Token Company](https://thetokencompany.ai)** | Cloud compression API | Text sent to their API | API call | None | No | No |
|
||||
|
|
@ -338,7 +338,7 @@ Context compression is a new space. Here's how the approaches differ:
|
|||
| Agno | **Stable** | [Agno Guide](docs/agno.md) |
|
||||
| MCP (Claude Code, Cursor, etc.) | **Stable** | [MCP Guide](docs/mcp.md) |
|
||||
| Strands | **Stable** | [Strands Guide](docs/strands.md) |
|
||||
| LangChain | **Experimental** | [LangChain Guide](docs/langchain.md) |
|
||||
| LangChain | **Stable** | [LangChain Guide](docs/langchain.md) |
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -11,6 +11,8 @@ Generated: 2026-02-24 01:11 UTC
|
|||
- **Python**: 3.11.11
|
||||
- **Headroom**: v0.3.7
|
||||
|
||||
> **Note:** These benchmarks were captured on v0.3.7. Since then, v0.5.6 added parallel message compression, eliminated redundant token counting, and optimized hot-path hashing. Expect lower latency on current versions. Re-benchmarking is planned.
|
||||
|
||||
## TL;DR
|
||||
|
||||
- Average compression: **93%** token reduction
|
||||
|
|
|
|||
|
|
@ -33,15 +33,16 @@ That's it. Your existing code works unchanged, with 40-90% fewer tokens.
|
|||
### Option 2: Python SDK
|
||||
|
||||
```python
|
||||
from headroom import Headroom
|
||||
from headroom import compress
|
||||
|
||||
hr = Headroom()
|
||||
# Compress messages before sending to LLM
|
||||
result = compress(messages, model="claude-sonnet-4-5-20250929")
|
||||
response = client.messages.create(
|
||||
model="claude-sonnet-4-5-20250929",
|
||||
messages=result.messages,
|
||||
)
|
||||
|
||||
# Compress tool output before sending to LLM
|
||||
compressed = hr.compress(large_tool_output)
|
||||
|
||||
# If LLM needs the full data, retrieve it
|
||||
original = hr.retrieve(compressed)
|
||||
print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -112,6 +113,19 @@ Your App → Headroom → LLM Provider
|
|||
agent = Agent(model=model)
|
||||
```
|
||||
|
||||
=== "Strands"
|
||||
|
||||
```python
|
||||
from strands import Agent
|
||||
from strands.models.bedrock import BedrockModel
|
||||
from headroom.integrations.strands import HeadroomStrandsModel
|
||||
|
||||
model = HeadroomStrandsModel(wrapped_model=BedrockModel(
|
||||
model_id="us.anthropic.claude-sonnet-4-20250514-v1:0"
|
||||
))
|
||||
agent = Agent(model=model)
|
||||
```
|
||||
|
||||
=== "AWS Bedrock"
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -235,7 +235,7 @@ See [Agno Guide](agno.md) for hooks, multi-provider, and streaming.
|
|||
|
||||
## LangChain
|
||||
|
||||
> **Experimental.** Core compression works. Streaming callbacks and async chains are still being tested.
|
||||
Full integration with LangChain — chat models, memory, retrievers, tool wrappers, and streaming.
|
||||
|
||||
```python
|
||||
from langchain_openai import ChatOpenAI
|
||||
|
|
@ -289,4 +289,4 @@ Headroom stores originals in CCR (Compress-Cache-Retrieve). The LLM can call `he
|
|||
Yes. Compression happens before the request is sent. Streaming responses are unaffected.
|
||||
|
||||
**Q: How much latency does it add?**
|
||||
1-5ms for compression. The token savings typically save more time on the LLM side than compression adds.
|
||||
15-200ms depending on content size and type. Small JSON arrays take ~15ms, large tool outputs take 100-200ms. The token savings typically save far more time on the LLM side than compression adds — a 50% token reduction on a Sonnet call saves seconds of generation time. See [Latency Benchmarks](LATENCY_BENCHMARKS.md) for real numbers.
|
||||
|
|
|
|||
175
docs/strands.md
Normal file
175
docs/strands.md
Normal file
|
|
@ -0,0 +1,175 @@
|
|||
# Strands Integration
|
||||
|
||||
Headroom integrates with [Strands Agents](https://github.com/strands-agents/sdk-python) to provide automatic context optimization. Two integration patterns: wrap the model, or hook into tool calls.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install headroom-ai strands-agents
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
from strands import Agent
|
||||
from strands.models.bedrock import BedrockModel
|
||||
from headroom.integrations.strands import HeadroomStrandsModel
|
||||
|
||||
# Wrap your model
|
||||
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
|
||||
optimized = HeadroomStrandsModel(wrapped_model=model)
|
||||
|
||||
# Create agent as usual
|
||||
agent = Agent(model=optimized)
|
||||
response = agent("Investigate the production incident")
|
||||
|
||||
# Check savings
|
||||
print(f"Tokens saved: {optimized.total_tokens_saved}")
|
||||
```
|
||||
|
||||
Every API call the agent makes — including tool result round-trips — gets compressed automatically.
|
||||
|
||||
---
|
||||
|
||||
## Integration Patterns
|
||||
|
||||
### 1. Model Wrapping
|
||||
|
||||
Wraps the Strands `Model` interface. Every call to `stream()` compresses the messages before they hit the provider.
|
||||
|
||||
```python
|
||||
from strands.models.bedrock import BedrockModel
|
||||
from headroom.integrations.strands import HeadroomStrandsModel
|
||||
|
||||
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
|
||||
optimized = HeadroomStrandsModel(wrapped_model=model)
|
||||
|
||||
# Streaming works identically
|
||||
agent = Agent(model=optimized)
|
||||
response = agent("Analyze these logs")
|
||||
```
|
||||
|
||||
With custom config:
|
||||
|
||||
```python
|
||||
from headroom import HeadroomConfig
|
||||
|
||||
config = HeadroomConfig()
|
||||
optimized = HeadroomStrandsModel(wrapped_model=model, config=config)
|
||||
```
|
||||
|
||||
### 2. Hook Provider (Tool Output Compression)
|
||||
|
||||
Compresses tool call results via Strands' hook system. Uses SmartCrusher on JSON arrays returned by tools.
|
||||
|
||||
```python
|
||||
from strands import Agent
|
||||
from strands.models.bedrock import BedrockModel
|
||||
from headroom.integrations.strands import HeadroomHookProvider
|
||||
|
||||
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
|
||||
hooks = HeadroomHookProvider(
|
||||
compress_tool_outputs=True,
|
||||
min_tokens_to_compress=200,
|
||||
preserve_errors=True,
|
||||
)
|
||||
|
||||
agent = Agent(model=model, hooks=[hooks])
|
||||
response = agent("Search the database for recent failures")
|
||||
|
||||
# Check tool compression savings
|
||||
print(f"Tokens saved by hooks: {hooks.total_tokens_saved}")
|
||||
```
|
||||
|
||||
The hook preserves:
|
||||
|
||||
- Error items (error indicators, exceptions)
|
||||
- Anomalous values (statistical outliers)
|
||||
- Items matching the user's query context
|
||||
- First/last items for boundary context
|
||||
|
||||
### 3. Both Together
|
||||
|
||||
Model wrapping compresses conversation history. Hooks compress individual tool results. Use both for maximum savings.
|
||||
|
||||
```python
|
||||
from headroom.integrations.strands import HeadroomStrandsModel, HeadroomHookProvider
|
||||
|
||||
optimized = HeadroomStrandsModel(wrapped_model=model)
|
||||
hooks = HeadroomHookProvider(compress_tool_outputs=True)
|
||||
|
||||
agent = Agent(model=optimized, hooks=[hooks])
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Structured Output
|
||||
|
||||
HeadroomStrandsModel supports Strands' structured output feature:
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel
|
||||
|
||||
class Analysis(BaseModel):
|
||||
severity: str
|
||||
root_cause: str
|
||||
recommendation: str
|
||||
|
||||
result = optimized.structured_output(Analysis, messages)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Metrics
|
||||
|
||||
```python
|
||||
# Per-request metrics
|
||||
for m in optimized.metrics_history:
|
||||
print(f" {m.tokens_before} → {m.tokens_after} ({m.tokens_saved} saved)")
|
||||
|
||||
# Running total
|
||||
print(f"Total saved: {optimized.total_tokens_saved}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
Agent decides to call tool
|
||||
│
|
||||
▼
|
||||
Tool executes, returns result
|
||||
│
|
||||
▼
|
||||
HeadroomHookProvider (optional)
|
||||
compresses tool result JSON
|
||||
│
|
||||
▼
|
||||
Agent builds next API request
|
||||
│
|
||||
▼
|
||||
HeadroomStrandsModel.stream()
|
||||
compresses full message list
|
||||
│
|
||||
▼
|
||||
Provider API (Bedrock, etc.)
|
||||
```
|
||||
|
||||
The model wrapper uses Headroom's full pipeline (CacheAligner → ContentRouter → IntelligentContext). The hook provider uses SmartCrusher directly for fast JSON compression of individual tool results.
|
||||
|
||||
---
|
||||
|
||||
## Supported Providers
|
||||
|
||||
HeadroomStrandsModel auto-detects the provider from the wrapped model:
|
||||
|
||||
| Strands Model | Provider Detected |
|
||||
|--------------|-------------------|
|
||||
| `BedrockModel` | Anthropic (via Bedrock) |
|
||||
| `OllamaModel` | OpenAI-compatible |
|
||||
| Custom `Model` | Falls back to estimation |
|
||||
|
|
@ -68,8 +68,11 @@ nav:
|
|||
- CCR (Compress-Cache-Retrieve): ccr.md
|
||||
- Configuration: configuration.md
|
||||
- Integrations:
|
||||
- Integration Guide: integration-guide.md
|
||||
- LangChain: langchain.md
|
||||
- Agno: agno.md
|
||||
- Strands: strands.md
|
||||
- MCP Tools: mcp.md
|
||||
- Advanced:
|
||||
- LLMLingua: llmlingua.md
|
||||
- Text Compression: text-compression.md
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue