diff --git a/docs/index.md b/docs/index.md index 0f51a9cce..08f9307d9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,176 +1,372 @@ # Headroom +
+ **The Context Optimization Layer for LLM Applications** -Tool outputs are 70-95% redundant. Headroom compresses that away—without losing information. +Compress everything your AI agent reads. Same answers, fraction of the tokens. + +
+ +
+ +[![PyPI](https://img.shields.io/pypi/v/headroom-ai.svg)](https://pypi.org/project/headroom-ai/) +[![Python](https://img.shields.io/pypi/pyversions/headroom-ai.svg)](https://pypi.org/project/headroom-ai/) +[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/chopratejas/headroom/blob/main/LICENSE) +[![Discord](https://img.shields.io/badge/Discord-Join%20us-5865F2?logo=discord&logoColor=white)](https://discord.gg/QEyNhaGA) + +
+ +
+ +
+87% +Avg Token Reduction +
+
+100% +Answer Accuracy +
+
+6 +Compression Algorithms +
+
+100+ +LLM Providers +
+ +
--- -## Quick Install +## What It Does + +Every tool call, DB query, file read, and RAG retrieval your agent makes is 70-95% boilerplate. Headroom compresses it away before it hits the model. The LLM sees less noise, responds faster, and costs less. -```bash -pip install headroom-ai[all] ``` +Your Agent / App + │ + │ tool outputs, logs, DB reads, RAG results, file reads, API responses + ▼ + Headroom ← proxy, Python library, or framework integration + │ + ▼ + LLM Provider (OpenAI, Anthropic, Google, Bedrock, 100+ via LiteLLM) +``` + +Headroom works as a **transparent proxy** (zero code changes), a **Python function** (`compress()`), or a **framework integration** (LangChain, Agno, Strands, LiteLLM, MCP). + +--- ## Quick Start -### Option 1: Proxy (Zero Code Changes) +=== "Proxy (Zero Code Changes)" -Start the proxy: + ```bash + pip install "headroom-ai[all]" + headroom proxy + ``` -```bash -headroom proxy -``` + ```bash + # Point any tool at the proxy + ANTHROPIC_BASE_URL=http://localhost:8787 claude + OPENAI_BASE_URL=http://localhost:8787/v1 your-app + ``` -Point your tools at it: + That's it. Your existing code works unchanged, with 40-90% fewer tokens. -```bash -ANTHROPIC_BASE_URL=http://localhost:8787 claude -``` +=== "Python SDK" -That's it. Your existing code works unchanged, with 40-90% fewer tokens. + ```python + from headroom import compress -### Option 2: Python SDK + result = compress(messages, model="claude-sonnet-4-5-20250929") + response = client.messages.create( + model="claude-sonnet-4-5-20250929", + messages=result.messages, + ) + print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})") + ``` + + Works with any Python LLM client. [Full SDK guide →](sdk.md) + +=== "Coding Agents" + + ```bash + headroom wrap claude # Claude Code + headroom wrap codex # OpenAI Codex CLI + headroom wrap aider # Aider + headroom wrap cursor # Cursor + ``` + + Starts the proxy, points your tool at it, compresses everything automatically. + +=== "LiteLLM Callback" + + ```python + import litellm + from headroom.integrations.litellm_callback import HeadroomCallback + + litellm.callbacks = [HeadroomCallback()] + # All 100+ providers now compressed automatically + ``` + +--- + +## Framework Integrations + +
+ +
+ +### LangChain + +Wrap any chat model. Supports memory, retrievers, tools, streaming, async. ```python -from headroom import compress +from headroom.integrations import HeadroomChatModel -# Compress messages before sending to LLM -result = compress(messages, model="claude-sonnet-4-5-20250929") -response = client.messages.create( - model="claude-sonnet-4-5-20250929", - messages=result.messages, -) - -print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})") +llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o")) ``` ---- +[LangChain Guide →](langchain.md) -## Why Headroom? +
-| Problem | Solution | -|---------|----------| -| Tool outputs bloat context with repetitive JSON | Statistical compression removes redundancy | -| Dynamic content breaks provider caching | Cache alignment stabilizes prefixes | -| Long conversations exceed context limits | Intelligent scoring drops low-value messages | -| Compressed data might be needed later | CCR stores originals for on-demand retrieval | +
---- +### Agno -## Results +Full agent framework integration with observability hooks. -**100 log entries. One critical error buried at position 67.** +```python +from headroom.integrations.agno import HeadroomAgnoModel -| Metric | Baseline | Headroom | -|--------|----------|----------| -| Input tokens | 10,144 | 1,260 | -| Correct answers | 4/4 | 4/4 | +model = HeadroomAgnoModel(Claude(id="claude-sonnet-4-20250514")) +agent = Agent(model=model) +``` -**87.6% fewer tokens. Same answer.** +[Agno Guide →](agno.md) -The FATAL error was automatically preserved—no configuration needed. +
+ +
+ +### Strands + +Model wrapping + tool output hook provider for Strands Agents. + +```python +from headroom.integrations.strands import HeadroomStrandsModel + +model = HeadroomStrandsModel(wrapped_model=bedrock_model) +agent = Agent(model=model) +``` + +[Strands Guide →](strands.md) + +
+ +
+ +### MCP Tools + +Three tools for Claude Code, Cursor, or any MCP client: `headroom_compress`, `headroom_retrieve`, `headroom_stats`. + +```bash +headroom mcp install && claude +``` + +[MCP Guide →](mcp.md) + +
+ +
+ +[All integration patterns →](integration-guide.md){ .md-button } --- ## How It Works -``` -Your App → Headroom → LLM Provider - ↓ - Compression - Caching - Retrieval +Headroom runs a three-stage pipeline on every request: + +```mermaid +graph LR + A[Your Prompt] --> B[CacheAligner] + B --> C[ContentRouter] + C --> D[IntelligentContext] + D --> E[LLM Provider] + + C -->|JSON| F[SmartCrusher] + C -->|Code| G[CodeCompressor] + C -->|Text| H[Kompress] + C -->|Logs| I[LogCompressor] + + F --> D + G --> D + H --> D + I --> D ``` -1. **Intercepts context** — Tool outputs, logs, search results -2. **Compresses intelligently** — Keeps errors, outliers, boundaries -3. **Stores originals** — Full data available if LLM requests it -4. **Aligns for caching** — Provider caches actually hit +**Stage 1: CacheAligner** — Stabilizes message prefixes so the provider's KV cache actually hits. Claude offers a 90% read discount on cached prefixes; CacheAligner makes that work. + +**Stage 2: ContentRouter** — Auto-detects content type (JSON, code, logs, search results, diffs, HTML, plain text) and routes each to the optimal compressor: + +| Content Type | Compressor | How It Works | +|-------------|-----------|-------------| +| JSON arrays | **SmartCrusher** | Statistical analysis: keeps errors, anomalies, boundaries. No hardcoded rules. | +| Source code | **CodeCompressor** | AST-aware (tree-sitter). Preserves function signatures, collapses bodies. | +| Plain text | **Kompress** | ModernBERT token classification. Removes redundant tokens while preserving meaning. | +| Build/test logs | **LogCompressor** | Keeps failures, errors, warnings. Drops passing noise. | +| Search results | **SearchCompressor** | Ranks by relevance to user query, keeps top matches. | +| Git diffs | **DiffCompressor** | Preserves change hunks, drops unchanged context. | +| HTML | **HTMLExtractor** | Strips markup, extracts readable content. | + +**Stage 3: IntelligentContext** — If the conversation still exceeds the model's context limit, scores each message by importance (recency, references, density) and drops the lowest-value ones. + +**Nothing is lost.** Compressed content goes into the CCR store (Compress-Cache-Retrieve). The LLM gets a `headroom_retrieve` tool and can fetch full originals when it needs more detail. + +[Full architecture deep dive →](ARCHITECTURE.md) --- -## Integrations +## Results -=== "LangChain" +**100 production log entries. One critical error buried at position 67.** - ```python - from langchain_openai import ChatOpenAI - from headroom.integrations import HeadroomChatModel +| Metric | Baseline | Headroom | +|--------|----------|----------| +| Input tokens | 10,144 | 1,260 | +| Correct answers | **4/4** | **4/4** | - llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o")) - response = llm.invoke("Hello!") - ``` +**87.6% fewer tokens. Same answer.** The FATAL error was automatically preserved — not by keyword matching, but by statistical analysis of field variance. -=== "Agno" +### Real Workloads - ```python - from agno.agent import Agent - from agno.models.openai import OpenAIChat - from headroom.integrations.agno import HeadroomAgnoModel +| Scenario | Before | After | Savings | +|----------|--------|-------|---------| +| Code search (100 results) | 17,765 | 1,408 | **92%** | +| SRE incident debugging | 65,694 | 5,118 | **92%** | +| Codebase exploration | 78,502 | 41,254 | **47%** | +| GitHub issue triage | 54,174 | 14,761 | **73%** | - model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o")) - agent = Agent(model=model) - ``` +### Accuracy Benchmarks -=== "Strands" +| Benchmark | Category | N | Accuracy | Compression | +|-----------|----------|---|----------|-------------| +| GSM8K | Math | 100 | 0.870 | 0.000 delta | +| TruthfulQA | Factual | 100 | 0.560 | +0.030 delta | +| SQuAD v2 | QA | 100 | **97%** | 19% reduction | +| BFCL | Tool/Function | 100 | **97%** | 32% reduction | +| CCR Needle | Lossless | 50 | **100%** | 77% reduction | - ```python - from strands import Agent - from strands.models.bedrock import BedrockModel - from headroom.integrations.strands import HeadroomStrandsModel - - model = HeadroomStrandsModel(wrapped_model=BedrockModel( - model_id="us.anthropic.claude-sonnet-4-20250514-v1:0" - )) - agent = Agent(model=model) - ``` - -=== "AWS Bedrock" - - ```bash - # Start proxy with Bedrock backend - headroom proxy --backend bedrock --region us-east-1 - - # Point Claude Code at it - ANTHROPIC_API_KEY="sk-ant-dummy" \ - ANTHROPIC_BASE_URL=http://localhost:8787 \ - claude - ``` +[Full benchmark methodology →](benchmarks.md) | [Known limitations →](LIMITATIONS.md) --- -## Features +## Key Features -**Compression** +
-- Statistical JSON array compression (no hardcoded rules) -- ML-based text compression via LLMLingua -- AST-aware code compression -- Image optimization (40-90% reduction) +
+### Lossless Compression (CCR) +Compresses aggressively, stores originals, gives the LLM a tool to retrieve full details. Nothing is thrown away. +[Learn more →](ccr.md) +
-**Context Management** +
+### Smart Content Detection +Auto-detects JSON, code, logs, text, diffs, HTML. Routes each to the best compressor. Zero configuration needed. +[Learn more →](compression.md) +
-- Intelligent message scoring and dropping -- Compress-Cache-Retrieve (CCR) for lossless compression -- Provider cache alignment for better hit rates +
+### Cache Optimization +Stabilizes prefixes so provider KV caches hit. Tracks frozen messages to preserve the 90% read discount. +[Learn more →](ccr.md) +
-**Operations** +
+### Image Compression +40-90% token reduction via trained ML router. Automatically selects resize/quality tradeoff per image. +[Learn more →](image-compression.md) +
-- Prometheus metrics endpoint -- Request logging and cost tracking -- Budget limits and rate limiting +
+### Persistent Memory +Hierarchical memory (user/session/agent/turn) with SQLite + HNSW backends. Survives across conversations. +[Learn more →](memory.md) +
+ +
+### Failure Learning +Reads past sessions, finds failed tool calls, correlates with what succeeded, writes learnings to CLAUDE.md. +[Learn more →](learn.md) +
+ +
+### Multi-Agent Context +Compress what moves between agents. Any framework. +```python +ctx = SharedContext() +ctx.put("research", big_output) +summary = ctx.get("research") # ~80% smaller +``` +[Learn more →](shared-context.md) +
+ +
+### Metrics & Observability +Prometheus endpoint, per-request logging, cost tracking, budget limits, pipeline timing breakdowns. +[Learn more →](metrics.md) +
+ +
+ +--- + +## Cloud Providers + +Works with any LLM provider out of the box: + +```bash +headroom proxy # Direct Anthropic/OpenAI +headroom proxy --backend bedrock --region us-east-1 # AWS Bedrock +headroom proxy --backend vertex_ai --region us-central1 # Google Vertex AI +headroom proxy --backend azure # Azure OpenAI +headroom proxy --backend openrouter # OpenRouter (400+ models) +``` + +Or via LiteLLM for 100+ providers (Together, Groq, Fireworks, Ollama, vLLM, etc.). + +--- + +## Installation + +```bash +pip install headroom-ai # Core library +pip install "headroom-ai[all]" # Everything (recommended) +pip install "headroom-ai[proxy]" # Proxy server + MCP tools +pip install "headroom-ai[ml]" # ML compression (Kompress, requires torch) +pip install "headroom-ai[langchain]" # LangChain integration +pip install "headroom-ai[agno]" # Agno integration +pip install "headroom-ai[evals]" # Evaluation framework +``` + +Requires Python 3.10+. --- ## Next Steps -- [Quickstart Guide](quickstart.md) — Get running in 5 minutes -- [Proxy Documentation](proxy.md) — Configure the optimization proxy -- [Architecture](ARCHITECTURE.md) — Deep dive into how it works +- **[Quickstart](quickstart.md)** — Running in 5 minutes +- **[Integration Guide](integration-guide.md)** — Every way to add Headroom to your stack +- **[Architecture](ARCHITECTURE.md)** — How the pipeline works under the hood +- **[Benchmarks](benchmarks.md)** — Accuracy and latency data +- **[Limitations](LIMITATIONS.md)** — When compression helps and when it doesn't --- -## License - -Apache 2.0 — Free for commercial use. +Apache 2.0 — Free for commercial use. [GitHub](https://github.com/chopratejas/headroom) | [PyPI](https://pypi.org/project/headroom-ai/) | [Discord](https://discord.gg/QEyNhaGA) diff --git a/docs/overrides/.gitkeep b/docs/overrides/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css new file mode 100644 index 000000000..79a9c7e04 --- /dev/null +++ b/docs/stylesheets/extra.css @@ -0,0 +1,103 @@ +/* Headroom docs custom styles */ + +/* Hero section on landing page */ +.md-typeset .hero { + text-align: center; + padding: 2rem 0 1rem; +} + +.md-typeset .hero h1 { + font-size: 2.5rem; + font-weight: 700; + margin-bottom: 0.5rem; +} + +.md-typeset .hero .tagline { + font-size: 1.25rem; + color: var(--md-default-fg-color--light); + margin-bottom: 1.5rem; +} + +/* Feature grid on landing page */ +.grid-container { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(280px, 1fr)); + gap: 1rem; + margin: 1.5rem 0; +} + +.grid-item { + border: 1px solid var(--md-default-fg-color--lightest); + border-radius: 0.5rem; + padding: 1.25rem; + transition: border-color 0.2s, box-shadow 0.2s; +} + +.grid-item:hover { + border-color: var(--md-accent-fg-color); + box-shadow: 0 2px 8px rgba(0, 0, 0, 0.08); +} + +.grid-item h3 { + margin-top: 0 !important; + font-size: 1rem; +} + +.grid-item p { + font-size: 0.85rem; + color: var(--md-default-fg-color--light); + margin-bottom: 0; +} + +/* Tighter code blocks */ +.md-typeset pre > code { + font-size: 0.8rem; +} + +/* Stats bar */ +.stats-bar { + display: flex; + justify-content: center; + gap: 3rem; + padding: 1rem 0; + flex-wrap: wrap; +} + +.stat { + text-align: center; +} + +.stat .number { + display: block; + font-size: 2rem; + font-weight: 700; + color: var(--md-primary-fg-color); +} + +.stat .label { + font-size: 0.8rem; + color: var(--md-default-fg-color--light); + text-transform: uppercase; + letter-spacing: 0.05em; +} + +/* Badge row */ +.badges { + text-align: center; + margin-bottom: 1.5rem; +} + +.badges img { + margin: 0 0.15rem; +} + +/* Better admonition for "how it works" */ +.md-typeset .admonition.pipeline, +.md-typeset details.pipeline { + border-color: var(--md-accent-fg-color); +} + +/* Hide title on home page (we use hero instead) */ +.md-typeset h1:first-child { + /* keep visible but reduce gap */ +} diff --git a/mkdocs.yml b/mkdocs.yml index 717a723a8..5642a553c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,5 +1,5 @@ site_name: Headroom -site_description: The Context Optimization Layer for LLM Applications +site_description: "The Context Optimization Layer for LLM Applications — compress everything your AI agent reads." site_url: https://chopratejas.github.io/headroom repo_url: https://github.com/chopratejas/headroom repo_name: chopratejas/headroom @@ -7,35 +7,52 @@ edit_uri: edit/main/docs/ theme: name: material + custom_dir: docs/overrides palette: - - scheme: default - primary: indigo - accent: indigo + - media: "(prefers-color-scheme: light)" + scheme: default + primary: deep purple + accent: amber toggle: icon: material/brightness-7 name: Switch to dark mode - - scheme: slate - primary: indigo - accent: indigo + - media: "(prefers-color-scheme: dark)" + scheme: slate + primary: deep purple + accent: amber toggle: icon: material/brightness-4 name: Switch to light mode + font: + text: Inter + code: JetBrains Mono features: - content.code.copy + - content.code.annotate - content.tabs.link - navigation.instant + - navigation.instant.progress - navigation.tabs + - navigation.tabs.sticky - navigation.sections + - navigation.expand - navigation.top + - navigation.indexes - search.highlight - search.suggest + - search.share - toc.follow + - header.autohide icon: repo: fontawesome/brands/github + logo: material/layers-outline plugins: - search +extra_css: + - stylesheets/extra.css + markdown_extensions: - admonition - attr_list @@ -50,44 +67,57 @@ markdown_extensions: pygments_lang_class: true - pymdownx.inlinehilite - pymdownx.snippets - - pymdownx.superfences + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format - pymdownx.tabbed: alternate_style: true - pymdownx.details + - pymdownx.emoji: + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg + - pymdownx.tasklist: + custom_checkbox: true nav: - Home: index.md - Getting Started: - Quickstart: quickstart.md - Installation: getting-started.md - - Benchmarks: benchmarks.md + - Configuration: configuration.md - User Guide: - Proxy Server: proxy.md - - Compression: compression.md + - Compression Pipeline: compression.md + - CCR (Lossless Retrieval): ccr.md - Image Compression: image-compression.md - - CCR (Compress-Cache-Retrieve): ccr.md - - Configuration: configuration.md + - Memory System: memory.md + - Shared Context: shared-context.md + - Failure Learning: learn.md - Integrations: - - Integration Guide: integration-guide.md + - Overview: integration-guide.md - LangChain: langchain.md - Agno: agno.md - Strands: strands.md - MCP Tools: mcp.md - - Advanced: - - LLMLingua: llmlingua.md - - Text Compression: text-compression.md - - Transforms: transforms.md + - How It Works: - Architecture: ARCHITECTURE.md + - Transforms Pipeline: transforms.md + - Text Compression: text-compression.md + - LLMLingua: llmlingua.md + - Benchmarks: + - Accuracy: benchmarks.md + - Latency: LATENCY_BENCHMARKS.md + - Limitations: LIMITATIONS.md - Reference: - API: api.md - SDK: sdk.md - - Metrics: metrics.md - - Errors: errors.md + - Metrics & Observability: metrics.md + - Error Codes: errors.md - Troubleshooting: troubleshooting.md - Deployment: - - macOS: macos-deployment.md - - Memory: - - Overview: memory.md + - macOS LaunchAgent: macos-deployment.md extra: social: @@ -95,3 +125,6 @@ extra: link: https://github.com/chopratejas/headroom - icon: fontawesome/brands/python link: https://pypi.org/project/headroom-ai/ + - icon: fontawesome/brands/discord + link: https://discord.gg/QEyNhaGA + generator: false