diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 000000000..602859474 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,33 @@ +name: Deploy Documentation + +on: + push: + branches: + - main + paths: + - 'docs/**' + - 'mkdocs.yml' + - '.github/workflows/docs.yml' + workflow_dispatch: + +permissions: + contents: write + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Install dependencies + run: pip install mkdocs-material + + - name: Build and deploy + run: mkdocs gh-deploy --force diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index ff3ecebb0..000000000 --- a/docs/README.md +++ /dev/null @@ -1,76 +0,0 @@ -# Headroom Documentation - -Welcome to the Headroom documentation. - -## Getting Started - -| Guide | Description | -|-------|-------------| -| [Quickstart](quickstart.md) | 5-minute setup | -| [SDK Guide](sdk.md) | Python SDK usage | -| [Proxy Guide](proxy.md) | Proxy server deployment | - -## Framework Integrations - -| Framework | Description | -|-----------|-------------| -| [LangChain](langchain.md) | Chat models, memory, retrievers, agents, streaming | -| [Agno](agno.md) | Model wrapper, hooks, multi-provider support | -| MCP | See [CCR Guide](ccr.md) for tool compression | - -## Core Concepts - -| Topic | Description | -|-------|-------------| -| [Universal Compression](compression.md) | ML-based content detection + structure preservation | -| [Image Compression](image-compression.md) | 40-90% token reduction for images via trained ML router | -| [Transforms](transforms.md) | How compression works | -| [CCR](ccr.md) | Reversible compression architecture | -| [Configuration](configuration.md) | All configuration options | - -## Advanced - -| Topic | Description | -|-------|-------------| -| [Text Compression](text-compression.md) | Opt-in utilities for search/logs | -| [LLMLingua](llmlingua.md) | ML-based compression | -| [Metrics](metrics.md) | Monitoring and observability | -| [Errors](errors.md) | Error handling | - -## Deployment & Operations - -| Guide | Description | -|-------|-------------| -| [macOS Deployment](macos-deployment.md) | Run proxy as background service on macOS | - -## Reference - -| Topic | Description | -|-------|-------------| -| [API Reference](api.md) | Complete API docs | -| [Architecture](ARCHITECTURE.md) | Internal design | -| [Troubleshooting](troubleshooting.md) | Common issues | - -## Overview - -Headroom is the Context Optimization Layer for LLM applications. It reduces your LLM costs by 50-90% through intelligent context compression. - -### How It Works - -1. **Universal Compression** — ML-based content detection with structure-preserving compression -2. **SmartCrusher** — Compresses JSON tool outputs, keeping errors, anomalies, and relevant items -3. **CacheAligner** — Stabilizes message prefixes so provider caching works -4. **IntelligentContextManager** — Score-based context dropping using TOIN-learned importance (default) -5. **CCR** — Caches original data so compression is reversible - -### Safety Guarantees - -- Never removes human content -- Never breaks tool call ordering -- Parse failures pass through unchanged -- LLM can always retrieve original data - -### Getting Help - -- [GitHub Issues](https://github.com/chopratejas/headroom/issues) — Bug reports -- [GitHub Discussions](https://github.com/chopratejas/headroom/discussions) — Questions diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 000000000..a444c2ef7 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,162 @@ +# Headroom + +**The Context Optimization Layer for LLM Applications** + +Tool outputs are 70-95% redundant. Headroom compresses that away—without losing information. + +--- + +## Quick Install + +```bash +pip install headroom-ai[all] +``` + +## Quick Start + +### Option 1: Proxy (Zero Code Changes) + +Start the proxy: + +```bash +headroom proxy +``` + +Point your tools at it: + +```bash +ANTHROPIC_BASE_URL=http://localhost:8787 claude +``` + +That's it. Your existing code works unchanged, with 40-90% fewer tokens. + +### Option 2: Python SDK + +```python +from headroom import Headroom + +hr = Headroom() + +# Compress tool output before sending to LLM +compressed = hr.compress(large_tool_output) + +# If LLM needs the full data, retrieve it +original = hr.retrieve(compressed) +``` + +--- + +## Why Headroom? + +| Problem | Solution | +|---------|----------| +| Tool outputs bloat context with repetitive JSON | Statistical compression removes redundancy | +| Dynamic content breaks provider caching | Cache alignment stabilizes prefixes | +| Long conversations exceed context limits | Intelligent scoring drops low-value messages | +| Compressed data might be needed later | CCR stores originals for on-demand retrieval | + +--- + +## Results + +**100 log entries. One critical error buried at position 67.** + +| Metric | Baseline | Headroom | +|--------|----------|----------| +| Input tokens | 10,144 | 1,260 | +| Correct answers | 4/4 | 4/4 | + +**87.6% fewer tokens. Same answer.** + +The FATAL error was automatically preserved—no configuration needed. + +--- + +## How It Works + +``` +Your App → Headroom → LLM Provider + ↓ + Compression + Caching + Retrieval +``` + +1. **Intercepts context** — Tool outputs, logs, search results +2. **Compresses intelligently** — Keeps errors, outliers, boundaries +3. **Stores originals** — Full data available if LLM requests it +4. **Aligns for caching** — Provider caches actually hit + +--- + +## Integrations + +=== "LangChain" + + ```python + from langchain_openai import ChatOpenAI + from headroom.integrations import HeadroomChatModel + + llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o")) + response = llm.invoke("Hello!") + ``` + +=== "Agno" + + ```python + from agno.agent import Agent + from agno.models.openai import OpenAIChat + from headroom.integrations.agno import HeadroomAgnoModel + + model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o")) + agent = Agent(model=model) + ``` + +=== "AWS Bedrock" + + ```bash + # Start proxy with Bedrock backend + headroom proxy --backend bedrock --region us-east-1 + + # Point Claude Code at it + ANTHROPIC_API_KEY="sk-ant-dummy" \ + ANTHROPIC_BASE_URL=http://localhost:8787 \ + claude + ``` + +--- + +## Features + +**Compression** + +- Statistical JSON array compression (no hardcoded rules) +- ML-based text compression via LLMLingua +- AST-aware code compression +- Image optimization (40-90% reduction) + +**Context Management** + +- Intelligent message scoring and dropping +- Compress-Cache-Retrieve (CCR) for lossless compression +- Provider cache alignment for better hit rates + +**Operations** + +- Prometheus metrics endpoint +- Request logging and cost tracking +- Budget limits and rate limiting + +--- + +## Next Steps + +- [Quickstart Guide](quickstart.md) — Get running in 5 minutes +- [Proxy Documentation](proxy.md) — Configure the optimization proxy +- [Architecture](ARCHITECTURE.md) — Deep dive into how it works + +--- + +## License + +Apache 2.0 — Free for commercial use. diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 000000000..6968addd9 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,93 @@ +site_name: Headroom +site_description: The Context Optimization Layer for LLM Applications +site_url: https://chopratejas.github.io/headroom +repo_url: https://github.com/chopratejas/headroom +repo_name: chopratejas/headroom +edit_uri: edit/main/docs/ + +theme: + name: material + palette: + - scheme: default + primary: indigo + accent: indigo + toggle: + icon: material/brightness-7 + name: Switch to dark mode + - scheme: slate + primary: indigo + accent: indigo + toggle: + icon: material/brightness-4 + name: Switch to light mode + features: + - content.code.copy + - content.tabs.link + - navigation.instant + - navigation.tabs + - navigation.sections + - navigation.top + - search.highlight + - search.suggest + - toc.follow + icon: + repo: fontawesome/brands/github + +plugins: + - search + +markdown_extensions: + - admonition + - attr_list + - def_list + - md_in_html + - tables + - toc: + permalink: true + - pymdownx.highlight: + anchor_linenums: true + line_spans: __span + pygments_lang_class: true + - pymdownx.inlinehilite + - pymdownx.snippets + - pymdownx.superfences + - pymdownx.tabbed: + alternate_style: true + - pymdownx.details + +nav: + - Home: index.md + - Getting Started: + - Quickstart: quickstart.md + - Installation: getting-started.md + - User Guide: + - Proxy Server: proxy.md + - Compression: compression.md + - Image Compression: image-compression.md + - CCR (Compress-Cache-Retrieve): ccr.md + - Configuration: configuration.md + - Integrations: + - LangChain: langchain.md + - Agno: agno.md + - Advanced: + - LLMLingua: llmlingua.md + - Text Compression: text-compression.md + - Transforms: transforms.md + - Architecture: ARCHITECTURE.md + - Reference: + - API: api.md + - SDK: sdk.md + - Metrics: metrics.md + - Errors: errors.md + - Troubleshooting: troubleshooting.md + - Deployment: + - macOS: macos-deployment.md + - Memory: + - Overview: memory.md + +extra: + social: + - icon: fontawesome/brands/github + link: https://github.com/chopratejas/headroom + - icon: fontawesome/brands/python + link: https://pypi.org/project/headroom-ai/