headroom/docs/proxy.md

Proxy Server Documentation

The Headroom proxy server is a production-ready HTTP server that applies context optimization to all requests passing through it.

Starting the Proxy

# Basic usage
headroom proxy

# Custom port
headroom proxy --port 8080

# With all options
headroom proxy \
  --host 0.0.0.0 \
  --port 8787 \
  --log-file /var/log/headroom.jsonl \
  --budget 100.0

Command Line Options

Option	Default	Description
--host	127.0.0.1	Host to bind to
--port	8787	Port to bind to
--no-optimize	false	Disable optimization (passthrough mode)
--no-cache	false	Disable semantic caching
--no-rate-limit	false	Disable rate limiting
--log-file	None	Path to JSONL log file
--budget	None	Daily budget limit in USD

API Endpoints

Health Check

curl http://localhost:8787/health

Response:

{
  "status": "healthy",
  "optimize": true,
  "stats": {
    "total_requests": 42,
    "tokens_saved": 15000,
    "savings_percent": 45.2
  }
}

Detailed Statistics

curl http://localhost:8787/stats

Prometheus Metrics

curl http://localhost:8787/metrics

LLM APIs

The proxy supports both Anthropic and OpenAI API formats:

# Anthropic format
POST /v1/messages

# OpenAI format
POST /v1/chat/completions

Using with Claude Code

# Start proxy
headroom proxy --port 8787

# In another terminal
ANTHROPIC_BASE_URL=http://localhost:8787 claude

Using with Cursor

1. Start the proxy: headroom proxy
2. In Cursor settings, set the base URL to http://localhost:8787

Using with OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8787/v1",
    api_key="your-api-key",  # Still needed for upstream
)

Features

Semantic Caching

The proxy caches responses for repeated queries:

• LRU eviction with configurable max entries
• TTL-based expiration
• Cache key based on message content hash

Rate Limiting

Token bucket rate limiting protects against runaway costs:

• Configurable requests per minute
• Configurable tokens per minute
• Per-API-key tracking

Cost Tracking

Track spending and enforce budgets:

• Real-time cost estimation
• Budget periods: hourly, daily, monthly
• Automatic request rejection when over budget

Prometheus Metrics

Export metrics for monitoring:

headroom_requests_total
headroom_tokens_saved_total
headroom_cost_usd_total
headroom_latency_ms_sum

Configuration via Environment

export HEADROOM_HOST=0.0.0.0
export HEADROOM_PORT=8787
export HEADROOM_BUDGET=100.0
headroom proxy

Running in Production

For production deployments:

# Use a process manager
pip install gunicorn

# Run with gunicorn
gunicorn headroom.proxy.server:app \
  --workers 4 \
  --bind 0.0.0.0:8787 \
  --worker-class uvicorn.workers.UvicornWorker

Or with Docker:

FROM python:3.11-slim
RUN pip install headroom[proxy]
EXPOSE 8787
CMD ["headroom", "proxy", "--host", "0.0.0.0"]