Anthropic's image token formula: (width * height) / 750, capped at
~1600 tokens after auto-resize to 1568px max edge. The previous 85
was too low and the even bigger problem was that type="image"
(Anthropic format) fell through to json.dumps counting base64 as
text tokens (1MB = ~330K fake tokens).
Source: https://platform.claude.com/docs/en/build-with-claude/vision
Anthropic images use type="image" (not "image_url" like OpenAI).
The tokenizer's _count_content_parts fell through to the else branch
which json.dumps'd the entire base64 string and counted it as text
tokens. A 1MB image = ~330K fake "tokens", inflating savings by 10x.
Now handles type="image" with the same fixed 85-token cost as
type="image_url". This fixes the bogus 1.8B/513M token savings
reported by Windows instances using image-heavy Claude Code sessions.
Manual ssl.create_default_context() + certifi doesn't load the Windows
system certificate store, causing HTTP 500 on wss:// connections to
OpenAI. Using ssl=True lets the websockets library handle SSL natively
with proper cross-platform cert store loading.
CacheAligner was hardcoded to enabled=True in the pipeline despite
the config default being False. It extracts dynamic content from the
system prompt middle and reinserts at the end, which:
1. CHANGES the prefix bytes → provider cache miss (loses 90% discount)
2. ADDS ~341 tokens of formatting overhead per request
3. Net effect: more expensive, worse caching
Now uses the config default (enabled=False). The CacheAligner still
exists for users who explicitly opt in, but the proxy no longer
forces it on.
- Fix WS /v1/responses: forward Sec-WebSocket-Protocol (subprotocol) to
upstream instead of stripping it — root cause of Codex HTTP 500 errors
- Fix WS relay: handle binary messages properly instead of crashing on
.decode(), add debug logging instead of silent except:pass
- Add Authorization header fallback from OPENAI_API_KEY env var for WS
- Extract response body from websockets InvalidStatus for error debugging
- Fix streaming /v1/responses: pass optimized_tokens (not original_tokens
twice) so compression savings appear in streaming metrics
- Fix hardcoded provider="bedrock" in 4 metrics/log locations — now uses
self.anthropic_backend.name so LiteLLM backends report correctly
- Forward --backend, --anyllm-provider, --region flags from wrap commands
(codex, aider) to the proxy subprocess via _start_proxy()
- Forward API key from request headers to LiteLLM acompletion() calls
- Forward region to Vertex AI (vertex_location) not just Bedrock
- Redesign proxy startup banner: show routing table instead of misleading
"Backend: Anthropic" label
Closes#86
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move ProxyConfig, RequestLog, CacheEntry, RateLimitState to
headroom/proxy/models.py. Re-exported from server.py for backward
compatibility — all existing imports continue to work.
server.py: 8835 → 8643 lines (-192)
models.py: 199 lines (new)
Part of the server.py split effort to improve maintainability.
transforms_summary is a counted dict (e.g. {"router:tool_result:text": 4})
alongside the raw transforms_applied list. Cleaner display for users
without losing the raw data for debugging.
- Fix beacon spam: file lock ensures only one beacon per proxy regardless
of worker count. Workers > 1 caused N beacons firing N rows per cycle.
- Beacon upsert: on_conflict=session_id prevents duplicate rows.
- Beacon stop() guard: skip final report if uptime < 2 minutes.
- Fix dashboard cost: savings_usd now uses model list price (monotonic),
not moving average. Separate breakdown for compression/cache/rtk.
Fixes#83
savings_usd is now tokens_saved * model list input price (monotonic,
transparent). Removed non-monotonic moving-average repricing and
confusing cost_without_headroom counterfactual.
Dashboard hero shows "Compression Savings" with clear subtitle.
Savings Breakdown section shows compression, cache, and RTK separately
with distinct colors and no scope mixing.
All beacon/telemetry fields preserved. RTK token counts still reported.
Fixes#83
OpenAI's WebSocket Responses API requires the header
'OpenAI-Beta: responses-api=v1'. Without it, the server returns HTTP 500
on the WebSocket upgrade. Also forward all client headers (not just auth)
to upstream, skipping only hop-by-hop headers.
Tested with real OpenAI API: basic text, large tool output compression,
all working through WebSocket proxy.
Fixes#82, updates #79
KompressCompressor now tries ONNX Runtime first (156MB INT8 model),
falls back to PyTorch only if ONNX unavailable. No torch needed for
text compression — just onnxruntime (~50MB) + transformers (tokenizer).
Changes:
- Add onnxruntime + transformers to [proxy] extra in pyproject.toml
- Add _OnnxModel wrapper with get_scores/get_keep_mask interface
- _load_kompress() tries ONNX first, falls back to PyTorch
- is_kompress_available() returns True if EITHER backend available
- compress() handles both numpy (ONNX) and tensor (PyTorch) outputs
Dependency impact:
Before: pip install headroom-ai[proxy] → no text compression
After: pip install headroom-ai[proxy] → Kompress ONNX INT8 (156MB)
[ml] extra still available for full PyTorch (600MB, GPU support)
Codex v0.117.0+ with newer models uses WebSocket instead of HTTP POST
for the Responses API. Added @app.websocket("/v1/responses") handler that:
- Accepts ws:// connections and forwards to wss://api.openai.com
- Compresses input on first message using existing pipeline
- Relays all response events bidirectionally
- Handles SSL (certifi), graceful disconnect, missing websockets lib
Tested with real OpenAI API: basic text, large tool outputs (200 records),
parallel function calls, instructions preservation.
Addresses #79
The /v1/responses handler was passing through without compression,
meaning Codex CLI users got zero savings. Now converts Responses API
items (function_call, function_call_output, reasoning, message) to
Chat Completions format, runs the existing pipeline, and converts back.
- New: headroom/proxy/responses_converter.py — pure conversion functions
- 21 unit tests + 3 integration tests (tested with real OpenAI API)
- Preserves reasoning items, images, unknown types verbatim
- Skips compression when previous_response_id is set
- 27% compression on real Codex-pattern payloads (500 records → 14K tokens saved)
Closes#73
Bedrock requires role=tool messages immediately after assistant tool_calls.
The previous fix inserted a user text message in between when the message
contained both text and tool_result blocks, breaking the pairing.
Drop text alongside tool_result (Claude Code never sends it in practice).
Added ordering regression tests for the Bedrock constraint.
Use stream_reader instead of decompress() for zstd request bodies.
Codex uses streaming zstd (no content size in frame header), which
causes decompress() to fail. stream_reader handles both cases.
Also added zstandard to the [proxy] optional dependencies.
_decode_project_path now detects single-letter first component as a
Windows drive letter: -C-MQ2-macros → C:\MQ2\macros instead of
/C/MQ2/macros (which becomes \\C\MQ2\macros on Windows).
- Add Windows drive detection before Unix path attempts
- Fix fallback path construction for Windows patterns
- Add Linux /home/ support in greedy decoder
- Add 2 tests for Windows drive letter patterns
