Three bugs fixed:
1. observations counter: get_recommendation() now increments an
observations counter on the ToolPattern, so we can track whether
TOIN is being consulted during compression.
2. streaming CCR feedback: when the LLM generates headroom_retrieve
tool_use in a streaming response, the proxy now extracts the
hash/query and calls store.retrieve() to trigger the full feedback
chain (_log_retrieval → process_pending_feedback → toin.record_retrieval).
Previously, streaming responses (all Claude Code traffic) never
triggered TOIN feedback, leaving preserve_fields empty.
3. preserve_fields population: with retrievals now flowing to TOIN,
field_retrieval_frequency gets populated, which feeds into
_build_recommendation() → preserve_fields in CompressionHint.
SmartCrusher can now learn which specific fields to preserve.
Root cause: headroom_retrieve tool calls in streaming responses were
returned to the client but never recorded as retrieval events in TOIN.
The entire feedback loop was write-only — TOIN learned compression
patterns but never learned from retrieval signals.
Adds 15 new tests covering observations counter, retrieval recording,
preserve_fields population, and CCR feedback extraction.
Enables managed/enterprise customers to run the proxy in their own
environment while reporting aggregate usage back to Headroom cloud.
- HEADROOM_LICENSE_KEY env var activates managed mode
- UsageReporter validates license on startup, caches to ~/.headroom/
- Reports aggregate stats every 5 min (tokens, costs, models — no content)
- Graceful degradation: 7-day grace period if cloud unreachable
- Expired license → passthrough mode (proxy works, compression stops)
- Zero impact on OSS users (no license key = no reporter)
When Claude Code sends the system prompt as a list of content blocks
(with cache_control markers for prompt caching), the memory context
injection was replacing the entire system prompt with just the memory
context string. This caused 400 invalid_request_error from the
Anthropic API because Claude Code's full system instructions were lost.
Root cause: the else branch in inject_memory_context set
body["system"] = context instead of appending to the existing list.
Also adds:
- X-Headroom-Bypass / X-Headroom-Mode: passthrough header support
to skip all compression (useful for debugging)
- Full diagnostic dump to ~/.headroom/logs/debug_400/ on any upstream
4xx/5xx, capturing pre/post compression messages, tools, headers,
and error response for offline analysis
- Bypass guards on CCR tool injection and image compression
Input cost was computed at list price for all tokens, ignoring that
~90% are cache reads at 10% price (Anthropic). Now uses LiteLLM's
native cache_read_input_token_cost and cache_creation_input_token_cost
for accurate cost calculation. Also adds dashboard note that cost
covers message tokens only (excludes system prompt & tool definitions).
- Change default HEADROOM_MODE from cost_savings to token_headroom
across server.py, cli/proxy.py, and mcp_server.py. Prefix caching
is native to providers; Headroom's value-add is compression.
- Fix undefined _compression_failed variable in Gemini handler
(ruff + mypy error).
- Apply ruff format fixes.
Extract duplicated regex patterns and error keywords from diff_compressor,
intelligent_context, search_compressor, smart_crusher, and text_compressor
into a shared headroom/transforms/error_detection.py module.
Added ignore_unknown_options to all wrap subcommands so flags like
--resume, --model etc. are forwarded to the wrapped tool instead of
being rejected by Click.
Two fixes:
1. `headroom wrap claude` ignored HEADROOM_MODE env var — the Click-based
proxy CLI never passed `mode=` to ProxyConfig, so it always defaulted
to cost_savings. Added --mode flag to `headroom proxy` and forwarded
HEADROOM_MODE from wrap's _start_proxy().
2. Cache write premium (1.25x) was subtracted from savings as a penalty,
but Claude Code already pays this baseline cost regardless of Headroom.
Renamed bust_penalty_usd → write_premium_usd (observability only) and
stopped deducting it from net_savings_usd.
In token_headroom mode, original_tokens was being overwritten with the
pipeline's input view (post-Zone-1-swap), causing tokens_saved to only
reflect Zone 2 compression. Zone 1 savings from cached content swaps
were invisible in stats, cost tracking, and the session summary.
Now keeps original_tokens as the real original (from Claude Code's
uncompressed messages) so tokens_saved = Zone 1 + Zone 2.
Token headroom mode (HEADROOM_MODE=token_headroom) compresses older
messages to extend session length, trading prefix cache cost savings
for token reduction. Content-addressed CompressionCache avoids
re-compression across turns. Works for both Anthropic and OpenAI.
Also: claude-opus-4-6 model entry, clean session summary in /stats
and MCP headroom_stats tool.
The /stats endpoint now includes a "summary" section at the top with:
- Avg/best compression % on requests that actually compressed
- Breakdown of why uncompressed requests were skipped
- Cost impact: without vs with Headroom, total saved
- Actionable tip when token_headroom mode would help
The MCP headroom_stats tool returns clean formatted text instead of
dumping raw JSON when the proxy is reachable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- compute_frozen_count now checks _cache directly instead of calling
get_compressed, which was inflating hit/miss stats on every turn
- update_from_result uses len//4 for rough token estimate instead of
raw character count
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add three methods and supporting helpers for token headroom mode:
- compute_frozen_count: counts consecutive stable messages from start
- apply_cached: swaps cached compressions into tool results (immutable)
- update_from_result: learns new compressions from original/compressed pairs
Supports both Anthropic and OpenAI tool result formats.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduces dual-mode optimization (cost_savings vs token_headroom) to address
low compression rates in long Claude Code sessions caused by prefix freeze
consuming all messages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- SharedContext: compressed inter-agent context sharing via put()/get()
over existing CCR compression pipeline. Zero new dependencies.
- README rewrite: lead with "any agent" positioning, not just coding
agents. Add headroom wrap, SharedContext, MCP tools to Quick Start.
Reorder integration table: universal first, coding shortcuts last.
Update compression pipeline references (LLMLingua → Kompress).
- Fix proxy cleanup in headroom wrap: don't kill shared proxy if other
clients are still using it (was orphaning terminals 2-N).
- New docs: docs/shared-context.md
- Fix proxy crash when torch not installed: make kompress_compressor.py
imports lazy so `is_kompress_available()` works without [ml] extra
- Rewrite MCP server from 1 tool (retrieve-only) to 3 tools:
headroom_compress (on-demand compression, no proxy needed),
headroom_retrieve (local store first, proxy fallback),
headroom_stats (session stats + sub-agent aggregation + proxy cache)
- Add shared stats file (~/.headroom/session_stats.jsonl) so sub-agent
compression stats are visible from the main session
- Add mcp to [proxy] extras so proxy users get MCP tools automatically
- Remove dead TextCompressor from exports and pipeline (was never called)
- Update mcp install messaging to clarify proxy vs MCP roles
- Fix fcntl Windows compat, asyncio deprecation, httpx timeout race
Bump version to 0.4.6.
The proxy startup crashed with `ModuleNotFoundError: No module named
'torch'` when installed with just `[proxy]` extras because
kompress_compressor.py had unconditional top-level torch imports.
Moved torch/transformers imports to be lazy so the module is safely
importable without the [ml] extra. Added tests for import safety.
Bumped version to 0.4.5
- headroom wrap codex: injects rtk instructions into AGENTS.md
- headroom wrap cursor: injects into .cursorrules, prints config steps
- headroom wrap aider: injects into CONVENTIONS.md, sets both env vars
- All with --no-rtk flag to skip rtk setup
Unified savings metrics across three layers:
- cli_filtering: tokens avoided by rtk before reaching context
- compression: tokens removed by proxy (SmartCrusher, etc.)
- prefix_cache: provider cache discount with honest attribution
Fix flaky test_process_stats_collected when psutil not installed.
Fix OpenAI streaming with backends and /v1 double-path bug
Add stream_openai_message() to LiteLLM and any-llm backends so
/v1/chat/completions with stream:true returns SSE events instead
of a JSON blob. Clients (Kilo Code, Cursor, etc.) were hanging
because the proxy ignored the stream flag when routing through
a backend.
Also strip trailing /v1 from OPENAI_TARGET_API_URL to prevent
double-path URLs like /v1/v1/models.
Move heavy deps to optional extras: sentence-transformers, torch,
numpy, pillow, datasets, accelerate out of core. Remove unused deps
entirely (semantic-router, protobuf, sentencepiece).
New extras: [ml] for Kompress, [image] for image compression,
[langchain] for LangChain integration. Guard memory/image imports
so core install works without numpy/torch.
Core (tiktoken, pydantic, litellm, click, rich) gives full
compression: SmartCrusher, ContentRouter, CCR, TOIN, CLI.
LiteLLM removed claude-3-5-sonnet-20241022 from its cost database.
Add alias fallback map so retired model names resolve to current
equivalents for pricing lookups.
- CLI now reads OPENAI_TARGET_API_URL, GEMINI_TARGET_API_URL, and
HEADROOM_ANYLLM_PROVIDER environment variables
- Add --openai-api-url and --gemini-api-url CLI flags
- Remove --backend choices restriction so litellm-* backends work
- Forward tools/tool_choice through LiteLLM and any-llm backends
- Parse tool call arguments from JSON string to dict (Anthropic format)
- Forward top_p, stop_sequences, tools in streaming paths
- Update Vertex AI model map with Claude 3 through 4.6 (from official docs)
- Bump version to 0.4.1
