savings_usd is now tokens_saved * model list input price (monotonic,
transparent). Removed non-monotonic moving-average repricing and
confusing cost_without_headroom counterfactual.
Dashboard hero shows "Compression Savings" with clear subtitle.
Savings Breakdown section shows compression, cache, and RTK separately
with distinct colors and no scope mixing.
All beacon/telemetry fields preserved. RTK token counts still reported.
Fixes#83
KompressCompressor now tries ONNX Runtime first (156MB INT8 model),
falls back to PyTorch only if ONNX unavailable. No torch needed for
text compression — just onnxruntime (~50MB) + transformers (tokenizer).
Changes:
- Add onnxruntime + transformers to [proxy] extra in pyproject.toml
- Add _OnnxModel wrapper with get_scores/get_keep_mask interface
- _load_kompress() tries ONNX first, falls back to PyTorch
- is_kompress_available() returns True if EITHER backend available
- compress() handles both numpy (ONNX) and tensor (PyTorch) outputs
Dependency impact:
Before: pip install headroom-ai[proxy] → no text compression
After: pip install headroom-ai[proxy] → Kompress ONNX INT8 (156MB)
[ml] extra still available for full PyTorch (600MB, GPU support)
The /v1/responses handler was passing through without compression,
meaning Codex CLI users got zero savings. Now converts Responses API
items (function_call, function_call_output, reasoning, message) to
Chat Completions format, runs the existing pipeline, and converts back.
- New: headroom/proxy/responses_converter.py — pure conversion functions
- 21 unit tests + 3 integration tests (tested with real OpenAI API)
- Preserves reasoning items, images, unknown types verbatim
- Skips compression when previous_response_id is set
- 27% compression on real Codex-pattern payloads (500 records → 14K tokens saved)
Closes#73
Bedrock requires role=tool messages immediately after assistant tool_calls.
The previous fix inserted a user text message in between when the message
contained both text and tool_result blocks, breaking the pairing.
Drop text alongside tool_result (Claude Code never sends it in practice).
Added ordering regression tests for the Bedrock constraint.
_decode_project_path now detects single-letter first component as a
Windows drive letter: -C-MQ2-macros → C:\MQ2\macros instead of
/C/MQ2/macros (which becomes \\C\MQ2\macros on Windows).
- Add Windows drive detection before Unix path attempts
- Fix fallback path construction for Windows patterns
- Add Linux /home/ support in greedy decoder
- Add 2 tests for Windows drive letter patterns
OpenClaw plugin:
- Fix assistant content always returned as array (fixes flatMap crash)
- Clean up debug logging, restore clean plugin entry point
- Add test for text-only assistant round-trip
- Set headroom-ai dependency to npm (not local path)
Telemetry:
- Switch from sb_publishable_ to JWT anon key (PostgREST needs JWT for RLS)
- Switch from upsert to plain INSERT (anon role upsert was failing with 42501)
- Verified: full payload with all JSONB columns writes successfully (201)
Cost tracker:
- Use output_buffer instead of hardcoded 500 for cost estimation in client.py
- CostTracker counterfactual: value removed tokens at avg effective $/token
(actual_spend / billed_tokens) instead of uncached list price — aligns
savings with real billing mix (cache reads, writes, uncached)
- Add debug log for /stats summary payload
- Add 3 tests for new cost savings calculation
TypeScript SDK:
- Add tokenBudget param to compress(), HeadroomClient, CompressOptions
- Proxy /v1/compress accepts optional token_budget to override model limit
- New `compress()` function: HTTP client calling POST /v1/compress on the proxy
- HeadroomClient: reusable client with retry, fallback, auth support
- Vercel AI SDK adapter: headroomMiddleware() for wrapLanguageModel()
- OpenAI SDK adapter: withHeadroom() Proxy wrapper
- Anthropic SDK adapter: withHeadroom() Proxy wrapper
- Format converters: Vercel AI SDK ↔ OpenAI message format round-trip
- POST /v1/compress proxy endpoint: compression without LLM call
- 90 TypeScript tests (84 unit + 6 integration) + 9 Python tests
- Zero runtime dependencies, all framework peers optional
- Updated README, proxy docs, integration guide, and 6 other doc pages
- New docs/typescript-sdk.md with full SDK documentation
- Removed docs/superpowers/ from tracking (.gitignore)
Claude correctly rejects <system-reminder> in user messages as prompt
injection. Real workflow tags appear in tool outputs, not user messages.
Restructured test to use tool_call → tool result pattern.
Verified passing with real ANTHROPIC_API_KEY from .env.
LLMLingua was the original ML text compressor (BERT-based). Kompress
(ModernBERT, trained on 330K structured tool outputs) replaced it with
better compression quality and simpler architecture.
Removed across 35 files:
- Deleted headroom/transforms/llmlingua_compressor.py
- Deleted tests/test_transforms/test_llmlingua_compressor.py
- Deleted tests/test_proxy_llmlingua.py
- Removed all enable_llmlingua config, _get_llmlingua methods,
LLMLingua fallback paths, LLMLINGUA strategy enum values
- Removed CLI flags, model configs, compression handler references
- Simplified ContentRouter: Kompress is primary and only text compressor
LLM workflows use tags like <system-reminder>, <tool_call>, <thinking>
as structural markers. Kompress/LLMLingua treated these as droppable
HTML noise and silently removed them, breaking downstream tools.
Fix: tag_protector.py detects custom tags (anything NOT in KNOWN_HTML_TAGS),
replaces entire blocks with placeholders before compression, restores after.
Standard HTML tags are unaffected.
- KNOWN_HTML_TAGS: 120+ HTML5 Living Standard elements
- protect_tags / restore_tags utility functions
- Hooked into ContentRouter._try_ml_compressor
- Config: compress_tagged_content flag (default False)
- 28 new tests (unit + integration + real API gated by key)
Root fix: compute_optimal_k() now scales k with content diversity using
the SimHash uniqueness ratio already computed in the function.
diversity ~1.0 → keep 100% of items (all unique, dropping any loses info)
diversity ~0.5 → keep ~65%
diversity ~0.0 → keep ~30% (same as before for repetitive data)
No hardcoded RAG detection. No field name heuristics. Pure statistics —
works for any JSON array regardless of source (Pinecone, Chroma, Weaviate,
LangChain, custom APIs).
When all items are kept (high diversity), SmartCrusher tries to compress
text WITHIN each item's long string fields using Kompress (if available).
Falls back gracefully when Kompress is not installed.
Before: 12 unique RAG chunks → kept 2, dropped 10 (0/6 key concepts)
After: 12 unique RAG chunks → kept 12, compressed within (6/6 concepts)
Also adds tests/test_adaptive_sizer.py (16 tests covering high/low/moderate
diversity, knee interactions, bias, caps).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Update test_hash_uses_sha256_truncated → test_hash_uses_md5_truncated
to match the SHA256→MD5 change in compression_store.py
- Use errors="surrogatepass" in compute_hash to handle lone surrogates
in unicode content (fixes pre-existing UnicodeEncodeError)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three issues fixed:
1. _fetch_bedrock_inference_profiles crashed the proxy on startup when
boto3 was missing or the AWS API call failed (wrong credentials,
permissions, network). Now catches exceptions and falls back to a
static model map.
2. map_model_id produced invalid Bedrock model IDs for unmapped models.
Bare names like 'claude-sonnet-4-20250514' became
'bedrock/claude-sonnet-4-20250514' which is not a valid Bedrock
identifier. Now constructs region-prefixed IDs like
'bedrock/eu.anthropic.claude-sonnet-4-20250514-v1:0'.
3. No static fallback existed (_BEDROCK_MODEL_MAP was empty). Added
_build_bedrock_fallback_map() that generates region-aware model IDs
for all GA Claude models (us./eu./apac. prefixes).
Closes#28
Tests: 27 new tests covering region prefix mapping, static fallback map,
graceful degradation, and model ID mapping for EU/AP/US regions.
Three bugs fixed:
1. observations counter: get_recommendation() now increments an
observations counter on the ToolPattern, so we can track whether
TOIN is being consulted during compression.
2. streaming CCR feedback: when the LLM generates headroom_retrieve
tool_use in a streaming response, the proxy now extracts the
hash/query and calls store.retrieve() to trigger the full feedback
chain (_log_retrieval → process_pending_feedback → toin.record_retrieval).
Previously, streaming responses (all Claude Code traffic) never
triggered TOIN feedback, leaving preserve_fields empty.
3. preserve_fields population: with retrievals now flowing to TOIN,
field_retrieval_frequency gets populated, which feeds into
_build_recommendation() → preserve_fields in CompressionHint.
SmartCrusher can now learn which specific fields to preserve.
Root cause: headroom_retrieve tool calls in streaming responses were
returned to the client but never recorded as retrieval events in TOIN.
The entire feedback loop was write-only — TOIN learned compression
patterns but never learned from retrieval signals.
Adds 15 new tests covering observations counter, retrieval recording,
preserve_fields population, and CCR feedback extraction.
Add three methods and supporting helpers for token headroom mode:
- compute_frozen_count: counts consecutive stable messages from start
- apply_cached: swaps cached compressions into tool results (immutable)
- update_from_result: learns new compressions from original/compressed pairs
Supports both Anthropic and OpenAI tool result formats.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- SharedContext: compressed inter-agent context sharing via put()/get()
over existing CCR compression pipeline. Zero new dependencies.
- README rewrite: lead with "any agent" positioning, not just coding
agents. Add headroom wrap, SharedContext, MCP tools to Quick Start.
Reorder integration table: universal first, coding shortcuts last.
Update compression pipeline references (LLMLingua → Kompress).
- Fix proxy cleanup in headroom wrap: don't kill shared proxy if other
clients are still using it (was orphaning terminals 2-N).
- New docs: docs/shared-context.md
- Fix proxy crash when torch not installed: make kompress_compressor.py
imports lazy so `is_kompress_available()` works without [ml] extra
- Rewrite MCP server from 1 tool (retrieve-only) to 3 tools:
headroom_compress (on-demand compression, no proxy needed),
headroom_retrieve (local store first, proxy fallback),
headroom_stats (session stats + sub-agent aggregation + proxy cache)
- Add shared stats file (~/.headroom/session_stats.jsonl) so sub-agent
compression stats are visible from the main session
- Add mcp to [proxy] extras so proxy users get MCP tools automatically
- Remove dead TextCompressor from exports and pipeline (was never called)
- Update mcp install messaging to clarify proxy vs MCP roles
- Fix fcntl Windows compat, asyncio deprecation, httpx timeout race
Bump version to 0.4.6.
The proxy startup crashed with `ModuleNotFoundError: No module named
'torch'` when installed with just `[proxy]` extras because
kompress_compressor.py had unconditional top-level torch imports.
Moved torch/transformers imports to be lazy so the module is safely
importable without the [ml] extra. Added tests for import safety.
Bumped version to 0.4.5
- headroom wrap codex: injects rtk instructions into AGENTS.md
- headroom wrap cursor: injects into .cursorrules, prints config steps
- headroom wrap aider: injects into CONVENTIONS.md, sets both env vars
- All with --no-rtk flag to skip rtk setup
Unified savings metrics across three layers:
- cli_filtering: tokens avoided by rtk before reaching context
- compression: tokens removed by proxy (SmartCrusher, etc.)
- prefix_cache: provider cache discount with honest attribution
Fix flaky test_process_stats_collected when psutil not installed.
Fix OpenAI streaming with backends and /v1 double-path bug
Add stream_openai_message() to LiteLLM and any-llm backends so
/v1/chat/completions with stream:true returns SSE events instead
of a JSON blob. Clients (Kilo Code, Cursor, etc.) were hanging
because the proxy ignored the stream flag when routing through
a backend.
Also strip trailing /v1 from OPENAI_TARGET_API_URL to prevent
double-path URLs like /v1/v1/models.
Move heavy deps to optional extras: sentence-transformers, torch,
numpy, pillow, datasets, accelerate out of core. Remove unused deps
entirely (semantic-router, protobuf, sentencepiece).
New extras: [ml] for Kompress, [image] for image compression,
[langchain] for LangChain integration. Guard memory/image imports
so core install works without numpy/torch.
Core (tiktoken, pydantic, litellm, click, rich) gives full
compression: SmartCrusher, ContentRouter, CCR, TOIN, CLI.
LiteLLM removed claude-3-5-sonnet-20241022 from its cost database.
Add alias fallback map so retired model names resolve to current
equivalents for pricing lookups.
- CLI now reads OPENAI_TARGET_API_URL, GEMINI_TARGET_API_URL, and
HEADROOM_ANYLLM_PROVIDER environment variables
- Add --openai-api-url and --gemini-api-url CLI flags
- Remove --backend choices restriction so litellm-* backends work
- Forward tools/tool_choice through LiteLLM and any-llm backends
- Parse tool call arguments from JSON string to dict (Anthropic format)
- Forward top_p, stop_sequences, tools in streaming paths
- Update Vertex AI model map with Claude 3 through 4.6 (from official docs)
- Bump version to 0.4.1
