mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
8b3adeadd7
Replaces the ICM-based interceptor on /v1/messages and adds OpenAI
/v1/chat/completions and /v1/responses arms. The proxy now follows
exactly one rule, applied per-request:
Find the last user message. Freeze every byte before it. In the
new turn, walk for tool outputs and route them through the
CompressionPipeline (Diff/Log/Json offloads + JsonMinifier/
LogTemplate reformats). Everything else passes through.
Why this design:
- Provider prefix caches (Anthropic cache_control, OpenAI auto
prefix, OpenAI prompt_cache_key) are positional. We never modify
prefix bytes, so cache hits are preserved 100%.
- User text, system prompts, assistant text are never touched —
accuracy preserved.
- Tool outputs (file reads, search results, build logs, diffs)
are where the tokens are. The CompressionPipeline already in
headroom-core compresses them with CCR backup.
What this PR removes:
- ICM (IntelligentContextManager) is no longer wired into the
proxy. It's still in headroom-core for future use, but the
proxy crate doesn't import it. ICM message-dropping risks
cache busts that token mode rules out by construction.
What this PR adds (compression module rewrite):
- compression/walker.rs — compress_blob() shared helper
- compression/pipeline.rs — build CompressionPipeline + CcrStore
once at startup
- compression/anthropic.rs — token-mode walker (tool_result blocks)
- compression/openai.rs — token-mode walker (role:tool messages)
- compression/responses.rs — token-mode walker (function_call_output
items; previous_response_id skips)
- compression/mod.rs — endpoint classifier + dispatch
Per-shape rules documented per file. Highlights:
- Anthropic: walk tool_result blocks (string OR list); compress
text inside list-shaped blocks; preserve images
- OpenAI chat: only role:tool with string content
- OpenAI responses: only function_call_output with string output;
reasoning items, *_call items, image_generation_call all
preserved verbatim; previous_response_id triggers full
passthrough (server holds the conversation)
Tests:
- Unit (32 in compression module + 3 walker tests)
- Mock integration (8 tests, including prefix-byte-identical
assertion across Anthropic / OpenAI / Responses)
- Real OpenAI e2e (3 tests, gated on HEADROOM_E2E=1):
* /v1/chat/completions tool message: 70 prompt tokens received
(compressed from a ~120KB / ~30K-token raw log payload)
* /v1/responses function_call_output: 68 input tokens
(same payload, same compression)
* Prefix cache preservation: 3,456 of 3,518 tokens cached
on the second turn (98%) — proves byte-stable prefix
Verification:
- cargo test --workspace -> 904 passed, 0 failed
- cargo clippy --workspace --all-targets -- -D warnings -> clean
- cargo fmt --check -> clean
- HEADROOM_E2E=1 cargo test --test e2e_token_mode -> 3/3 pass
- make ci-precheck -> green
|
||
|---|---|---|
| .. | ||
| act | ||
| actions/headroom-e2e-setup | ||
| ISSUE_TEMPLATE | ||
| plugin | ||
| workflows | ||
| dependabot.yml | ||
| FUNDING.yml | ||
| PULL_REQUEST_TEMPLATE.md | ||