akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix(compress): resolve the /v1/compress tokenizer per model, and document the real contract (#2743)

Browse source 
## Description

`/v1/compress` does no format conversion — callers send whichever wire
shape they already use — but the pipeline pinned **one provider's token
counter for the whole route**.

`OpenAITokenCounter.count_message` walks list content for `text` and
`image_url` only and has **no else branch**, so Anthropic content blocks
contributed literally zero. A 599-token `tool_result` scored 8. A
request that really removed 235 characters reported `tokens_saved: 0` —
so a caller gating on `tokens_saved > 0` concludes compression is broken
while it is working.

Prompted by a Kong integration question ("do you support the Anthropic
native format?"). The answer is that we already did — we just reported
zeros for it, and the docs said otherwise.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

### Tokenizer resolution (no hardcoded lists)

Build the derived pipelines with `provider=None` so `TransformPipeline`
resolves the tokenizer from the **per-model registry**. Every registry
tokenizer derives from `BaseTokenizer`, whose `_count_content_parts`
ends in a serialize-and-count catch-all, which means:

- No block type counts as zero, and there is **no per-provider
block-type list to keep in sync**. An enumerated set was the first thing
I tried and it already missed `mcp_tool_result`,
`web_search_tool_result`, `document`, and `thinking`.
- Gemini / Mistral / DeepSeek / Kimi stop defaulting to a tiktoken count
when the registry already has a calibrated counter for them.
- Gateway aliases matching no vendor pattern still count correctly.

`mode="ccr"` now runs a derived pipeline too, for the same reason —
sharing `openai_pipeline` pinned its provider. Costs that mode its own
cold compression cache; correct metrics win.

### Tokenizer selection stays separate from context-limit resolution

Deliberately not welded together. `model_limit` feeds `context_pressure
-> min_ratio`, so letting a tokenizer decision pick the limit table
changes compression aggressiveness: `gpt-4-32k` answered by the
Anthropic table is **8,192 instead of 32,768**, a 4× under-estimate.
`test_tokenizer_choice_does_not_move_the_context_limit` pins the
independence.

### Docs, rewritten from the code

- **`proxy.mdx`** — the loopback-only default and **404-not-403**
behavior, previously undocumented *anywhere* in `docs/` despite shipping
in #2458 explicitly for gateway sidecars;
`HEADROOM_COMPRESS_ALLOW_REMOTE`; all four request fields; the whole
`config` object including every `mode` value and `frozen_message_count`;
`transforms_summary`; the 400/401/404/503 contract; and the timeout
fail-open shape (`compression_skipped` / `skip_reason`).
- **Corrected "never calls an LLM"** — accurate about *generative*
provider requests, misleading for a sidecar operator. Kompress (a
ModernBERT **encoder**, classification not generation) and Magika run
**in-process**, and `HEADROOM_KOMPRESS_ENDPOINT` offloads inference over
HTTP — **real egress**. Now stated explicitly, with
`HEADROOM_DISABLE_KOMPRESS=1` as the structural-only option.
- **Both wire formats documented as accepted**, and removed
`anthropic-sdk.mdx`'s claim that OpenAI format is "the compression
engine's native format" — the exact misconception that prompted this
work. The SDK's conversion is now framed as an SDK choice, not an API
requirement.
- **`litellm.mdx`** had no mention of the endpoint at all, despite the
code naming LiteLLM's guardrail as its primary consumer. Added the HTTP
deployment path, the `HEADROOM_COMPRESS_ALLOW_REMOTE` requirement, and
why to leave `config.mode` unset.
- **`index.mdx`** printed `compressionRatio * 100` labelled "Saved …%",
so a 77% saving displayed as **23%**. `api-reference.mdx` already
defined it correctly, so the docs contradicted each other.
- `openai-sdk.mdx`, `wiki/proxy.md`, `wiki/typescript-sdk.md` — same
corrections; dropped "any HTTP client", "Cloud", and a CacheAligner
claim (it is detector-only).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/ruff check headroom/ tests/ --exclude headroom/dashboard/templates
All checks passed!

$ .venv/bin/mypy headroom/
Success: no issues found in 511 source files

$ python -m pytest tests/test_compress_route_tokenizer_by_model.py \
    tests/test_proxy_compress_endpoint.py tests/test_compress_api.py \
    tests/test_platform_stabilization_functional.py tests/test_proxy_eager_preload_bind.py -q
99 passed, 2 warnings in 47.15s
```

Broader sweep (`-k "compress or litellm or gateway or guardrail"`):
**1625 passed, 4 failed** — all 4 pre-existing, verified by stashing
this diff and re-running on clean `main` (2 strands hook tests, 1 codex
WS semaphore-tail timing test, 1 unrelated local WIP test).

## Real Behavior Proof

- **Environment:** macOS 26.4 arm64, Python 3.12.6, repo `.venv`, branch
rebased on `upstream/main`.

**(1) Before → after, same request** (60-line grep payload in an
Anthropic `tool_result`):

| model | before | after |
| --- | --- | --- |
| `claude-sonnet-4-6` | `before=28 saved=0` | `before=1223 saved=58` |
| `bedrock/anthropic.claude-3-5-sonnet` | `saved=0` | `before=1037
saved=59` |
| `my-gateway/big-model` (alias) | `saved=0` | `before=1037 saved=59` |
| `gemini-2.5-pro` | `saved=0` | `before=1036 saved=59` |
| `gpt-4o` + OpenAI shape | `before=1225 saved=58` | `before=1225
saved=58` (unchanged) |

All three `config.mode` values verified for each. Response shape
preserved: `type=tool_result`, `tool_use_id` intact.

**(2) Counter-level root cause**, 6.8 KB body, `count_message()`:

```text
OpenAITokenCounter    string-content -> 1406    tool_result block -> 5
registry (BaseTokenizer) claude       tool_result=408  thinking=418  mcp_tool_result=421
                                      web_search_tool_result=421  document=422
```

**(3) Every documented behavior asserted against the running app** — 13
checks, all PASS: 400s for missing `messages`/`model`, invalid
`config.mode`, and all four invalid `frozen_message_count` forms; 200
for valid ones; non-dict `config` ignored; bypass and empty-messages
omit `transforms_summary`; success returns exactly the 8 documented
keys.

- **Not tested:** the docs site was not built (`docs/node_modules`
absent) — MDX was checked for balanced `<Callout>` tags only, so a
reviewer with the site running should eyeball rendering. No live
gateway/Kong request; verification is via `TestClient` against the real
ASGI app.
- **Note:** `HEADROOM_DISABLE_KOMPRESS` is read into `ProxyConfig` at
`server.py:4919` and by the CLI, not by `create_app(ProxyConfig(...))`
directly — I confirmed `disable_kompress=True` does reach the derived
pipeline.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

🤖 Generated with [Claude Code](https://claude.com/claude-code)
This commit is contained in:
Tejas Chopra 2026-08-03 12:20:33 -07:00 • committed by GitHub
parent 789a4f3060
commit 6422a80a58
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 566 additions and 39 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/content/docs/anthropic-sdk.mdx
View file

@ -41,7 +41,7 @@ Every call to `client.messages.create()` compresses messages first. The response
`withHeadroom()` returns a proxy around your Anthropic client that intercepts `messages.create()`: `withHeadroom()` returns a proxy around your Anthropic client that intercepts `messages.create()`:
1. Converts Anthropic-format messages to OpenAI format (the compression engine's native format) 1. Converts Anthropic-format messages to OpenAI format
2. Sends them to the Headroom proxy's `/v1/compress` endpoint 2. Sends them to the Headroom proxy's `/v1/compress` endpoint
3. Converts the compressed messages back to Anthropic format 3. Converts the compressed messages back to Anthropic format
4. Forwards the request to Anthropic as normal 4. Forwards the request to Anthropic as normal
@ -58,6 +58,10 @@ The adapter handles the full Anthropic message format including content blocks:
This conversion is lossless. Your request and response behave identically to an unwrapped client. This conversion is lossless. Your request and response behave identically to an unwrapped client.
<Callout type="info" title="The conversion is an SDK choice, not an API requirement">
`POST /v1/compress` does no format conversion and compresses Anthropic content blocks natively — see [Message format](/docs/proxy#message-format). If you are calling the endpoint directly (from a gateway, or LiteLLM's `headroom` guardrail), send Anthropic-shaped messages as-is; you get the same shape back. Only this TypeScript adapter converts, because it normalises on OpenAI types internally.
</Callout>
## Options ## Options
Pass compression options as the second argument: Pass compression options as the second argument:

3
docs/content/docs/index.mdx
View file

@ -17,7 +17,8 @@ const messages = [
]; ];
const result = await compress(messages, { model: 'gpt-4o' }); const result = await compress(messages, { model: 'gpt-4o' });
console.log(`Saved ${result.tokensSaved} tokens (${(result.compressionRatio * 100).toFixed(0)}%)`); // compressionRatio is tokensAfter / tokensBefore, so savings is 1 - ratio.
console.log(`Saved ${result.tokensSaved} tokens (${((1 - result.compressionRatio) * 100).toFixed(0)}%)`);
``` ```
</Tab> </Tab>
<Tab value="Python"> <Tab value="Python">

21
docs/content/docs/litellm.mdx
View file

@ -94,3 +94,24 @@ app.add_middleware(CompressionMiddleware)
``` ```
Response headers include `x-headroom-compressed: true` and `x-headroom-tokens-saved: 1234`. Response headers include `x-headroom-compressed: true` and `x-headroom-tokens-saved: 1234`.
## Over HTTP (guardrail / gateway)
The options above run Headroom **in** the LiteLLM process. If instead LiteLLM runs as its own proxy and you want it to call Headroom over the network — the guardrail deployment — point it at [`POST /v1/compress`](/docs/proxy#post-v1compress). LiteLLM swaps `messages` for the compressed result and forwards to the provider.
Two things this deployment needs:
```bash
# Headroom is loopback-only by default and answers remote callers with 404.
HEADROOM_COMPRESS_ALLOW_REMOTE=1 headroom proxy
```
<Callout type="warning" title="404 means blocked, not missing">
Without `HEADROOM_COMPRESS_ALLOW_REMOTE=1` a remote caller gets `404`, not `403` — so a misconfigured guardrail looks exactly like a wrong URL. If you also set `HEADROOM_PROXY_TOKEN`, send it as `X-Headroom-Proxy-Token` or you get `401`.
</Callout>
Leave `config.mode` unset. The default pipeline is marker-free, which is what a forward-only caller wants: `mode: "ccr"` emits retrieval markers that are a dangling pointer unless you also inject the `headroom_retrieve` tool and can reach `/v1/retrieve`.
Because LiteLLM passes model names through, send the real one — `claude-sonnet-4-6`, `bedrock/anthropic.claude-3-5-sonnet`, `gemini-2.5-pro` — so Headroom resolves the right tokenizer and context limit. Anthropic-shaped messages need no conversion; see [Message format](/docs/proxy#message-format).
For multi-turn agent loops, set `config.frozen_message_count` to the number of messages the provider has already cached, **and send back the messages you previously forwarded rather than the pristine originals**. Getting this wrong silently destroys the provider's prefix cache — see [Multi-turn usage](/docs/proxy#multi-turn-usage-keeping-the-prefix-cache) for the loop.

4
docs/content/docs/openai-sdk.mdx
View file

@ -42,10 +42,12 @@ That's it. Every call to `client.chat.completions.create()` compresses the messa
`withHeadroom()` returns a proxy around your OpenAI client that intercepts `chat.completions.create()`: `withHeadroom()` returns a proxy around your OpenAI client that intercepts `chat.completions.create()`:
1. Extracts `messages` from the request params 1. Extracts `messages` from the request params
2. Sends them to the Headroom proxy's `/v1/compress` endpoint 2. Sends them to the Headroom proxy's [`POST /v1/compress`](/docs/proxy#post-v1compress) endpoint
3. Replaces the original messages with the compressed result 3. Replaces the original messages with the compressed result
4. Forwards the request to OpenAI as normal 4. Forwards the request to OpenAI as normal
The SDK talks to a **local** proxy, which is why no extra configuration is needed: `/v1/compress` is loopback-only by default. If you move the proxy to another host, set `HEADROOM_COMPRESS_ALLOW_REMOTE=1` on it or requests come back `404`.
All other client methods are untouched: All other client methods are untouched:
```ts twoslash ```ts twoslash

146
docs/content/docs/proxy.mdx
View file

@ -294,6 +294,7 @@ See [Metrics](/docs/metrics) for the Prometheus and Grafana setup.
| Flag / env | Default | Effect | | Flag / env | Default | Effect |
|---|---|---| |---|---|---|
| `HEADROOM_PROXY_TOKEN` | none | Require a bearer token (`X-Headroom-Proxy-Token`) from non-loopback callers. | | `HEADROOM_PROXY_TOKEN` | none | Require a bearer token (`X-Headroom-Proxy-Token`) from non-loopback callers. |
| `HEADROOM_COMPRESS_ALLOW_REMOTE` | `false` | Allow non-loopback callers to reach [`POST /v1/compress`](#post-v1compress). Required to run Headroom as a gateway/sidecar; without it remote callers get `404`. |
| `--offline` / `HEADROOM_OFFLINE` | `false` | Air-gap mode: hard-disable **all** egress (telemetry, update checks, license reporting, model downloads). | | `--offline` / `HEADROOM_OFFLINE` | `false` | Air-gap mode: hard-disable **all** egress (telemetry, update checks, license reporting, model downloads). |
| `--stateless` / `HEADROOM_STATELESS` | `false` | Keep all state in memory; no filesystem writes (disables logs, memory, TOIN). | | `--stateless` / `HEADROOM_STATELESS` | `false` | Keep all state in memory; no filesystem writes (disables logs, memory, TOIN). |
| `HEADROOM_STRIP_INTERNAL_HEADERS` | `enabled` | Strip internal `x-headroom-*` headers before forwarding upstream. | | `HEADROOM_STRIP_INTERNAL_HEADERS` | `enabled` | Strip internal `x-headroom-*` headers before forwarding upstream. |
@ -410,17 +411,79 @@ The proxy also accepts:
### `POST /v1/compress` ### `POST /v1/compress`
Compression-only endpoint. Compresses messages without calling any LLM. Used by the TypeScript SDK. Compression-only endpoint. Compresses messages and returns them without ever making a **completion request to an LLM provider** — no generation, no provider API key, no upstream chat call. Used by the TypeScript SDK, by LiteLLM's `headroom` guardrail, and by API gateways running Headroom as a sidecar.
**Request:** <Callout type="info" title="It does run local ML models">
```json "No LLM call" means no *generative* request to a provider. Compression itself is ML-backed: **Kompress** is a ModernBERT encoder that scores tokens for retention (classification, not generation), and Magika classifies content types. Both run in-process by default, so budget CPU and memory for the sidecar accordingly.
{
"messages": [{ "role": "user", "content": "..." }], If `HEADROOM_KOMPRESS_ENDPOINT` is set, Kompress inference is offloaded over HTTP to that model server — **real egress from the sidecar**, which matters if you deployed it expecting none. Only inference goes remote: the CCR store and retrieval markers stay proxy-local, and original content never persists off-box. Leave the variable unset to keep everything in-process, or run with `HEADROOM_DISABLE_KOMPRESS=1` for structural compression only.
"model": "gpt-4o" </Callout>
}
``` <Callout type="warning" title="Loopback-only by default">
This route is restricted to loopback callers and answers everyone else with **`404`**, not `403` — deliberately, so it stays invisible to external scanners. A gateway calling it from another host or pod therefore sees what looks like a missing route.
Both the client IP and the inbound `Host:` header must name loopback. To allow remote callers, set `HEADROOM_COMPRESS_ALLOW_REMOTE=1`. `HEADROOM_PROXY_TOKEN` still applies if set.
</Callout>
#### Message format
The endpoint does **no format conversion**. Whatever shape you send in `messages` is the shape you get back, and both wire formats are compressed natively:
- **OpenAI shape** — `role: "tool"` messages with `tool_call_id`, assistant `tool_calls`
- **Anthropic shape** — content-block lists with `tool_use` / `tool_result` / `thinking` blocks
So an Anthropic-native caller does not need to convert to OpenAI format first. Block types, `tool_use_id`s and message order are all preserved.
`model` selects the tokenizer (per-model, from Headroom's tokenizer registry) and the context limit. Send the real model name — including gateway-prefixed forms like `bedrock/anthropic.claude-3-5-sonnet` or `vertex_ai/claude-sonnet-4@20250514` — so token counts and compression aggressiveness are right.
#### Request
| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `messages` | array | yes | Messages to compress, in either wire format. `400` if missing. Empty array returns immediately with zero metrics. |
| `model` | string | yes | Model name. Drives tokenizer + context-limit resolution. `400` if missing. |
| `token_budget` | integer | no | Overrides the model's context limit. Used by callers that need to fit a tighter budget. |
| `config` | object | no | Compression options, below. A non-object value is ignored rather than rejected. |
<Callout type="warning" title="system and tools are ignored">
Only the four fields above are read. Anthropic sends `system` and `tools` **out of band**, alongside `messages` — this endpoint accepts them without complaint (you get a `200`, no warning) and returns neither, so neither is compressed.
Keep carrying both yourself and send them upstream unchanged. Two consequences worth knowing:
- An Anthropic system prompt is not compressed here, even though it is resent on every request.
- Tool-schema compaction and tool-search deferral are not reachable through this endpoint — on tool-heavy traffic those can be the largest share of available savings. Run Headroom as the proxy (rather than calling `/v1/compress`) if you need them.
</Callout>
`config` fields:
| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `mode` | string | unset | `ccr`, `lossy_inline`, or `lossless_then_lossy`. Unset selects the default marker-free pipeline. Any other value is a `400`. |
| `frozen_message_count` | integer | unset | Pin a prefix: the first N messages are returned byte-for-byte unchanged while staying visible to cross-message transforms like dedup. Set it to the number of messages the provider has already cached so compression cannot rewrite the prefix and bust that cache. Must be a non-negative integer; anything else is a `400`. |
| `compress_user_messages` | boolean | `false` | Also compress user-role messages. |
| `target_ratio` | number | unset | Target compression ratio. |
| `protect_recent` | integer | unset | Leave the last N messages uncompressed. |
| `protect_analysis_context` | boolean | unset | Preserve analysis context blocks. |
**`config.mode` values:**
- **unset (default)** — marker-free. Emits no `<<ccr:…>>` retrieval markers and writes nothing to the CCR store, so you can forward the returned messages straight to a provider. This is the right mode for a gateway or guardrail that just swaps `messages` and forwards.
- **`ccr`** — emits CCR markers and writes to the store. Only for callers that also inject the `headroom_retrieve` tool *and* can reach `/v1/retrieve` (itself loopback-only). Markers are a dangling pointer for the model otherwise.
- **`lossy_inline`** (alias `lossless_then_lossy`) — runs the lossless byte/data fold first, then compresses the folded remainder. Marker-free.
#### Response
| Field | Type | Description |
| --- | --- | --- |
| `messages` | array | Compressed messages, in the shape you sent. |
| `tokens_before` | integer | Token count before compression. |
| `tokens_after` | integer | Token count after compression. |
| `tokens_saved` | integer | `tokens_before - tokens_after`. |
| `compression_ratio` | number | `tokens_after / tokens_before` — so **lower is better**. A ratio of `0.23` means a 77% reduction, not 23%. `1.0` when nothing was compressed. |
| `transforms_applied` | array | Transform labels that ran. |
| `transforms_summary` | object | Per-transform counts. |
| `ccr_hashes` | array | Retrieval hashes for markers inserted (empty unless `mode: "ccr"`). |
**Response:**
```json ```json
{ {
"messages": [{ "role": "user", "content": "..." }], "messages": [{ "role": "user", "content": "..." }],
@ -429,11 +492,72 @@ Compression-only endpoint. Compresses messages without calling any LLM. Used by
"tokens_saved": 11500, "tokens_saved": 11500,
"compression_ratio": 0.23, "compression_ratio": 0.23,
"transforms_applied": ["router:smart_crusher:0.35"], "transforms_applied": ["router:smart_crusher:0.35"],
"ccr_hashes": ["a1b2c3"] "transforms_summary": { "router:smart_crusher:0.35": 1 },
"ccr_hashes": []
} }
``` ```
Set `x-headroom-bypass: true` to skip compression. #### Headers
`x-headroom-bypass: true` (case-insensitive) skips compression entirely and echoes your messages back with zeroed metrics. The bypass and empty-messages responses omit `transforms_summary`.
#### Errors and fail-open
| Status | Body | When |
| --- | --- | --- |
| `400` | `error.type = "invalid_request"` | Missing `messages` or `model`, malformed JSON, invalid `config.mode`, or invalid `config.frozen_message_count`. |
| `401` | — | `HEADROOM_PROXY_TOKEN` is set and the bearer token is missing or wrong. |
| `404` | — | Non-loopback caller without `HEADROOM_COMPRESS_ALLOW_REMOTE=1`. |
| `503` | `error.type = "compression_error"` | Compression failed unexpectedly. |
Compression **fails open on timeout**: you get `200` with your original messages, zeroed metrics, plus `compression_skipped: true` and `skip_reason: "compression_timeout"`. Always check `compression_skipped` if you need to know whether compression actually ran.
Requests are recorded under `provider="compress"` in `/stats` and `/metrics`.
#### Multi-turn usage: keeping the prefix cache
This is the single most important thing to get right, and the default is not safe for an agent loop.
When Headroom proxies a request itself it watches the provider's cache hit rate turn over turn and freezes the already-cached prefix. `/v1/compress` **cannot do that — it is stateless.** It sees one isolated call and has no idea what the provider already cached.
The provider caches the bytes you **forwarded**. Compression changed those bytes, so your original messages and the ones the provider cached are no longer the same thing — and it is the forwarded version you have to keep reproducing. Send the pristine originals again next turn and the provider sees a different prefix and re-reads it from scratch. On Anthropic a cache read is ~90% cheaper than fresh input, so that can easily cost more than the compression saves.
Compression is also not uniform over a conversation: how hard a message is compressed depends partly on how far it now sits from the end, so an older tool result can fall outside the recent-read protection window as the conversation grows and be compressed harder than it was last turn. Another reason not to rely on re-compression reproducing earlier output.
Two rules:
1. **Pass `config.frozen_message_count`** — how many leading messages the provider has already cached.
2. **Send back your own previous output, not the original messages.** `frozen_message_count` returns those leading messages *exactly as you passed them in* — it pins whatever you hand it. Hand it pristine originals and you get pristine originals back, which is precisely the prefix the provider does not have.
```python
# Keep what you FORWARDED, not what you started with.
forwarded: list[dict] = []
def next_turn(new_messages: list[dict]) -> list[dict]:
body = {
"messages": forwarded + new_messages,
"model": "claude-sonnet-4-6",
# Everything already forwarded is already cached upstream — pin it.
"config": {"frozen_message_count": len(forwarded)},
}
result = requests.post(f"{proxy}/v1/compress", json=body).json()
forwarded[:] = result["messages"] # becomes next turn's frozen prefix
return forwarded
```
<Callout type="warning" title="Re-sending pristine messages every turn silently busts the cache">
Compressing the full original conversation on each turn looks correct — you get a `200` and a positive `tokens_saved` — but the leading messages come back different from the ones the provider cached. You pay for compression *and* for a cache miss. Nothing in the response tells you this happened; watch your provider's cache-read tokens.
</Callout>
Also for multi-turn callers:
- **Leave `config.mode` unset.** The default is marker-free, which is what a forward-only caller wants.
- **Send the real model name** so the tokenizer and context limit resolve correctly — including gateway-prefixed forms.
- **`protect_recent` is not a substitute.** It guards the newest messages; `frozen_message_count` guards the oldest, which is the cached end.
<Callout type="info" title="Not the same as HEADROOM_KOMPRESS_ENDPOINT">
`HEADROOM_KOMPRESS_ENDPOINT` points *outbound* at a remote Kompress ML model server that happens to expose a `/compress` path. It is unrelated to this inbound endpoint.
</Callout>
## Agent wrapping ## Agent wrapping

66
headroom/proxy/handlers/openai.py
View file

@ -8325,6 +8325,28 @@ class OpenAIHandlerMixin:
marker-free path and vice versa. Sharing would be a correctness bug, marker-free path and vice versa. Sharing would be a correctness bug,
not an optimisation — the duplicate cache is the intended trade. not an optimisation — the duplicate cache is the intended trade.
Built with ``provider=None`` on purpose: ``TransformPipeline`` then
resolves the tokenizer from the per-model registry
(``headroom.tokenizers.get_tokenizer``) on every call instead of pinning
one provider's counter for the whole route.
That matters because this route does no format conversion — callers send
whichever wire shape they already use. The provider counters walk only the
block types they own (``OpenAITokenCounter`` handles ``text`` and
``image_url``; everything else falls through with NO else branch, so an
Anthropic ``tool_result`` contributed literally zero and ``tokens_saved``
reported 0 on requests where compression really ran). Every registry
tokenizer derives from ``BaseTokenizer``, whose ``_count_content_parts``
ends in a serialize-and-count catch-all, so no block type counts as zero
and there is no per-provider type list to keep in sync. It also stops
defaulting Gemini/Mistral/DeepSeek/Kimi traffic to a tiktoken count when
the registry already has a calibrated counter for them.
Note the pipeline's own ``_provider_name()`` is therefore ``None`` here.
That is honest for a provider-agnostic route — it used to report
``"openai"`` even for Claude payloads — and the request outcome still
records ``provider="compress"`` for /stats and /metrics.
ponytail: a first-request race just builds it twice — both are ponytail: a first-request race just builds it twice — both are
equivalent and Kompress weights are cached at module level, so no lock. equivalent and Kompress weights are cached at module level, so no lock.
""" """
@ -8344,7 +8366,7 @@ class OpenAIHandlerMixin:
return self.openai_pipeline return self.openai_pipeline
pipeline = TransformPipeline( pipeline = TransformPipeline(
transforms=[ContentRouter(replace(base.config, **overrides), observer=self.metrics)], transforms=[ContentRouter(replace(base.config, **overrides), observer=self.metrics)],
provider=self.openai_provider, provider=None, # per-model registry tokenizer — see docstring
) )
cache[key] = pipeline cache[key] = pipeline
return pipeline return pipeline
@ -8372,6 +8394,16 @@ class OpenAIHandlerMixin:
ccr_enabled=False, # no CCR store writes ccr_enabled=False, # no CCR store writes
) )
def _ccr_pipeline(self) -> Any:
"""Pipeline for ``/v1/compress`` ``config.mode="ccr"``.
No config overrides: markers and CCR store writes are exactly what this
mode asks for, so it inherits the live router's settings verbatim. It is
still a DERIVED pipeline rather than ``openai_pipeline`` so the tokenizer
comes from the per-model registry instead of a pinned provider counter.
"""
return self._derived_compress_pipeline("ccr")
def _lossy_inline_pipeline(self) -> Any: def _lossy_inline_pipeline(self) -> Any:
"""Pipeline for ``/v1/compress`` ``config.mode="lossy_inline"``. """Pipeline for ``/v1/compress`` ``config.mode="lossy_inline"``.
@ -8503,17 +8535,21 @@ class OpenAIHandlerMixin:
# Allow optional token_budget to override model's context limit # Allow optional token_budget to override model's context limit
# (used by OpenClaw compact() and other callers that need tighter budgets) # (used by OpenClaw compact() and other callers that need tighter budgets)
token_budget = body.get("token_budget") token_budget = body.get("token_budget")
# Resolve the context limit against the model's own provider. This # Resolve the CONTEXT LIMIT against the model's own provider. This
# route is OpenAI-shaped, but LiteLLM's `headroom` guardrail passes # route accepts either wire shape, and LiteLLM's `headroom` guardrail
# Anthropic model names straight through (claude-sonnet-4-5-..., # passes Anthropic model names straight through
# bedrock/anthropic.claude-3-5-sonnet, anthropic/claude-opus-4), and # (claude-sonnet-4-5-..., bedrock/anthropic.claude-3-5-sonnet,
# the OpenAI provider answers those with its 128K default instead of # anthropic/claude-opus-4), where the OpenAI provider answers with its
# 200K+. Substring match is enough here — no model registry. # 128K default instead of 200K+. Substring match is enough — no model
# registry.
# #
# Known gap: the TOKENIZER still comes from the OpenAI pipeline's # Deliberately separate from tokenizer selection, which the derived
# provider, so token counts remain approximate for Claude models. # pipelines resolve per model from the tokenizer registry (see
# Fixing that means selecting the whole pipeline per model family, # _derived_compress_pipeline). Welding the two together would let a
# which is a larger change and out of scope for this route. # tokenizer decision move `model_limit`, and `model_limit` feeds
# context_pressure -> min_ratio: e.g. gpt-4-32k answered by the
# Anthropic table is 8,192 instead of 32,768, a 4x under-estimate that
# silently changes compression aggressiveness.
model_name = model if isinstance(model, str) else str(model) model_name = model if isinstance(model, str) else str(model)
limit_provider = ( limit_provider = (
self.anthropic_provider self.anthropic_provider
@ -8578,7 +8614,13 @@ class OpenAIHandlerMixin:
if mode in ("lossy_inline", "lossless_then_lossy"): if mode in ("lossy_inline", "lossless_then_lossy"):
pipeline = self._lossy_inline_pipeline() pipeline = self._lossy_inline_pipeline()
elif mode == "ccr": elif mode == "ccr":
pipeline = self.openai_pipeline # Markers + store writes wanted, so inherit the live router config
# unchanged — but as a DERIVED pipeline, not `openai_pipeline`
# itself, so the per-model registry tokenizer applies here too.
# Sharing the request path's instance pinned the OpenAI counter,
# which reports zero for Anthropic content blocks. Costs this mode
# its own (cold) compression cache; correct metrics win.
pipeline = self._ccr_pipeline()
else: else:
pipeline = self._no_ccr_pipeline() pipeline = self._no_ccr_pipeline()

289
tests/test_compress_route_tokenizer_by_model.py Normal file
View file

@ -0,0 +1,289 @@
"""`/v1/compress` must count tokens for ANY wire shape and ANY model family.
The route does no format conversion — callers send whichever shape they already
use. Pinning one provider's token counter for the whole route silently reported
zero savings for Anthropic-shaped payloads: ``OpenAITokenCounter.count_message``
walks list content for ``text`` and ``image_url`` only, and has no else branch,
so an Anthropic ``tool_result`` block contributed literally nothing. A real
request that removed 235 characters reported ``tokens_saved: 0``.
The derived pipelines are built with ``provider=None`` so ``TransformPipeline``
resolves the tokenizer from the per-model registry instead. Every registry
tokenizer derives from ``BaseTokenizer``, whose ``_count_content_parts`` ends in
a serialize-and-count catch-all, so no block type counts as zero and there is no
per-provider block-type list to keep in sync.
"""
from __future__ import annotations
import pytest
pytest.importorskip("fastapi")
from fastapi.testclient import TestClient
from headroom.proxy.server import ProxyConfig, create_app
# Big enough that any real tokenizer must report hundreds of tokens, and
# compressible so the router actually folds it (repeated grep-shaped lines).
_GREP = "\n".join(
f"src/module_{i}.py:{i * 7}: result = compute_value(item_{i}, flag=True)" for i in range(60)
)
@pytest.fixture
def client():
app = create_app(
ProxyConfig(
optimize=True,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
)
)
# /v1/compress is loopback-gated (#1227).
with TestClient(app, base_url="http://127.0.0.1", client=("127.0.0.1", 12345)) as c:
yield c
def _anthropic_messages() -> list[dict]:
"""Anthropic native shape: tool_use / tool_result content blocks."""
return [
{"role": "user", "content": [{"type": "text", "text": "find compute_value"}]},
{
"role": "assistant",
"content": [
{"type": "tool_use", "id": "c1", "name": "grep", "input": {"pattern": "compute"}}
],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "c1", "content": _GREP}],
},
]
def _openai_messages() -> list[dict]:
"""OpenAI native shape: tool_calls + role=tool."""
return [
{"role": "user", "content": "find compute_value"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "c1",
"type": "function",
"function": {"name": "grep", "arguments": '{"pattern":"compute"}'},
}
],
},
{"role": "tool", "tool_call_id": "c1", "content": _GREP},
]
# Model names spanning every routing path a gateway realistically sends, including
# a custom alias that matches no known vendor pattern.
_MODELS = [
"claude-sonnet-4-6",
"bedrock/anthropic.claude-3-5-sonnet",
"vertex_ai/claude-sonnet-4@20250514",
"gemini-2.5-pro",
"deepseek/deepseek-v4",
"moonshotai/kimi-k2",
"my-gateway/big-model",
"gpt-4o",
]
@pytest.mark.parametrize("model", _MODELS)
def test_anthropic_shape_is_counted_for_every_model_family(client, model):
"""No model name may produce a zero token count for Anthropic content blocks."""
response = client.post("/v1/compress", json={"messages": _anthropic_messages(), "model": model})
assert response.status_code == 200, response.text
body = response.json()
# The payload is ~4 KB of text. Any honest tokenizer reports hundreds; the
# pinned OpenAI counter reported 28 for this exact request.
assert body["tokens_before"] > 500, f"{model} undercounted: {body['tokens_before']}"
assert body["tokens_saved"] > 0, f"{model} reported no savings: {body}"
assert body["compression_ratio"] < 1.0
@pytest.mark.parametrize("model", _MODELS)
def test_openai_shape_still_counted_for_every_model_family(client, model):
"""The OpenAI-shaped path must not regress while fixing the Anthropic one."""
response = client.post("/v1/compress", json={"messages": _openai_messages(), "model": model})
assert response.status_code == 200, response.text
body = response.json()
assert body["tokens_before"] > 500, f"{model} undercounted: {body['tokens_before']}"
assert body["tokens_saved"] > 0, f"{model} reported no savings: {body}"
@pytest.mark.parametrize("mode", [None, "ccr", "lossy_inline", "lossless_then_lossy"])
def test_every_mode_counts_anthropic_shape(client, mode):
"""mode="ccr" used to share the request pipeline, which pinned the OpenAI counter."""
payload: dict = {"messages": _anthropic_messages(), "model": "claude-sonnet-4-6"}
if mode is not None:
payload["config"] = {"mode": mode}
response = client.post("/v1/compress", json=payload)
assert response.status_code == 200, response.text
body = response.json()
assert body["tokens_before"] > 500, f"mode={mode} undercounted: {body['tokens_before']}"
assert body["tokens_saved"] > 0, f"mode={mode} reported no savings: {body}"
def test_response_preserves_the_anthropic_wire_shape(client):
"""Passthrough contract: no format conversion in either direction."""
response = client.post(
"/v1/compress",
json={"messages": _anthropic_messages(), "model": "claude-sonnet-4-6"},
)
assert response.status_code == 200
block = response.json()["messages"][2]["content"][0]
assert block["type"] == "tool_result"
assert block["tool_use_id"] == "c1"
# Content was folded, not dropped or restructured.
assert 0 < len(block["content"]) < len(_GREP)
def test_tokenizer_choice_does_not_move_the_context_limit(client):
"""Regression guard: the two resolutions must stay independent.
`model_limit` feeds context_pressure -> min_ratio, so letting a tokenizer
decision pick the limit table changes compression aggressiveness. gpt-4-32k
answered by the Anthropic table is 8,192 instead of 32,768 — a 4x
under-estimate — even though its payload needs a non-OpenAI tokenizer.
"""
seen: dict = {}
proxy = client.app.state.proxy
original = proxy._no_ccr_pipeline().apply
def spy(**kwargs):
seen.update(kwargs)
return original(**kwargs)
proxy._no_ccr_pipeline().apply = spy
try:
response = client.post(
"/v1/compress",
json={"messages": _anthropic_messages(), "model": "gpt-4-32k"},
)
finally:
proxy._no_ccr_pipeline().apply = original
assert response.status_code == 200
# OpenAI's table, because the MODEL is an OpenAI model — regardless of the
# Anthropic-shaped body that drives tokenizer selection.
assert seen["model_limit"] == 32_768
# ── The documented multi-turn recipe ──────────────────────────────────────────
# The endpoint is stateless: unlike the proxy's own request path it runs no
# CacheAligner and tracks no provider cache state, so keeping the prefix stable is
# the caller's job. docs/content/docs/proxy.mdx documents the loop; these two tests
# pin both halves of it so the guidance cannot rot.
def _turn(i: int) -> list[dict]:
body = "\n".join(f"src/mod_{i}_{j}.py:{j}: match compute_value(x{j})" for j in range(40))
return [
{
"role": "assistant",
"content": [{"type": "tool_use", "id": f"c{i}", "name": "grep", "input": {"p": "x"}}],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": f"c{i}", "content": body}],
},
]
def _compress(client, messages: list[dict], frozen: int | None = None) -> list[dict]:
body: dict = {"messages": messages, "model": "claude-sonnet-4-6"}
if frozen is not None:
body["config"] = {"frozen_message_count": frozen}
response = client.post("/v1/compress", json=body)
assert response.status_code == 200, response.text
return response.json()["messages"]
def test_system_and_tools_are_accepted_and_ignored(client):
"""Documented contract: only messages/model/token_budget/config are read.
Anthropic sends `system` and `tools` out of band. The endpoint takes them
without complaint and returns neither, so neither is compressed — callers must
keep carrying them. Pinned because the silence is the hazard: a caller has no
signal that the fields did nothing. If this ever starts returning them, the
contract changed and docs/content/docs/proxy.mdx needs updating with it.
"""
response = client.post(
"/v1/compress",
json={
"messages": _anthropic_messages(),
"model": "claude-sonnet-4-6",
"system": "You are a coding agent. " * 200,
"tools": [
{
"name": "read",
"description": "Read a file from disk. " * 20,
"input_schema": {"type": "object", "properties": {"path": {"type": "string"}}},
}
],
},
)
assert response.status_code == 200, response.text
body = response.json()
assert "system" not in body
assert "tools" not in body
# Messages are still compressed normally alongside the ignored fields.
assert body["tokens_saved"] > 0
def test_documented_loop_keeps_the_cached_prefix_byte_identical(client):
"""Feed back previous OUTPUT + frozen_message_count -> stable prefix every turn."""
import json
forwarded = _compress(client, [{"role": "user", "content": [{"type": "text", "text": "go"}]}])
for i in range(1, 6):
previous = forwarded
forwarded = _compress(client, previous + _turn(i), frozen=len(previous))
replayed = forwarded[: len(previous)]
assert [json.dumps(m, sort_keys=True) for m in replayed] == [
json.dumps(m, sort_keys=True) for m in previous
], f"turn {i} rewrote the cached prefix"
def test_frozen_prefix_replays_what_you_sent_not_what_you_forwarded(client):
"""Why re-sending pristine originals busts the cache — the documented trap.
`frozen_message_count` returns the leading messages *exactly as passed in*. So
the bytes you get back depend entirely on which version you sent: feed it your
previous OUTPUT and the prefix matches what the provider cached; feed it the
pristine ORIGINALS and you hand the provider different bytes for a message it
already cached, paying for compression and a cache miss at once.
"""
import json
base = [{"role": "user", "content": [{"type": "text", "text": "go"}]}]
originals = base + _turn(1)
forwarded = _compress(client, originals)
# Precondition: compression actually changed the prefix, so the two candidate
# inputs for next turn genuinely differ.
assert json.dumps(forwarded) != json.dumps(originals)
# Correct: previous output in, same bytes back.
good = _compress(client, forwarded + _turn(2), frozen=len(forwarded))
assert good[: len(forwarded)] == forwarded
# The trap: pristine originals in, pristine originals back — which is NOT what
# was forwarded last turn, so the provider's cached prefix no longer matches.
trap = _compress(client, originals + _turn(2), frozen=len(originals))
assert trap[: len(originals)] == originals
assert trap[: len(forwarded)] != forwarded

7
tests/test_proxy_compress_endpoint.py
View file

@ -671,7 +671,7 @@ class TestCompressEndpointDoesNotBlockLoop:
markers_inserted=[], markers_inserted=[],
) )
monkeypatch.setattr(proxy.openai_pipeline, "apply", blocking_apply) monkeypatch.setattr(proxy._ccr_pipeline(), "apply", blocking_apply)
# /v1/compress is loopback-gated (#1227) — present as 127.0.0.1. # /v1/compress is loopback-gated (#1227) — present as 127.0.0.1.
transport = httpx.ASGITransport(app=app, client=("127.0.0.1", 12345)) transport = httpx.ASGITransport(app=app, client=("127.0.0.1", 12345))
@ -682,8 +682,9 @@ class TestCompressEndpointDoesNotBlockLoop:
json={ json={
"messages": [{"role": "user", "content": "hello world"}], "messages": [{"role": "user", "content": "hello world"}],
"model": "gpt-4", "model": "gpt-4",
# mode="ccr" routes to `openai_pipeline` (the default is a # Every mode now runs a DERIVED pipeline so the tokenizer
# derived marker-free pipeline); this test patches that one. # comes from the per-model registry; mode="ccr" is the
# marker-on one, patched above.
"config": {"mode": "ccr"}, "config": {"mode": "ccr"},
}, },
) )

57
wiki/proxy.md
View file

@ -2,7 +2,7 @@
The Headroom proxy server is a production-ready HTTP server that applies context optimization to all requests passing through it. The Headroom proxy server is a production-ready HTTP server that applies context optimization to all requests passing through it.
> **New:** The proxy now supports the [TypeScript SDK](typescript-sdk.md) via the `POST /v1/compress` endpoint, enabling compression-as-a-service for any HTTP client without calling an LLM. > The proxy exposes compression-as-a-service via the `POST /v1/compress` endpoint — used by the [TypeScript SDK](typescript-sdk.md), LiteLLM's `headroom` guardrail, and gateway sidecars. It is loopback-only by default; see the endpoint section below.
## Starting the Proxy ## Starting the Proxy
@ -266,13 +266,30 @@ POST /v1/chat/completions
### `POST /v1/compress` ### `POST /v1/compress`
Compression-only endpoint. Compresses messages without calling any LLM. Used by the [TypeScript SDK](typescript-sdk.md) and any HTTP client that wants compression as a service. Compression-only endpoint. Compresses messages without ever making a **completion request to an LLM provider** — no generation, no provider API key. Used by the [TypeScript SDK](typescript-sdk.md), LiteLLM's `headroom` guardrail, and gateway sidecars.
**It does run local ML models.** Compression is ML-backed: Kompress is a ModernBERT encoder scoring tokens for retention (classification, not generation) and Magika classifies content types, both in-process by default. If `HEADROOM_KOMPRESS_ENDPOINT` is set, Kompress inference is offloaded over HTTP to that model server — real egress from the sidecar. Only inference goes remote; the CCR store and markers stay proxy-local. `HEADROOM_DISABLE_KOMPRESS=1` gives structural compression only.
**Loopback-only by default.** Non-loopback callers get **404** (not 403 — the route stays invisible to scanners). Set `HEADROOM_COMPRESS_ALLOW_REMOTE=1` to allow remote callers.
**No format conversion.** `messages` may be OpenAI-shaped (`role: "tool"` + `tool_call_id`) or Anthropic-shaped (`tool_use` / `tool_result` content blocks); the same shape comes back. `model` selects the tokenizer and context limit — send the real name, including gateway-prefixed forms like `bedrock/anthropic.claude-3-5-sonnet`.
**`system` and `tools` are ignored.** Anthropic sends both out of band. This endpoint accepts them without complaint (200, no warning) and returns neither, so neither is compressed — keep carrying them yourself. That means the Anthropic system prompt is not compressed here, and tool-schema compaction / tool-search deferral are not reachable through this route; run Headroom as the proxy if you need those.
**Request:** **Request:**
```json ```json
{ {
"messages": [...], // OpenAI chat format "messages": [...], // either wire format
"model": "gpt-4o" // model name (for token counting) "model": "gpt-4o", // tokenizer + context limit
"token_budget": 8000, // optional: override the context limit
"config": { // optional
"mode": "lossy_inline", // ccr | lossy_inline | lossless_then_lossy
"frozen_message_count": 12, // pin an already-cached prefix
"compress_user_messages": false,
"target_ratio": 0.5,
"protect_recent": 2,
"protect_analysis_context": true
}
} }
``` ```
@ -283,16 +300,40 @@ Compression-only endpoint. Compresses messages without calling any LLM. Used by
"tokens_before": 15000, "tokens_before": 15000,
"tokens_after": 3500, "tokens_after": 3500,
"tokens_saved": 11500, "tokens_saved": 11500,
"compression_ratio": 0.23, "compression_ratio": 0.23, // tokens_after / tokens_before — LOWER is better
"transforms_applied": ["router:smart_crusher:0.35"], "transforms_applied": ["router:smart_crusher:0.35"],
"ccr_hashes": ["a1b2c3"] "transforms_summary": {"router:smart_crusher:0.35": 1},
"ccr_hashes": [] // non-empty only with mode="ccr"
} }
``` ```
**Headers:** **Headers:**
- `x-headroom-bypass: true` — skip compression, return messages as-is - `x-headroom-bypass: true` — skip compression, return messages as-is with zeroed metrics
**Error responses:** 400 (missing fields), 401 (bad API key), 503 (compression failed) **Error responses:** 400 (missing/invalid fields, bad `config.mode` or `config.frozen_message_count`), 401 (bad `HEADROOM_PROXY_TOKEN`), 404 (non-loopback without `HEADROOM_COMPRESS_ALLOW_REMOTE=1`), 503 (compression failed)
**Fail-open:** on timeout you get 200 with the original messages plus `compression_skipped: true` and `skip_reason: "compression_timeout"`.
**Multi-turn callers — don't lose the prefix cache.** This endpoint is stateless: unlike the proxy's own request path (which runs a CacheAligner and tracks provider cache hits across turns), it has no idea what the provider already cached.
The provider caches the bytes you *forwarded*, which compression already changed — so your originals and the cached prefix are no longer the same thing, and it is the forwarded version you must keep reproducing. Compression also varies with position: an older tool result can fall outside the recent-read protection window as the conversation grows and be compressed harder than last turn, so re-compression is not guaranteed to reproduce earlier output either. Two rules:
1. Pass `config.frozen_message_count` = the number of leading messages already cached upstream.
2. Send back the messages you **previously forwarded**, not the pristine originals. `frozen_message_count` returns leading messages exactly as passed in, so feeding it originals hands the provider different bytes than last turn and busts the cache anyway.
```python
forwarded = []
def next_turn(new_messages):
r = requests.post(f"{proxy}/v1/compress", json={
"messages": forwarded + new_messages,
"model": "claude-sonnet-4-6",
"config": {"frozen_message_count": len(forwarded)},
}).json()
forwarded[:] = r["messages"] # next turn's frozen prefix
return forwarded
```
Note `protect_recent` is not a substitute — it guards the newest messages, while `frozen_message_count` guards the oldest, which is the cached end.
## Using with Claude Code ## Using with Claude Code

6
wiki/typescript-sdk.md
View file

@ -26,7 +26,9 @@ const response = await openai.chat.completions.create({
## How It Works ## How It Works
The TypeScript SDK is an HTTP client. When you call `compress()`, it sends your messages to the Headroom proxy's `POST /v1/compress` endpoint. The proxy runs the full compression pipeline (SmartCrusher, ContentRouter, CacheAligner, etc.) and returns compressed messages. No compression logic runs in Node.js — all the heavy lifting happens in the proxy. The TypeScript SDK is an HTTP client. When you call `compress()`, it sends your messages to the Headroom proxy's `POST /v1/compress` endpoint. The proxy runs the compression pipeline (ContentRouter and its compressors, including SmartCrusher) and returns compressed messages. No compression logic runs in Node.js — all the heavy lifting happens in the proxy.
The proxy must be reachable on **loopback**: `/v1/compress` rejects remote callers with `404` unless it was started with `HEADROOM_COMPRESS_ALLOW_REMOTE=1`.
``` ```
Your TypeScript App Your TypeScript App
@ -37,7 +39,7 @@ headroom-ai (npm) ← HTTP client
│ │
│ POST /v1/compress │ POST /v1/compress
▼ ▼
Headroom Proxy / Cloud ← compression pipeline (Python) Headroom Proxy (loopback) ← compression pipeline (Python)
│ │
│ compressed messages │ compressed messages
▼ ▼