docs(compress): state that /v1/compress ignores system and tools

No behaviour change — this documents current behaviour and pins it.

The endpoint reads only messages/model/token_budget/config. Anthropic sends
`system` and `tools` out of band, and the endpoint accepts both without complaint
(200, no warning) and returns neither, so neither is compressed. The silence is
the hazard: an Anthropic-native caller has no signal that the fields did nothing,
and the request table listing four fields does not tell them a fifth was dropped.

Spells out the two consequences: the Anthropic system prompt is not compressed
here despite being resent every request, and tool-schema compaction /
tool-search deferral are unreachable through this route — on tool-heavy traffic
those can be the largest share of available savings, so run Headroom as the proxy
if you need them.

A test pins the contract so it cannot drift silently; if the endpoint ever starts
returning these fields, the test fails and points at the doc that must change with
it.

ruff clean; 25 compress-route tests pass.
This commit is contained in:
Tejas Chopra 2026-08-03 12:06:23 -07:00
parent 2a06e82834
commit fd7f0d1841
3 changed files with 44 additions and 0 deletions

View file

@ -445,6 +445,15 @@ So an Anthropic-native caller does not need to convert to OpenAI format first. B
| `token_budget` | integer | no | Overrides the model's context limit. Used by callers that need to fit a tighter budget. |
| `config` | object | no | Compression options, below. A non-object value is ignored rather than rejected. |
<Callout type="warning" title="system and tools are ignored">
Only the four fields above are read. Anthropic sends `system` and `tools` **out of band**, alongside `messages` — this endpoint accepts them without complaint (you get a `200`, no warning) and returns neither, so neither is compressed.
Keep carrying both yourself and send them upstream unchanged. Two consequences worth knowing:
- An Anthropic system prompt is not compressed here, even though it is resent on every request.
- Tool-schema compaction and tool-search deferral are not reachable through this endpoint — on tool-heavy traffic those can be the largest share of available savings. Run Headroom as the proxy (rather than calling `/v1/compress`) if you need them.
</Callout>
`config` fields:
| Field | Type | Default | Description |

View file

@ -212,6 +212,39 @@ def _compress(client, messages: list[dict], frozen: int | None = None) -> list[d
return response.json()["messages"]
def test_system_and_tools_are_accepted_and_ignored(client):
"""Documented contract: only messages/model/token_budget/config are read.
Anthropic sends `system` and `tools` out of band. The endpoint takes them
without complaint and returns neither, so neither is compressed callers must
keep carrying them. Pinned because the silence is the hazard: a caller has no
signal that the fields did nothing. If this ever starts returning them, the
contract changed and docs/content/docs/proxy.mdx needs updating with it.
"""
response = client.post(
"/v1/compress",
json={
"messages": _anthropic_messages(),
"model": "claude-sonnet-4-6",
"system": "You are a coding agent. " * 200,
"tools": [
{
"name": "read",
"description": "Read a file from disk. " * 20,
"input_schema": {"type": "object", "properties": {"path": {"type": "string"}}},
}
],
},
)
assert response.status_code == 200, response.text
body = response.json()
assert "system" not in body
assert "tools" not in body
# Messages are still compressed normally alongside the ignored fields.
assert body["tokens_saved"] > 0
def test_documented_loop_keeps_the_cached_prefix_byte_identical(client):
"""Feed back previous OUTPUT + frozen_message_count -> stable prefix every turn."""
import json

View file

@ -274,6 +274,8 @@ Compression-only endpoint. Compresses messages without ever making a **completio
**No format conversion.** `messages` may be OpenAI-shaped (`role: "tool"` + `tool_call_id`) or Anthropic-shaped (`tool_use` / `tool_result` content blocks); the same shape comes back. `model` selects the tokenizer and context limit — send the real name, including gateway-prefixed forms like `bedrock/anthropic.claude-3-5-sonnet`.
**`system` and `tools` are ignored.** Anthropic sends both out of band. This endpoint accepts them without complaint (200, no warning) and returns neither, so neither is compressed — keep carrying them yourself. That means the Anthropic system prompt is not compressed here, and tool-schema compaction / tool-search deferral are not reachable through this route; run Headroom as the proxy if you need those.
**Request:**
```json
{