mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
docs(compress): state that /v1/compress ignores system and tools
No behaviour change — this documents current behaviour and pins it. The endpoint reads only messages/model/token_budget/config. Anthropic sends `system` and `tools` out of band, and the endpoint accepts both without complaint (200, no warning) and returns neither, so neither is compressed. The silence is the hazard: an Anthropic-native caller has no signal that the fields did nothing, and the request table listing four fields does not tell them a fifth was dropped. Spells out the two consequences: the Anthropic system prompt is not compressed here despite being resent every request, and tool-schema compaction / tool-search deferral are unreachable through this route — on tool-heavy traffic those can be the largest share of available savings, so run Headroom as the proxy if you need them. A test pins the contract so it cannot drift silently; if the endpoint ever starts returning these fields, the test fails and points at the doc that must change with it. ruff clean; 25 compress-route tests pass.
This commit is contained in:
parent
2a06e82834
commit
fd7f0d1841
3 changed files with 44 additions and 0 deletions
|
|
@ -445,6 +445,15 @@ So an Anthropic-native caller does not need to convert to OpenAI format first. B
|
|||
| `token_budget` | integer | no | Overrides the model's context limit. Used by callers that need to fit a tighter budget. |
|
||||
| `config` | object | no | Compression options, below. A non-object value is ignored rather than rejected. |
|
||||
|
||||
<Callout type="warning" title="system and tools are ignored">
|
||||
Only the four fields above are read. Anthropic sends `system` and `tools` **out of band**, alongside `messages` — this endpoint accepts them without complaint (you get a `200`, no warning) and returns neither, so neither is compressed.
|
||||
|
||||
Keep carrying both yourself and send them upstream unchanged. Two consequences worth knowing:
|
||||
|
||||
- An Anthropic system prompt is not compressed here, even though it is resent on every request.
|
||||
- Tool-schema compaction and tool-search deferral are not reachable through this endpoint — on tool-heavy traffic those can be the largest share of available savings. Run Headroom as the proxy (rather than calling `/v1/compress`) if you need them.
|
||||
</Callout>
|
||||
|
||||
`config` fields:
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|
|
|
|||
|
|
@ -212,6 +212,39 @@ def _compress(client, messages: list[dict], frozen: int | None = None) -> list[d
|
|||
return response.json()["messages"]
|
||||
|
||||
|
||||
def test_system_and_tools_are_accepted_and_ignored(client):
|
||||
"""Documented contract: only messages/model/token_budget/config are read.
|
||||
|
||||
Anthropic sends `system` and `tools` out of band. The endpoint takes them
|
||||
without complaint and returns neither, so neither is compressed — callers must
|
||||
keep carrying them. Pinned because the silence is the hazard: a caller has no
|
||||
signal that the fields did nothing. If this ever starts returning them, the
|
||||
contract changed and docs/content/docs/proxy.mdx needs updating with it.
|
||||
"""
|
||||
response = client.post(
|
||||
"/v1/compress",
|
||||
json={
|
||||
"messages": _anthropic_messages(),
|
||||
"model": "claude-sonnet-4-6",
|
||||
"system": "You are a coding agent. " * 200,
|
||||
"tools": [
|
||||
{
|
||||
"name": "read",
|
||||
"description": "Read a file from disk. " * 20,
|
||||
"input_schema": {"type": "object", "properties": {"path": {"type": "string"}}},
|
||||
}
|
||||
],
|
||||
},
|
||||
)
|
||||
|
||||
assert response.status_code == 200, response.text
|
||||
body = response.json()
|
||||
assert "system" not in body
|
||||
assert "tools" not in body
|
||||
# Messages are still compressed normally alongside the ignored fields.
|
||||
assert body["tokens_saved"] > 0
|
||||
|
||||
|
||||
def test_documented_loop_keeps_the_cached_prefix_byte_identical(client):
|
||||
"""Feed back previous OUTPUT + frozen_message_count -> stable prefix every turn."""
|
||||
import json
|
||||
|
|
|
|||
|
|
@ -274,6 +274,8 @@ Compression-only endpoint. Compresses messages without ever making a **completio
|
|||
|
||||
**No format conversion.** `messages` may be OpenAI-shaped (`role: "tool"` + `tool_call_id`) or Anthropic-shaped (`tool_use` / `tool_result` content blocks); the same shape comes back. `model` selects the tokenizer and context limit — send the real name, including gateway-prefixed forms like `bedrock/anthropic.claude-3-5-sonnet`.
|
||||
|
||||
**`system` and `tools` are ignored.** Anthropic sends both out of band. This endpoint accepts them without complaint (200, no warning) and returns neither, so neither is compressed — keep carrying them yourself. That means the Anthropic system prompt is not compressed here, and tool-schema compaction / tool-search deferral are not reachable through this route; run Headroom as the proxy if you need those.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue