PR #431 (merged) added text-block compression to support DeepSeek + Cline,
but the gate ("skip user/system") leaves assistant text blocks compressible
by default. Assistant content is echoed back by the client in subsequent
turns and becomes part of the upstream provider's prefix cache (Anthropic
explicit cache_control, DeepSeek/OpenAI auto-prefix). Compressing it
silently changes the bytes the next turn must match for a cache hit —
turning a 90% read discount into a 25% write penalty on Anthropic, or a
full prefill on DeepSeek/OpenAI when the in-process result cache evicts
or differs across restarts.
Re-aligns the design around prefix-cache safety:
* Block-level cache_control protection (defense in depth). Any block
carrying cache_control is the client's explicit cache breakpoint;
never modified, regardless of role or block type. Closes the gap
that frozen_message_count alone leaves — that count is a coarse
message-level approximation; this is the per-block guarantee.
Applies to both tool_result and text paths.
* compress_assistant_text_blocks defaults to False (off). Assistant
text blocks are skipped by default, restoring pre-#431 cache safety
for Anthropic flows. Per-request opt-in via kwargs (or via
ContentRouterConfig.compress_assistant_text_blocks for deployment-
wide enable) preserves the Cline + DeepSeek goal — only enable
when the backend doesn't honor cache_control AND compression is
deterministic enough that the auto-prefix cache still hits across
eviction/restart.
* Unknown roles default-skip too (was: compressed). developer/judge/
custom roles are safer to leave untouched than to compress
aggressively without thinking through their cache semantics.
* Online streaming usage parser. Replaces the per-stream
list[bytes] buffer with a single last_completion_tokens int updated
per chunk via a module-level _parse_completion_tokens_from_sse_chunk
helper. Streaming memory is now O(1) regardless of stream length —
important for 200K-output reasoning models and DeepSeek V4 Pro's
384K max output.
* Renames the unused min_tokens parameter to min_chars (the threshold
has always been chars, not tokens, in both the tool_result and text
paths). Now also wired through ContentRouterConfig
.min_chars_for_block_compression so the threshold is configurable
per Realignment build constraints.
Tests:
* 17 new tests in tests/test_transforms_content_router.py covering
the role matrix (user / system / assistant / tool / unknown),
cache_control protection on both paths, opt-in semantics, the
min_chars threshold, and idempotent pinning detection.
* 9 new tests in tests/test_streaming_usage_parser.py covering the
online parser's success and edge cases (usage frame, [DONE],
invalid JSON, multi-frame chunks, zero tokens, non-dict payloads,
invalid UTF-8).
Trade-off: deployments pointed at non-cache-aware backends (DeepSeek
direct, OpenAI direct) lose blanket assistant-text compression by
default — they opt in via config. Anthropic flows go back to being
prefix-cache-safe out of the box.
