mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tejas Chopra
|
8884d87378
|
fix(transforms): stop compression garbling mixed subagent output (#3286)
## The report
A user's model called compressed subagent output "too garbled to use"
and burned CCR retrievals to reconstruct it — **not** because it needed
more context. One retrieval returned nothing but the Claude Code harness
sanitizer banner, reported as `original_item_count: 33,
compressed_item_count: 25`.
Root-cause chain (verified by reproduction): the harness prepends a
bracket-delimited banner (`[harness: ... you.]` — exactly 33
whitespace-delimited words) and neutralizes `<` → `<\`. Headroom's
mixed-content splitter typed the banner as JSON (bracket balance, no
validation) → SmartCrusher couldn't parse it → the fallback chain fed it
to lossy Kompress → Kompress word-dropped the banner 33→25 and stored it
behind a retrieval hash. Meanwhile tabular sections rendered as
quote-wrapped JSON-string blobs with `\n` as two-character escapes, and
`ensure_ascii=True` boundaries turned the output's unicode (`→ └ ✓`)
into `\uXXXX` soup. The model reasonably concluded the output was
garbled.
## Fixes
1. **`split_into_sections` validates JSON before typing a block
`JSON_ARRAY`** — same validation its own mixed-content gate
(`_has_valid_json_block_with_text`) has always used. Tag-protection
placeholders, which self-isolated only by accident of that bug
(`{{HEADROOM_TAG_N}}` bracket-balances), are now isolated explicitly via
a new `isolate=` parameter fed by the router; contiguous prose fragments
re-coalesce so the `\n\n` reassembly stops doubling newlines in
uncompressed prose.
2. **Kompress gets a real floor: `min_input_words = 64`**
(config-tunable, clamped at the historical 10), applied on the
in-process, batch, apply, and remote paths. Below it, lossy
word-dropping is a net loss — the retrieval marker alone is ~20 words —
and short blocks are disproportionately instruction-like.
3. **The mixed path unwraps SmartCrusher's whole-array CSV render** when
it comes back as a bare JSON string, splicing raw readable lines into
the text instead of a quoted escape blob.
4. **`ensure_ascii=False` at model-visible boundaries**: MCP
retrieve/stats responses and the audit-safe splice reserialization
(which now also matches serde_json's non-escaping behavior).
5. **Kompress honesty**: the marker says `N words compressed to M`
(shared `ccr_retrieval_marker` helper, unit-tested), and
`store_kompress_in_ccr` no longer writes word counts into the store's
*item count* fields — token counts already carry the size story.
The upstream trigger (the harness's `<` → `<\` neutralization corrupting
JSON semantics) is not Headroom's to fix, but with #1 and #2 the banner
now passes through byte-intact and nothing lossy touches it.
## Testing
- New `tests/test_garbled_compression_fixes.py` (12 tests) pins every
fix, including an end-to-end router pass over a reconstructed
harness-sanitized fixture asserting the banner survives byte-identical
and no `\uXXXX` appears.
- Existing small-fixture kompress/router tests updated to set
`min_input_words=10` explicitly (they test other mechanics; fixtures sit
under the new production floor by design).
- Affected sweep (`-k "compress or ccr or crusher or router or mixed or
kompress or hermes"`, ~2.9k tests): green apart from order-dependent
flakes that shift identity between runs (deepseek tokenizer `AutoConfig`
import, hermes/proxy-ccr) — each passes standalone and in direct
combination with the new tests; the full CI shards are the authoritative
check.
- ruff 0.16.3 `check` + `format --check` clean.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01EWKCmcH47hvvoQ35wftXhE
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Rod Boev
|
15ac650d40
|
fix(proxy): fail open when kompress saturation would exhaust pre-upstream budget (#1430)
## Description
Concurrent Anthropic `/v1/messages` traffic can still exhaust Headroom's
pre-upstream budget because Kompress ONNX execution waits on the request
critical path. When Kompress saturates, requests eventually fail with
`503 pre-upstream queue saturated` even though compression can safely
degrade to passthrough.
This PR makes Kompress saturation fail open on the Anthropic hot path,
so requests continue uncompressed when compression capacity is under
pressure. It keeps the executor and stage-timing evidence intact, and it
preserves blocking model-load validation so runtime pressure does not
silently skip the validation path.
Closes #1025
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- add a bounded execution-slot acquire path so Anthropic requests fail
open to passthrough when Kompress saturation would consume the
pre-upstream budget
- preserve explicit execution-timeout counters and Anthropic
passthrough/stage-timing observability instead of hiding the pressure
path
- keep `_validate_pytorch_device()` on blocking acquire semantics so
model-load validation still waits for capacity instead of failing open
- make the blocking validation acquire explicit to `mypy` without
changing runtime behavior
- extend focused regressions for pre-upstream backpressure, Kompress
saturation, execution-skip observability, and validation waiting
- align the CLI timeout help text and `ProxyConfig` comment with the
fail-open runtime behavior
- update `CHANGELOG.md` for the proxy runtime fix
## Testing
- [x] Unit tests pass (`uv run pytest
tests/test_anthropic_pre_upstream_backpressure.py
tests/test_proxy_compression_executor.py
tests/test_kompress_request_nonblocking.py`)
- [x] Linting passes (`uv run ruff check
tests/test_anthropic_pre_upstream_backpressure.py` and `uv run ruff
format tests/test_anthropic_pre_upstream_backpressure.py --check`)
- [x] Type checking passes (`uv run mypy headroom
--ignore-missing-imports`)
- [x] New tests added for new functionality when applicable
- [ ] Manual testing performed
### Test Output
```text
Focused local validation passed:
- uv run pytest tests/test_anthropic_pre_upstream_backpressure.py tests/test_proxy_compression_executor.py tests/test_kompress_request_nonblocking.py -x -v
37 passed, 1 warning in 12.01s
- uv run ruff check headroom/proxy/handlers/anthropic.py headroom/transforms/kompress_compressor.py tests/test_anthropic_pre_upstream_backpressure.py tests/test_proxy_compression_executor.py tests/test_kompress_request_nonblocking.py
All checks passed!
- uv run ruff format headroom/proxy/handlers/anthropic.py headroom/transforms/kompress_compressor.py tests/test_anthropic_pre_upstream_backpressure.py tests/test_proxy_compression_executor.py tests/test_kompress_request_nonblocking.py --check
5 files already formatted
- uv run mypy headroom --ignore-missing-imports
Success: no issues found in 398 source files
Base-branch proof on origin/main (
|
||
|
xspawnnn
|
3fc2a78a5e
|
fix(kompress): never block the request path on the cold-cache model download (#1161)
Closes #1146.
## Problem
On a cold cache, the first request that reaches the Kompress deep
compressor triggers an inline `hf_hub_download` of the 274 MB
`chopratejas/kompress-v2-base` ONNX model **on the request thread**.
That download races the proxy's compression budget
(`HEADROOM_COMPRESSION_TIMEOUT_SECONDS`, default 30s — the
`compression_first_stage` timeout): the fetch is cancelled mid-transfer,
**nothing finalizes in the HF cache**, and the request fails open
(uncompressed). Because the partial blob never lands, every subsequent
request repeats the same ~30s hang + fail-open, so the deep compressor
never actually becomes available through the proxy.
This is a **distinct root cause from #946** (which concerns the timeout
itself). Here the model must simply never be fetched synchronously on a
latency-sensitive request.
## Fix
Make the request path cache-only and move the one-time download
off-thread.
**`kompress_compressor.py`**
- `compress(..., allow_download=False)` — new keyword (default `True`,
so the direct API and `compress_batch` are unchanged) that resolves the
model cache-only; on a cold cache it raises `KompressModelNotCached` and
passes through instead of blocking on the network.
- `is_ready()` — lockless cache-membership check, safe to call on the
hot path.
- `ensure_background_download(model_id, device)` — starts at most one
daemon thread per model to pull the artifact down out of band (a
finished/failed thread is replaced, so a transient failure can be
retried by a later request). The compression timeout does not bound this
thread.
**`content_router.py`** — gate the deep path on readiness:
- not ready → return passthrough immediately and kick off the background
download;
- ready → `compress(allow_download=False)` (cache-only, no network on
the request thread).
Net effect: the cold-cache deep path returns in ~0 ms (passthrough)
instead of hanging ~30 s; the model downloads once in the background;
subsequent requests transparently use the deep compressor once it is
cached.
## Verification
Clean install of `headroom-ai==0.26.0` (main `@
|