mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
74392b238e
## Summary Replaces `chopratejas/kompress-base` with **`chopratejas/kompress-v2-base`** as the default Kompress text-compression model (the fallback for content not handled by structured compressors), using a new **weight-only int8 ONNX** artifact that is fp32-equivalent at 2.2x less memory. ## Why v2 is the same dual-head ModernBERT (token classifier + span CNN), LoRA-trained on the v2 dataset. The HF repo originally shipped PyTorch weights only — pointing Headroom at it naively would have forced the heavy `[ml]` (torch) path on every `[proxy]` install. We exported ONNX artifacts reproducing the v1 loader contract (single `final_scores` output) and published them to the HF repo. ## Eval (labeled dataset_v2 test split, n=500, threshold 0.5) | artifact | size | f1 | must_keep_recall | keep_rate | fp32 agreement | |---|---|---|---|---|---| | fp32 (reference) | 601 MB | 0.9128 | 0.9770 | 0.8100 | 100% | | **int8-wo (default)** | **261 MB** | **0.9130** | **0.9765** | **0.8097** | **99.6%** | | fp16 | 301 MB | 0.9129 | 0.9771 | 0.8099 | 99.9% | | int4-wo | 230 MB | 0.9102 | 0.9825 | 0.8354 ⚠️ | 94.5% | | int8-dynamic | 271 MB | 0.9041 | 0.9868 | 0.8857 ⚠️ | 90.5% | Weight-only int8 (MatMulNBits) keeps activations fp32, avoiding the upward score bias that makes dynamic int8 keep ~7% more tokens (≈40% less compression savings). Quantized candidates were generated and eval-gated by a Modal job in the kompress repo (`modal_jobs/export_onnx_v2.py`) against the labeled test split. ## Changes - Default model id → `chopratejas/kompress-v2-base` - ONNX artifact resolution tries candidates in order (**int8-wo → fp32 → v1 int8**), falling through on download miss **or session-load failure** — onnxruntime builds without the MatMulNBits 8-bit kernel fall back to fp32 instead of losing Kompress - `HEADROOM_KOMPRESS_ONNX_FILENAME` env pins an exact artifact - `scripts/export_kompress_v2_onnx.py`: reproducible fp32 export (loads the merged v2 checkpoint, traces the `final_scores` contract, verifies vs PyTorch) - `.gitignore`: local `onnx/` artifacts dir; allowlist the export script ## Testing - End-to-end: fresh `KompressCompressor().preload()` resolves int8-wo from HF, loads on onnxruntime 1.23 (CPU, no torch), compresses with error/traceback content preserved - fp32 export verified lossless vs PyTorch (max |Δscore| = 0.0, 100% keep agreement) - ruff check + format clean, mypy clean, 63 targeted tests pass |
||
|---|---|---|
| .. | ||
| fixtures | ||
| tests | ||
| audit_wheel_glibc_symbols.py | ||
| build_rust_extension.sh | ||
| changelog-gen.py | ||
| export_kompress_v2_onnx.py | ||
| install-git-hooks.sh | ||
| install.ps1 | ||
| install.sh | ||
| README.md | ||
| record_fixtures.py | ||
| refresh_model_limits.sh | ||
| replay_codex_ws_load.py | ||
| repro_codex_replay.py | ||
| smoke_issue_327.py | ||
| sync-plugin-versions.py | ||
| validate-workflows.sh | ||
| verify-versions.py | ||
| version-sync.py | ||
scripts/
Utility scripts bundled with the Headroom repo. Most are one-off operator tools; a few are runnable as part of development workflows.
Reproducing the reconnect storm
repro_codex_replay.py reproduces the multi-agent Codex reconnect/retry storm
against a local Headroom proxy (default http://127.0.0.1:8787), as described
in wiki/plans/2026-04-17-codex-proxy-runtime-analysis.md under "Latest
Correction". Use it to:
- Regression-check that
/livezstays responsive under a cold-start storm. - Empirically tune the Unit 4 pre-upstream semaphore default
(
HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY). - Exercise the Codex WS lifecycle + Anthropic HTTP path simultaneously without needing to replay captured production traffic.
Run
# Default: 8 WS + 4 HTTP clients, 30s storm, p99 /livez must stay <= 500ms.
python scripts/repro_codex_replay.py
# Tighter budget, shorter run:
python scripts/repro_codex_replay.py \
--url http://127.0.0.1:8787 \
--ws-clients 16 \
--anthropic-clients 8 \
--duration 60 \
--livez-threshold-ms 100
# Dump the full summary as JSON for downstream tooling:
python scripts/repro_codex_replay.py --json
Exit code:
0— warmup succeeded (or was skipped), storm ran for the requested duration, and/livezp99 stayed under--livez-threshold-ms.1— soft assertion failed, proxy unreachable, or unhandled exception. Proxy-unreachable is detected and reported within ~5 seconds.
Fixtures
The script loads two hand-crafted, fully synthetic JSON fixtures:
scripts/fixtures/anthropic_replay_body.json— shape of a large agent reconnect replay/v1/messages?beta=truePOST body.scripts/fixtures/codex_response_create_frame.json— first Codex WS frame with the{"type": "response.create", "response": {...}}envelope.
Override via --ws-frame-fixture / --anthropic-body-fixture if you have
captured traffic to replay instead.
Interpretation
/livez p99under threshold means the event loop is not starved during the storm. If it rises with the semaphore unbounded (HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY=10000) and drops back under the default, Unit 4's backpressure is working.Codex WS: openedshould equal--ws-clients.response.completedtypically stays low when upstream auth isn't configured locally — the goal is handshake + relay wiring, not real upstream traffic.Anthropic HTTP: ok_2xx + non_2xx + timed_out + errorsshould roughly equalattempted. Sustained non-zerotimed_outduring the storm is the failure signal the plan targets.
A smoke test at tests/test_scripts/test_repro_codex_replay_smoke.py
exercises the script against a mock FastAPI server on every PR.
Install scripts
install.sh— POSIX installer.install.ps1— Windows PowerShell installer.
These are generated by the release pipeline; edit with care.