mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-10 14:27:00 -04:00
7e86bafb90
## Description The #956 symptom — `compression_first_stage` always timing out at ~30s with 0 tokens removed — is not a PyO3/event-loop issue (compression itself runs ~120ms on Python 3.14). Root cause: `tiktoken` downloads its BPE vocab via `requests.get(...)` with no timeout, loaded lazily inside the compression worker (`TiktokenCounter.encoding`, `AnthropicProvider.__init__`). On a firewalled network that blocks indefinitely, so the worker hangs and `asyncio.wait_for` trips at 30s on every request (the hung download never caches, so it repeats). Refs #956 (runtime half). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Load the encoding on a worker thread bounded by `HEADROOM_TIKTOKEN_LOAD_TIMEOUT_SECONDS` (default 10s); on timeout raise `TiktokenLoadError` and fall back to estimation (registry -> EstimatingTokenCounter; Anthropic provider -> character estimate). - Remember the first timed-out encoding so later requests fail fast instead of re-blocking. ## Testing - [x] Unit tests pass (`pytest`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_tokenizers/test_tiktoken_load_timeout.py -q 4 passed in 0.69s ``` ## Real Behavior Proof - Environment: Linux, Python 3.14 and 3.13 - Exact command / steps: timed the Rust compression call sync / via run_in_executor / 2x concurrent on both interpreters; ran the bounded-loader tests against a simulated stalled get_encoding - Observed result: compression ~118-120ms on both 3.14 and 3.13 (no event-loop block); the bounded loader raises/falls back within the timeout instead of hanging - Not tested: the real firewalled-network stall (could not reproduce on an unfirewalled host); the no-timeout requests.get is confirmed in tiktoken's source ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| test_tiktoken_load_timeout.py | ||