From 609698b2bac2ef5e2785c19bdc0604642a5be035 Mon Sep 17 00:00:00 2001 From: Adryan Eka Vandra Date: Sat, 18 Apr 2026 02:16:16 +0700 Subject: [PATCH] docs: document codex-proxy-resilience changes in CHANGELOG and wiki MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - wiki/cli.md: add --anthropic-pre-upstream-concurrency option row and HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY env-var note. - CHANGELOG.md: under Unreleased add Added/Fixed/Internal entries for the codex-proxy resilience work — stage timings, shared warmup, WS session registry, pre-upstream semaphore, loopback debug endpoints, repro harness, the fixes (Event.wait leak, py3.10 compat, proxy_headers, first-frame timeout, sem leak, gauge drift), and the internal refactors (IPv6 loopback, lock-free accumulators, narrow suppress, jitter helper). --- CHANGELOG.md | 21 +++++++++++++++++++++ wiki/cli.md | 3 ++- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 33b09f290..05c9e288b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -45,6 +45,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **Codex-proxy resilience hardening** — reduces event-loop starvation under cold-start reconnect storms + - **Stage-timing instrumentation** — per-stage durations for both Codex WS accept and Anthropic `/v1/messages` pre-upstream phases emitted as a single `STAGE_TIMINGS` structured log line per request plus Prometheus histograms + - **Per-pipeline shared warmup** — Anthropic + OpenAI pipelines eagerly load compressors/parsers once at startup; status merged into `WarmupRegistry` for `/debug/warmup` and `/readyz` + - **WS session registry** — first-class tracking of active Codex WS sessions with deterministic relay-task cancellation and termination-cause classification (`client_disconnect`, `upstream_error`, `client_timeout`, etc.) + - **Bounded pre-upstream Anthropic concurrency** — `--anthropic-pre-upstream-concurrency` / `HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY` caps simultaneous `/v1/messages` pre-upstream work (body read, deep copy, first compression stage, memory-context lookup, upstream connect) so replay storms cannot starve `/livez`, `/readyz`, and new Codex WS opens. Default: auto `max(2, min(8, cpu_count))`; `0` or negative disables (unbounded) + - **Loopback-only debug endpoints** — `/debug/tasks`, `/debug/ws-sessions`, `/debug/warmup` return `404` (not `403`) to non-loopback callers so external scanners cannot enumerate them + - **Reconnect-storm repro harness** — `scripts/repro_codex_replay.py` drives concurrent WS + HTTP replay traffic against a local proxy and asserts `/livez` p99 under threshold; `--json` output routes JSON to stdout and the human summary to stderr - **Proxy liveness and readiness health checks** - Adds `GET /livez` for process liveness and `GET /readyz` for traffic readiness - Keeps `GET /health` backward compatible while expanding it with readiness details and subsystem checks @@ -123,6 +130,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Graceful fallback for unknown models (no crashes) - Updated pricing data for all current models +### Fixed +- **Event.wait task leak in subscription trackers** — `asyncio.shield` pattern prevents cancellation of the outer `wait_for` from leaking the inner `Event.wait` task +- **Python 3.10 compatibility for memory-context fail-open** — catches `asyncio.TimeoutError` (the 3.10-compatible alias) rather than `TimeoutError` to preserve behaviour on older runtimes +- **uvicorn `proxy_headers=False`** — refuses `Forwarded` / `X-Forwarded-For` rewrites so the loopback guard on `/debug/*` cannot be spoofed by a misconfigured reverse proxy +- **First-frame timeout for Codex WS accepts** — guards against a client that opens a handshake and never sends the first frame; relays cancel deterministically with `client_timeout` +- **Semaphore leak on unexpected exception in Anthropic pre-upstream path** — the finalizer now releases the pre-upstream semaphore on every exit path (early 4xx, cache hit, upstream error, streaming handoff) +- **`active_relay_tasks` gauge double-decrement** — `deregister_and_count` returns `(handle, released_task_count)` atomically so the handler decrements the Prometheus gauge by the exact number it registered, eliminating drift + +### Internal +- **IPv6-mapped loopback recognition** — the loopback guard parses `::ffff:127.0.0.1` and other dual-stack literals through `ipaddress.ip_address(...).is_loopback` +- **Lock-free stage-timing accumulators** — `record_stage_timings` writes to per-path counters that do not contend with `/metrics` export or `record_request` +- **Narrow `contextlib.suppress` in relay classification** — only `CancelledError` is suppressed where we reclassify it; other exceptions propagate so termination cause stays truthful +- **`jitter_delay_ms` helper** — shared exponential-backoff + 50-150% jitter formula in `headroom/proxy/helpers.py`; used by three proxy retry sites and mirrored inline in the repro harness + ## [0.2.0] - 2025-01-07 ### Added diff --git a/wiki/cli.md b/wiki/cli.md index 9414b713b..8337fd005 100644 --- a/wiki/cli.md +++ b/wiki/cli.md @@ -246,6 +246,7 @@ headroom proxy --mode cache | `--no-rate-limit` | off | Disable rate limiting | | `--retry-max-attempts` | runtime default `3` | Maximum upstream retry attempts | | `--connect-timeout-seconds` | runtime default `10` | Upstream connection timeout | +| `--anthropic-pre-upstream-concurrency` | auto `max(2, min(8, cpu_count))` | Cap simultaneous pre-upstream work on `/v1/messages` (body read, deep copy, first compression stage, memory-context lookup, upstream connect). `0` or negative disables (unbounded); any positive integer is honoured verbatim. Prevents cold-start replay storms from starving `/livez`, `/readyz`, and new Codex WS opens. | | `--log-file` | unset | JSONL log output path | | `--budget` | unset | Daily USD budget limit | | `--no-code-aware` | off | Disable AST-aware code compression | @@ -273,7 +274,7 @@ headroom proxy --mode cache Notes: - `--learn` implies memory unless `--no-learn` is also set. -- Proxy startup can also read environment variables such as `HEADROOM_HOST`, `HEADROOM_PORT`, `HEADROOM_BUDGET`, `HEADROOM_MODE`, `HEADROOM_ANYLLM_PROVIDER`, `ANTHROPIC_TARGET_API_URL`, `OPENAI_TARGET_API_URL`, and `GEMINI_TARGET_API_URL`. +- Proxy startup can also read environment variables such as `HEADROOM_HOST`, `HEADROOM_PORT`, `HEADROOM_BUDGET`, `HEADROOM_MODE`, `HEADROOM_ANYLLM_PROVIDER`, `HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY`, `ANTHROPIC_TARGET_API_URL`, `OPENAI_TARGET_API_URL`, and `GEMINI_TARGET_API_URL`. CLI flags take precedence over environment variables. See also: [Proxy Server](proxy.md), [Configuration](configuration.md)