1 commit

Author	SHA1	Message	Date
Gaurav Dubey	5d14080c94	fix(proxy): retry passthrough on transient upstream connection close (#1513) ... ## Description `GET /v1/models` (and other buffered passthrough routes) returned an opaque HTTP **502** when an OpenAI-compatible upstream closed a pooled keep-alive connection mid-response, surfacing `httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)`. The same upstream answers a direct `curl` with 200 because curl opens a fresh connection per call, while Headroom reuses pooled keep-alive connections — so the first request issued on a stale connection fails even though the upstream is healthy. The fix makes the buffered passthrough path retry once on a fresh connection (exactly what curl does), and return a clear error only if the upstream is genuinely sending an incomplete response. Closes #1112 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom.proxy.helpers.request_with_transient_retry(client, *, request_id=None, max_retries=1, **request_kwargs)`: issues a buffered httpx request and retries on a **fresh connection** when (and only when) `httpx.RemoteProtocolError` is raised. Every other exception (`ConnectError`, timeouts, status errors) propagates immediately, so existing handling is unchanged. Documented as buffered-only (a streamed response can't be safely replayed once bytes reach the client). - Route `OpenAIHandlerMixin.handle_passthrough` through the helper, and add an `except httpx.RemoteProtocolError` arm that returns a clear `502` with error type `upstream_protocol_error` when the protocol error persists across the retry (instead of letting the raw error surface as an opaque/unhandled 502). - Add `tests/test_proxy_passthrough_transient_retry.py` (helper unit tests + handler-level tests covering the exact issue path). - Add a `CHANGELOG.md` entry under `Unreleased → Fixed`. Scope note: streaming `/v1/responses` is intentionally **out of scope** for this change — a streamed response cannot be safely retried after the first byte has been delivered to the client. The helper is written reusable so a streaming-aware follow-up can build on it. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/helpers.py headroom/proxy/handlers/openai.py tests/test_proxy_passthrough_transient_retry.py All checks passed! $ mypy headroom/proxy/helpers.py --ignore-missing-imports Success: no issues found in 1 source file $ pytest tests/test_proxy_passthrough_transient_retry.py -q tests/test_proxy_passthrough_transient_retry.py ....... [100%] 7 passed in 0.27s # no regressions in the surrounding passthrough/handler suites: $ pytest tests/test_proxy_passthrough_transient_retry.py tests/test_proxy_handler_helpers.py \ tests/test_proxy_byte_faithful_forwarding.py \ tests/test_proxy/test_compression_failure_action.py tests/test_proxy_copilot_auth_hooks.py -q 80 passed, 1 warning in 6.88s ``` ## Real Behavior Proof Reproduced against a **real local TCP server** (no mocks) that speaks HTTP/1.1 and, when armed, emits a chunked body then closes the socket **without** the terminating `0\r\n\r\n` — the exact condition that makes httpx raise the `incomplete chunked read` error from this issue. - Environment: macOS arm64, Python 3.12, httpx 0.28.1 (same httpx major as the report), real loopback sockets via `asyncio.start_server`. - Exact command / steps: start the local server; (1) issue a single buffered request — the pre-fix `handle_passthrough` behaviour; (2) issue the same request through `request_with_transient_retry` — the fix. Verbatim: `python repro_1112.py`. - Observed result: BEFORE the fix a single request raises `httpx.RemoteProtocolError` ("incomplete chunked read") which `handle_passthrough` surfaced as an opaque HTTP 502; AFTER the fix the same request returns **HTTP 200** (the retry opened a fresh connection, mirroring a direct `curl`). Full terminal output: ```text upstream listening on http://127.0.0.1:62374/v1/models BEFORE (single buffered request, pre-fix behaviour): raised httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) -> handle_passthrough surfaced this as an opaque HTTP 502 AFTER (request_with_transient_retry, the fix): HTTP 200 body={"object":"list","data":[]} -> first attempt hit the incomplete chunked read, retry on a fresh connection returned 200 (mirrors a direct curl) ``` The log line `Upstream closed connection mid-response (...incomplete chunked read); retrying on a fresh connection (attempt 1/1)` fires on the recovered request, confirming the retry path is what produced the 200. - Not tested: real third-party upstreams (LiteLLM/vLLM/etc.) — the local server reproduces the precise httpx error deterministically; the streaming `/v1/responses` path is intentionally out of scope (a streamed response cannot be safely retried after the first byte reaches the client). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - No new dependencies (httpx is already a proxy dependency), so no supply-chain justification is required. - The retry is deliberately narrow: only `httpx.RemoteProtocolError` is retried, capped at one retry, so a genuinely-down upstream still fails fast via the existing `ConnectError`/timeout path. - "Documentation" checklist item refers to the `CHANGELOG.md` entry; no user-facing docs pages needed for this internal resilience fix.	2026-07-06 18:35:39 -05:00