legion-ref/mesh-llm
1
0
Fork 0
mirror of https://github.com/Mesh-LLM/mesh-llm.git synced 2026-08-08 22:23:19 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

skippy: raise return-sink ready timeout 5s->20s for cold WAN bridge setup

Browse source 
The direct-prediction-return sink's ready handshake used a 5s read timeout, but
over a WAN mesh the remote ready byte only arrives after the bridge cold-
establishes a fresh stage QUIC connection (~10s budget) and the remote inbound
handler dials its local binary server. 5s timed out during that cold setup
(observed EAGAIN on a healthy ~26ms split) even though pooled forward lanes —
which get a 20s initial connect budget and are then reused — succeeded on the
same bridge. Match the forward-lane budget so the cold return path completes
instead of falling back to the slower upstream-reply path.
This commit is contained in:
Michael Neale 2026-07-21 09:45:51 +10:00
parent c340f74135
commit 46108cfc1b
1 changed files with 13 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

16
crates/skippy-server/src/binary_transport/direct_return.rs
View file

@ -302,14 +302,24 @@ impl PredictionReturnSinks {
/// Read timeout for the return-sink ready handshake. `recv_ready` is a blocking
/// `read_exact`; without this a stalled downstream connection hangs the open
/// forever, which mid-generation blocks the request from ever falling back to
/// the upstream reply. Kept short so a genuinely stalled peer fails fast to the
/// fallback; cleared afterwards so the sink's normal reads stay blocking.
/// the upstream reply. Cleared afterwards so the sink's normal reads stay
/// blocking.
///
/// Budget sizing (20s): over a WAN mesh the return sink connects to a LOCAL
/// bridge alias, but the remote `ready` byte only arrives after the bridge
/// COLD-establishes a fresh stage QUIC connection (up to ~10s) and the remote
/// inbound handler then dials its local binary server. A 5s budget timed out
/// during that cold setup (observed EAGAIN on a healthy ~26ms WAN split), even
/// though the pooled forward lanes — which get a 20s initial connect budget and
/// are then reused — succeeded on the same bridge. Matching the forward-lane
/// budget lets the cold return path complete instead of failing to the slower
/// upstream-reply fallback.
///
/// This is a *single bounded deadline*, not a retry budget: the sink is opened
/// on the generation hot path, and `connect_downstream_socket` already bounds
/// the connect itself, so wrapping this in an outer retry only compounds the
/// worst-case stall (see PR #1011 review).
const RETURN_SINK_READY_READ_TIMEOUT: Duration = Duration::from_secs(5);
const RETURN_SINK_READY_READ_TIMEOUT: Duration = Duration::from_secs(20);
/// Connect to `return_addr`, complete the ready handshake, and send the
/// prediction-return open message. Single bounded attempt — on failure the