* Enable adaptive verify window for ngram/draft speculation
The adaptive verify window was never enabled on the split-serving path:
to_embedded_openai_args hardcoded adaptive_speculative_window = false. With a
fixed window, an early reject never shrank the window, so a sustained reject
storm kept proposing at full depth and paying the full 2-round-trip recovery
cost per token. On a WAN split this measured as ~40% throughput loss with
N-gram speculation ON versus OFF, despite high per-token acceptance.
Enable the adaptive window whenever speculation actually proposes a window
(ngram or draft mode). The existing shrink_adaptive_window logic then narrows
the window toward the observed accept depth after an early reject, cutting
recovery frequency. Adds a regression test asserting ngram speculation turns
the adaptive window on.
* Replace speculative rollback with positional MTP n-gram pipelining
* Pipeline speculative verify windows across latency
* Fix positional correction and adaptive pipeline depth
* Continuously refill the speculative horizon
* Productionize pipelined MTP n-gram speculation
* Fix speculative docs and UI formatting
* Remove stale speculative projections and fix CI
* Handle fragmented direct-return fallback replies
* Replace speculative repair with fixed-depth positional pipeline
* Expose split-stage compute overlap telemetry
* Lock split topology placement
* Document locked split topology
* Address locked topology review feedback
* Fix SPEED-Bench timing JSONL output
* Bound benchmark telemetry finalization
* Hash SPEED-Bench request and response pairs
* mesh: stop re-applying formation-time RTT gate to operational stage streams
Ported from the WAN lab branch (wip/wan-direct-prediction-return, c340f741),
where it was validated live on a ~26ms WAN split. open_stage_transport_stream
re-applied the formation-time MAX_SPLIT_RTT_MS ceiling to every fresh
operational stream, so per-request direct-return sinks were rejected under
normal WAN RTT jitter while pooled forward lanes stayed healthy - surfacing as
ready-handshake timeouts and 502s on an already-admitted split. Split
admission still gates eligibility via gossiped, hysteresis-smoothed RTT plus
re-election; operational streams now warn and proceed.
* skippy: raise return-sink ready timeout 5s->20s for cold WAN bridge setup
Ported from the WAN lab branch (46108cfc). Over a WAN mesh the return sink
connects to a local bridge alias, but the remote ready byte only arrives after
the bridge cold-establishes a fresh stage QUIC connection (~10s budget) and the
remote handler dials its local server. 5s timed out during that cold setup on
a healthy ~26ms split; forward lanes already use a 20s budget. Match it.
* runtime: relaunch withdrawn splits when peers return instead of ending the model task
Observed live on a real WAN split (Sydney M5 <-> AU 4090): one transient
direct-return 502 led periodic_check to mark the remote stage unavailable;
after the 75s grace the coordinator withdrew the topology. The Withdraw event
returned StartupLoopControl::Break, so startup_local_model_loop tore down and
the task ended permanently - while the remote worker sat healthy, logging
'standing by for stage assignment' forever. Only recovery was manually
restarting both nodes with a fresh token.
Make withdraw non-terminal: a new RelaunchSplit control/outcome runs the full
existing teardown, then loops back to the launch phase and re-enters
wait_for_split_participants, relaunching the split when an eligible peer
returns. The stop channel is checked before relaunch so explicit shutdown
still wins. LocalFallback (model fits locally) is unchanged.
The participant-wait loop's 30s cadence and stable-participant gating act as
the natural retry throttle; no extra backoff added.
---------
Co-authored-by: Michael Neale <14976+michaelneale@users.noreply.github.com>
Co-authored-by: Michael Neale <michael.neale@gmail.com>
* Complete plugin web UI host surface
* Align plugin web UI contract and docs
* Harden plugin web UI release path
* Fix plugin navigation and settings UX
* Align local and CI repo consistency gates
* Run publish consistency in test-all
* Simplify native log test module wiring
Validation
* Validation tier: Tier 2R - post-review conflict/base refresh of an existing shared runtime integration PR; the manual conflict scope was README.md, with targeted Flash-MoE/runtime checks rerun on the final rebased diff.
* git fetch --no-tags origin main:refs/remotes/origin/main: PASS
* git rebase origin/main: PASS, resolved conflict in README.md.
* git diff --check origin/main...HEAD: PASS
* git diff --cached --check: PASS
* cargo fmt --all -- --check: PASS
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal rustup run stable cargo test -p mesh-llm-host-runtime flash_moe --lib: PASS, 13 passed, 0 failed
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal rustup run stable cargo check -p mesh-llm: PASS
* Ledger: not applicable - not required for selected validation tier/change family.
* Version: not applicable - not required for selected validation tier/change family.
* Not run: just build - not required for this conflict/base-refresh tier; targeted Rust checks covered the affected runtime paths and GitHub CI will rerun the final PR SHA.
* Not run: full cargo test -p mesh-llm-host-runtime --lib - not required for selected tier; targeted Flash-MoE tests covered the changed plugin/runtime path.
Rollback
* git revert HEAD
* Prepare workspace for skippy integration
* Copy skippy model crates
* Use skippy model crates in resolver
* Add mesh compatibility tests for model crates
