mirror of
https://github.com/Mesh-LLM/mesh-llm.git
synced 2026-08-08 22:23:19 -04:00
2c5dacf212
* Replace Skippy verify span with verify windows * Add verify window reply metadata * Pipeline direct-return n-gram verify windows * Pipeline MTP-anchored n-gram verify windows * Support static release builds without features * Fix split MTP activation-frame serving * Replace native MTP batched verifier with verify windows * Restore native MTP verify window batching * Replace native MTP anchor extension with composite proposals * Keep composite decode branch on development version * Expose decode timings for all generation modes * Retry transient staged lane readiness * Bound persistent lane readiness handshake * Keep pure N-gram decode free of MTP drafts * Report composite proposal totals in decode timings * Gate composite decode pipeline by candidate depth * Account direct GGUF MTP weights in split planning * Avoid MTP cooldown after N-gram tail rejection * Improve hybrid MTP verification telemetry * Pipeline native MTP verification replies * Require useful N-gram tails for hybrid MTP * Adapt N-gram MTP extensions to tail acceptance * Fix direct GGUF planning fallback * Gate N-gram tails on MTP prefix agreement * Widen initial async verify windows * Restore anchored N-gram MTP extensions * Retain ready stages across transient refresh failures * Document pipelined VerifyWindow decode * Use llama.cpp N-gram proposer for Skippy * Add cache-based N-gram proposer * Add declarative speculative proposer package schema * Productize Skippy speculative decode plans * Productize Skippy speculative decode plans * Support direct N-gram Skippy plans * Validate speculative package strategy plans * Add coding agent loop benchmark corpus * Expose Skippy speculative benchmark counters * Validate cache N-gram proposer limits * Document speculative decode configuration * Fix native MTP proposals and fused restore routing * Honor configured N-gram extension width * Document speculative runtime overrides * Refresh speculative config schema contracts * Keep N-gram tail rejects from penalizing MTP * Preserve MTP state after serial tail rejects * Report adaptive verify width changes accurately * Fix short simple N-gram extension budgets * Make VerifyWindow pipelining cost-aware * Profile prospective VerifyWindow widths * docs: WAN split performance model + measured latency/compute decomposition Adds docs/skippy/WAN_SPLIT_PERF.md: the single-stream per-token cost model (TPOT ~= C_total + (S-1)*2*RTT + (S-1)*P), compute-bound vs latency-bound criteria, when adding a stage helps (memory, concurrency/pipeline overlap, dense compute-bound models), and speculation as the WAN amortization lever. Backed by 2026-07-18 Sydney<->Melbourne 2-node measurements: solo 12.9 ms/tok compute, split 57.8 ms/tok, decomposing to 12.9 compute + 40 (2xRTT) + 4.9 protocol. Workload was latency-bound (~78% network). * docs: plan for fast-fail on new requests routed to a dead split stage Documents the measured ~30s hang when a new request routes to a killed split stage, the confirmed root cause (60s heartbeat / lenient failure threshold + slow lane-open timeouts), and a two-layer fix (short steady-state lane-open deadline; feed lane failures into target_health cooldown) plus an explicit validation gate. Mesh-timing changes are out of scope pending live multi-node validation. * Fast-fail lane reconnects so new requests don't hang on a dead split stage When a downstream split stage dies, a new request would open a fresh lane and wait the full ~20s warmup ready-deadline before erroring (observed as a ~30s hang in the Sydney<->Melbourne kill test). The 20s deadline is only needed during pool warmup, when the downstream may still be loading its model. Split the deadline: pool warmup keeps LANE_READY_READ_TIMEOUT (20s); mid-life reconnects from checkout()/replace_lane() on an already-serving mesh use a new LANE_STEADY_CONNECT_TIMEOUT (3s). A healthy peer answers in milliseconds, so a dead stage now fails in ~3s instead of ~30s. Restores receive_persistent_lane_ready as the shared bounded-handshake helper (dropped during the main merge) and removes a now-obsolete retry test that covered pre-#1011 retry behavior. Adds tests asserting the steady-state deadline stays well under the warmup deadline and that the handshake read fails fast on a silent downstream. * docs: latency-aware placement — current behaviour and many-node gaps Records verified planner behaviour (skippy-coordinator/topology.rs, skippy-topology, host-runtime call site): - latency is a placement cost (rtt_ms penalty), not just relay-only exclusion - planner selects a node subset; does not have to use every eligible node - stage count is gated on a decode-TPOT target (shallower-that-meets beats deeper-that-does-not) And the gaps that matter at many-node scale: - no peer-to-peer RTT matrix in production (edge_signals never wired; only coordinator-RTT is used) -> co-located nodes cannot be exploited - network estimate is max(coordinator RTT) x node_count, a worst-case proxy - no first-class prefer-fewer/never-place-above-Y policy beyond the TPOT gate * docs: measured speculative recovery cost over WAN (why ngram hurts a latency-bound split) * Discard dead pooled stage lanes before reuse (fast-fail improvement) A pooled downstream lane whose stage died while checked in was a dead TCP stream; reusing it blocked the next generation read forever (handshake read-timeout is cleared for pooled lanes so long generations don't truncate). checkout() now probes lane liveness with a nonblocking peek and discards a dead lane so it reconnects with the short steady-state deadline instead of hanging. Validated on a loopback 2-node split with a mid-flight worker kill: new request now fails faster than main (60s vs main's 90s baseline). Does not fully solve the recovered-local routing path, tracked as follow-up. --------- Co-authored-by: Michael Neale <14976+michaelneale@users.noreply.github.com>
1 line
691 B
JSON
1 line
691 B
JSON
{"id":"coding-agent-loop","category":"coding_agent_loop","session_group":"skippy-proposer-matrix","prompt":"Continue this coding-agent task transcript and choose the next concrete action.\nRepository task: MoneyWidget decompress method breaks form validation with disabled fields. The task asks the agent to inspect /testbed/djmoney/forms/widgets.py, reproduce the issue, edit non-test source files only, and rerun validation. Current transcript shows the agent found /testbed/djmoney/forms/widgets.py and observed that MoneyWidget.decompress immediately returns [None, self.default_currency] before checking whether value is not None. Continue with the next concrete coding-agent action."}
|