mesh-llm/docs/plans/gpu-benchmark-cli.md

Plan: add mesh-llm gpus detect

Goal

Add a CLI command at mesh-llm gpus detect (originally proposed as mesh-llm gpu benchmark) that forces a fresh benchmark run on the current platform and rewrites ~/.mesh-llm/benchmark-fingerprint.json.

Proposed approach

1. Change the GPU CLI shape

Update the existing GPU command from a bare top-level variant into a real subcommand surface.

• Today, crates/mesh-llm-host-runtime/src/cli/mod.rs defines Command::Gpus as a bare variant with the alias gpu.
• Change that to a subcommand-bearing variant so mesh-llm gpus detect becomes valid.
• Add a new GpuCommand enum for GPU-specific actions.

Expected command shape:

• mesh-llm gpus — keep existing GPU inspection behavior
• mesh-llm gpus detect — force rerun benchmark and rewrite cache

Optional compatibility decision during implementation:

• either preserve bare mesh-llm gpu as an alias for listing GPUs
• or require an explicit listing subcommand such as mesh-llm gpu list

Files to change

crates/mesh-llm-host-runtime/src/cli/mod.rs

• Replace the bare Command::Gpus variant with a subcommand-bearing form.
• Add a GpuCommand enum.
• Keep user-facing help text concise and consistent with nearby commands.

crates/mesh-llm-host-runtime/src/cli/commands/mod.rs

• Change dispatch from direct run_gpus() invocation to a GPU command dispatcher.
• Route gpus detect to a dedicated handler.

crates/mesh-llm-host-runtime/src/cli/commands/gpus.rs

• Keep run_gpus() for the current read-only inspection path.
• Add something like dispatch_gpu_command().
• Add run_gpu_benchmark() (called by detect) to perform the forced benchmark flow and print a short result summary.

crates/mesh-llm-system/src/benchmark.rs

• Add a helper for a forced rerun path.
• Do not reuse run_or_load() unchanged, because it prefers the cache when hardware matches.

Best implementation options:

1. Add a helper like run_and_save(...) that always:
   • detects the benchmark binary
   • runs it
   • builds the result
   • writes benchmark-fingerprint.json
2. Or extend run_or_load(...) with a force flag and bypass:
   • load_fingerprint()
   • hardware_changed()

Preferred direction: extract a dedicated helper rather than overloading run_or_load() too much, so runtime startup and explicit CLI forcing stay easy to reason about.

Desired CLI behavior

mesh-llm gpus detect should:

1. Survey current hardware.
2. Exit cleanly with a clear message if no GPUs are present.
3. Detect the correct platform-specific benchmark binary.
4. Run the benchmark with the existing timeout behavior.
5. Atomically rewrite ~/.mesh-llm/benchmark-fingerprint.json.
6. Print a short success summary including:
   • GPU count
   • total measured bandwidth
   • fingerprint cache path

Important constraints

• Keep mesh-llm gpus read-only.
• Do not silently reuse the cache for gpus detect.
• Reuse existing benchmark binary discovery and parsing logic.
• Preserve current atomic write behavior using the temp-file-plus-rename path.
• Surface soft failures clearly to the user.

Edge cases to handle

No GPU present

• Current runtime and benchmark code already short-circuit when gpu_count == 0.
• The new CLI should print a clear user-facing message and avoid writing a new fingerprint file.

Missing benchmark binary

• detect_benchmark_binary() can return None for unsupported or missing platform binaries.
• The CLI should report that explicitly instead of failing silently.

Benchmark timeout

• Reuse existing timeout behavior from run_benchmark().
• If the benchmark times out or exits unsuccessfully, do not claim success and do not present stale results as fresh.

Parse failures or invalid output

• If benchmark JSON is empty, malformed, or reports an error object, surface that as a benchmark failure.

Existing cache present

• The command should overwrite the existing fingerprint file with a newly generated one.
• The force path must bypass normal cache reuse.

Validation plan

Unit tests

Add or extend tests in crates/mesh-llm-system/src/benchmark.rs to cover:

• forced rerun path bypasses cache reuse
• forced path rewrites the fingerprint file
• no-GPU path does not write a new cache
• missing-binary path fails cleanly

CLI verification

Verify manually or with command-level tests that:

• mesh-llm gpus still shows current GPU inspection output
• mesh-llm gpus detect is accepted by clap
• mesh-llm gpus detect rewrites the fingerprint file even when one already exists
• error cases produce clear output

Regression checks

• Ensure startup benchmarking still uses the normal cached path.
• Ensure existing gpus output formatting remains unchanged unless intentionally improved.

Summary

This should be implemented as a small CLI expansion plus a focused benchmark helper in system/benchmark.rs. The key design requirement is that mesh-llm gpus detect must bypass cache reuse and always regenerate and rewrite benchmark-fingerprint.json, while the existing mesh-llm gpus path remains a read-only inspector of cached data.