Add Flash-MoE SSD backend plugin

Validation
* Validation tier: Tier 2R - post-review conflict/base refresh of an existing shared runtime integration PR; the manual conflict scope was README.md, with targeted Flash-MoE/runtime checks rerun on the final rebased diff.
* git fetch --no-tags origin main:refs/remotes/origin/main: PASS
* git rebase origin/main: PASS, resolved conflict in README.md.
* git diff --check origin/main...HEAD: PASS
* git diff --cached --check: PASS
* cargo fmt --all -- --check: PASS
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal rustup run stable cargo test -p mesh-llm-host-runtime flash_moe --lib: PASS, 13 passed, 0 failed
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal rustup run stable cargo check -p mesh-llm: PASS
* Ledger: not applicable - not required for selected validation tier/change family.
* Version: not applicable - not required for selected validation tier/change family.
* Not run: just build - not required for this conflict/base-refresh tier; targeted Rust checks covered the affected runtime paths and GitHub CI will rerun the final PR SHA.
* Not run: full cargo test -p mesh-llm-host-runtime --lib - not required for selected tier; targeted Flash-MoE tests covered the changed plugin/runtime path.

Rollback
* git revert HEAD

README.md

@@ -58,6 +58,7 @@ mesh-llm serve --auto --headless
 | Join by invite token | `mesh-llm serve --join <token>` | [docs/MESHES.md](docs/MESHES.md) |
 | Run an API-only client | `mesh-llm client --auto` | [docs/MESHES.md](docs/MESHES.md) |
 | Run a big model with splits | `mesh-llm serve --model hf://meshllm/<repo>@<rev> --split` | [docs/SKIPPY_SPLITS.md](docs/SKIPPY_SPLITS.md) |
+| Attach a Flash-MoE SSD backend | `mesh-llm serve` with `[[plugin]] name = "flash-moe"` | [docs/plugins/flash-moe.md](docs/plugins/flash-moe.md) |
 | Use Goose, OpenCode, Claude Code, or Pi | `mesh-llm goose`, `mesh-llm opencode`, `mesh-llm claude`, `mesh-llm pi` | [docs/AGENTS.md](docs/AGENTS.md) |
 | Build or contribute | `just build` | [CONTRIBUTING.md](CONTRIBUTING.md) |
 
@@ -132,6 +133,7 @@ plus `glslc`.
 | [docs/CLI.md](docs/CLI.md) | Command reference and JSON automation |
 | [docs/USAGE.md](docs/USAGE.md) | Longer operational usage guide, runtime control, owner-control operator flows |
 | [docs/design/TESTING.md](docs/design/TESTING.md) | Testing playbook, mixed-version QA, remote deploy checks |
+| [docs/plugins/flash-moe.md](docs/plugins/flash-moe.md) | Optional Flash-MoE SSD expert streaming backend setup |
 | [docs/skippy/FAMILY_STATUS.md](docs/skippy/FAMILY_STATUS.md) | Certified Skippy model-family status |
 | [docs/specs/layer-package-repos.md](docs/specs/layer-package-repos.md) | Manifest and artifact format spec |
 

ROADMAP.md

@@ -86,9 +86,9 @@ This is a single-node strategy. The goal is running e.g. Qwen3.5-397B-A17B (~209
 
 Today mesh-llm has two MoE modes: **solo** (model fits in memory, run it whole) and **split** (model doesn't fit, shard experts across nodes). SSD streaming would be a third mode: model doesn't fit in memory but *does* fit on one node's SSD. No mesh coordination, no cross-node traffic, no splitting — just one machine streaming experts from disk.
 
-**Plan:** Use flash-moe directly as an alternative backend, not hack SSD streaming into llama.cpp. llama.cpp's `ggml_mul_mat_id` assumes all expert weights resident in one contiguous tensor — changing that is deep surgery across ggml, the Metal backend, and the model loader. Flash-moe is a working engine. Mesh-llm spawns it like it spawns llama-server — process management + HTTP wrapper.
+**Status:** Initial mesh-llm integration exists as a built-in `flash-moe` plugin adapter. Mesh-llm can spawn a local Flash-MoE `infer --serve` process or attach an already-running OpenAI-compatible `/v1` endpoint, then route to it through the plugin inference path. The adapter intentionally does not vendor Flash-MoE, automate Flash-MoE installation, or prepare SSD-streaming artifacts.
 
-Only supports Qwen3.5-397B for now (hardcoded architecture). That's fine — it's the model we want to run.
+Flash-MoE only supports Qwen3.5-397B for now (hardcoded architecture). That's fine for the target model. Remaining mesh-llm work is install/artifact-prep documentation, real-machine smoke coverage for the target path, and revisiting out-of-tree adapter packaging once the plugin SDK and crates.io surface are stable enough.
 
 ## Blackboard ✅
 

crates/mesh-llm-host-runtime/src/cli/mod.rs

@@ -235,7 +235,7 @@ pub enum LogFormat {
     name = "mesh-llm",
     version = crate::VERSION,
     about = "Pool GPUs over the internet for LLM inference",
-    after_help = "Preferred runtime entrypoints:\n  mesh-llm serve\n  mesh-llm serve --model Qwen3-8B-Q4_K_M\n  mesh-llm client --auto\n  mesh-llm gpus\n\n`mesh-llm serve` loads startup models from ~/.mesh-llm/config.toml.\nRun with --help-advanced for all options.\n\nExternal backends (vLLM, TGI, Ollama):\n  Add to ~/.mesh-llm/config.toml:\n    [[plugin]]\n    name = \"openai-endpoint\"\n    url = \"http://gpu-box:8000/v1\"\n  Then: mesh-llm serve     (or: mesh-llm client  for client-only mode)"
+    after_help = "Preferred runtime entrypoints:\n  mesh-llm serve\n  mesh-llm serve --model Qwen3-8B-Q4_K_M\n  mesh-llm client --auto\n  mesh-llm gpus\n\n`mesh-llm serve` loads startup models from ~/.mesh-llm/config.toml.\nRun with --help-advanced for all options.\n\nExternal backends (vLLM, TGI, Ollama):\n  Add to ~/.mesh-llm/config.toml:\n    [[plugin]]\n    name = \"openai-endpoint\"\n    url = \"http://gpu-box:8000/v1\"\n  Then: mesh-llm serve     (or: mesh-llm client  for client-only mode)\n\nFlash-MoE SSD backend:\n  Add [[plugin]] name = \"flash-moe\" with either command/args or url.\n  Then: mesh-llm serve     (or: mesh-llm client  for client-only mode)"
 )]
 pub(crate) struct Cli {
     #[command(subcommand)]

crates/mesh-llm-host-runtime/src/plugin/config.rs

@@ -1,6 +1,6 @@
 use super::{
-    PluginSummary, BLACKBOARD_PLUGIN_ID, BLOBSTORE_PLUGIN_ID, OPENAI_ENDPOINT_PLUGIN_ID,
-    TELEMETRY_PLUGIN_ID,
+    PluginSummary, BLACKBOARD_PLUGIN_ID, BLOBSTORE_PLUGIN_ID, FLASH_MOE_PLUGIN_ID,
+    OPENAI_ENDPOINT_PLUGIN_ID, TELEMETRY_PLUGIN_ID,
 };
 use anyhow::{bail, Context, Result};
 use mesh_llm_plugin::MeshVisibility;
@@ -9,6 +9,10 @@ use skippy_protocol::FlashAttentionType;
 use std::collections::BTreeMap;
 use std::path::{Path, PathBuf};
 
+const FLASH_MOE_INSTALL_HINT: &str = "Install Flash-MoE separately and set \
+                                     `command` to its infer binary, or set \
+                                     `url` to an already-running Flash-MoE /v1 endpoint.";
+
 #[derive(Clone, Debug, Default, Deserialize, Serialize)]
 pub struct MeshConfig {
     #[serde(default)]
@@ -125,6 +129,8 @@ pub struct ExternalPluginSpec {
     pub args: Vec<String>,
     /// Backend URL for inference endpoint plugins.
     pub url: Option<String>,
+    /// Extra environment passed only to the plugin process.
+    pub env: BTreeMap<String, String>,
 }
 
 #[derive(Clone, Copy, Debug)]
@@ -286,6 +292,7 @@ pub fn resolve_plugins(config: &MeshConfig, _host_mode: PluginHostMode) -> Resul
     let mut blobstore_enabled = true;
     let mut openai_endpoint_enabled = false;
     let mut openai_endpoint_url: Option<String> = None;
+    let mut flash_moe_entry: Option<&PluginConfigEntry> = None;
     let mut telemetry_enabled = true;
     for entry in &config.plugins {
         if names.insert(entry.name.clone(), ()).is_some() {
@@ -325,6 +332,13 @@ pub fn resolve_plugins(config: &MeshConfig, _host_mode: PluginHostMode) -> Resul
             }
             continue;
         }
+        if entry.name == FLASH_MOE_PLUGIN_ID {
+            if !enabled {
+                continue;
+            }
+            flash_moe_entry = Some(entry);
+            continue;
+        }
         if entry.name == TELEMETRY_PLUGIN_ID {
             if entry.command.is_some() || !entry.args.is_empty() || entry.url.is_some() {
                 bail!(
@@ -347,6 +361,7 @@ pub fn resolve_plugins(config: &MeshConfig, _host_mode: PluginHostMode) -> Resul
             command,
             args: entry.args.clone(),
             url: None,
+            env: BTreeMap::new(),
         });
     }
 
@@ -362,6 +377,9 @@ pub fn resolve_plugins(config: &MeshConfig, _host_mode: PluginHostMode) -> Resul
         spec.url = openai_endpoint_url;
         externals.push(spec);
     }
+    if let Some(entry) = flash_moe_entry {
+        externals.push(flash_moe_plugin_spec(entry)?);
+    }
     if blobstore_enabled {
         externals.push(blobstore_plugin_spec()?);
     }
@@ -387,6 +405,7 @@ pub fn blackboard_plugin_spec() -> Result<ExternalPluginSpec> {
             BLACKBOARD_PLUGIN_ID.into(),
         ],
         url: None,
+        env: BTreeMap::new(),
     })
 }
 
@@ -405,6 +424,7 @@ pub fn blobstore_plugin_spec() -> Result<ExternalPluginSpec> {
             BLOBSTORE_PLUGIN_ID.into(),
         ],
         url: None,
+        env: BTreeMap::new(),
     })
 }
 
@@ -423,6 +443,79 @@ pub fn openai_endpoint_plugin_spec() -> Result<ExternalPluginSpec> {
             OPENAI_ENDPOINT_PLUGIN_ID.into(),
         ],
         url: None,
+        env: BTreeMap::new(),
+    })
+}
+
+pub fn flash_moe_plugin_spec(entry: &PluginConfigEntry) -> Result<ExternalPluginSpec> {
+    let backend_command = entry
+        .command
+        .as_deref()
+        .map(str::trim)
+        .filter(|value| !value.is_empty());
+    let endpoint_url = entry
+        .url
+        .as_deref()
+        .map(str::trim)
+        .filter(|value| !value.is_empty());
+
+    if backend_command.is_some() && endpoint_url.is_some() {
+        bail!(
+            "Plugin '{}' accepts either `command` for a managed flash-moe process or `url` for an already-running endpoint, not both",
+            FLASH_MOE_PLUGIN_ID
+        );
+    }
+    if backend_command.is_none() && endpoint_url.is_none() {
+        bail!(
+            "Plugin '{}' requires `command` or `url`. {}",
+            FLASH_MOE_PLUGIN_ID,
+            FLASH_MOE_INSTALL_HINT
+        );
+    }
+    if backend_command.is_none() && !entry.args.is_empty() {
+        bail!("Plugin '{}' args require `command`", FLASH_MOE_PLUGIN_ID);
+    }
+    if entry
+        .args
+        .iter()
+        .any(|arg| arg == "--serve" || arg.starts_with("--serve="))
+    {
+        bail!(
+            "Plugin '{}' owns the flash-moe `--serve` port; remove `--serve` from args",
+            FLASH_MOE_PLUGIN_ID
+        );
+    }
+
+    let command = std::env::current_exe()
+        .context("Cannot determine mesh-llm executable path")?
+        .display()
+        .to_string();
+    let mut env = BTreeMap::new();
+    if let Some(backend_command) = backend_command {
+        env.insert(
+            "MESH_LLM_FLASH_MOE_COMMAND".to_string(),
+            backend_command.to_string(),
+        );
+        env.insert(
+            "MESH_LLM_FLASH_MOE_ARGS_JSON".to_string(),
+            serde_json::to_string(&entry.args)?,
+        );
+    }
+    if let Some(url) = endpoint_url {
+        env.insert("MESH_LLM_FLASH_MOE_URL".to_string(), url.to_string());
+    }
+
+    Ok(ExternalPluginSpec {
+        name: FLASH_MOE_PLUGIN_ID.to_string(),
+        command,
+        args: vec![
+            "--log-format".into(),
+            "json".into(),
+            "--plugin".into(),
+            FLASH_MOE_PLUGIN_ID.into(),
+        ],
+        url: None,
+        env,
     })
 }
 
@@ -441,6 +534,7 @@ pub fn telemetry_plugin_spec() -> Result<ExternalPluginSpec> {
             TELEMETRY_PLUGIN_ID.into(),
         ],
         url: None,
+        env: BTreeMap::new(),
     })
 }
 
@@ -636,6 +730,25 @@ prompt_shape_metrics = true
         );
     }
 
+    #[test]
+    fn flash_moe_config_requires_external_command_or_endpoint_with_install_hint() {
+        let entry = PluginConfigEntry {
+            name: FLASH_MOE_PLUGIN_ID.to_string(),
+            enabled: Some(true),
+            command: None,
+            args: Vec::new(),
+            url: None,
+        };
+
+        let err = flash_moe_plugin_spec(&entry)
+            .expect_err("flash-moe requires a managed command or attached endpoint");
+        let message = err.to_string();
+
+        assert!(message.contains("Install Flash-MoE separately"));
+        assert!(message.contains("command"));
+        assert!(message.contains("url"));
+    }
+
     #[test]
     fn pinned_gpu_config_accepted_pinned_config() {
         let config: MeshConfig = toml::from_str(

crates/mesh-llm-host-runtime/src/plugin/mod.rs

@@ -50,6 +50,7 @@ use mesh_llm_plugin::MeshVisibility;
 
 pub const BLACKBOARD_PLUGIN_ID: &str = "blackboard";
 pub const BLOBSTORE_PLUGIN_ID: &str = "blobstore";
+pub const FLASH_MOE_PLUGIN_ID: &str = "flash-moe";
 pub const OPENAI_ENDPOINT_PLUGIN_ID: &str = "openai-endpoint";
 pub const TELEMETRY_PLUGIN_ID: &str = "telemetry";
 pub const TELEMETRY_CAPABILITY: &str = "telemetry.metrics.v1";
@@ -1718,6 +1719,7 @@ pub async fn run_plugin_process(name: String) -> Result<()> {
     match name.as_str() {
         BLACKBOARD_PLUGIN_ID => crate::plugins::blackboard::run_plugin(name).await,
         BLOBSTORE_PLUGIN_ID => crate::plugins::blobstore::run_plugin(name).await,
+        FLASH_MOE_PLUGIN_ID => crate::plugins::flash_moe::run_plugin(name).await,
         OPENAI_ENDPOINT_PLUGIN_ID => crate::plugins::openai_endpoint::run_plugin(name).await,
         TELEMETRY_PLUGIN_ID => crate::plugins::telemetry::run_plugin(name).await,
         _ => bail!("Unknown built-in plugin '{}'", name),
@@ -1922,6 +1924,112 @@ mod tests {
         assert!(result.is_err());
     }
 
+    #[test]
+    fn flash_moe_can_be_enabled_with_managed_command() {
+        let config = MeshConfig {
+            plugins: vec![PluginConfigEntry {
+                name: FLASH_MOE_PLUGIN_ID.into(),
+                enabled: Some(true),
+                command: Some(" /opt/flash-moe/infer ".into()),
+                args: vec![
+                    "--model".into(),
+                    "/models/qwen3.5".into(),
+                    "--weights".into(),
+                    "/models/experts.bin".into(),
+                ],
+                url: None,
+            }],
+            ..MeshConfig::default()
+        };
+
+        let resolved = resolve_plugins(&config, private_host_mode()).unwrap();
+
+        assert_eq!(resolved.externals.len(), 4);
+        assert_eq!(resolved.externals[0].name, BLACKBOARD_PLUGIN_ID);
+        assert_eq!(resolved.externals[1].name, TELEMETRY_PLUGIN_ID);
+        assert_eq!(resolved.externals[2].name, FLASH_MOE_PLUGIN_ID);
+        assert_eq!(resolved.externals[3].name, BLOBSTORE_PLUGIN_ID);
+        let spec = &resolved.externals[2];
+        assert!(spec.args.contains(&"--plugin".to_string()));
+        assert!(spec.args.contains(&FLASH_MOE_PLUGIN_ID.to_string()));
+        assert_eq!(
+            spec.env
+                .get("MESH_LLM_FLASH_MOE_COMMAND")
+                .map(String::as_str),
+            Some("/opt/flash-moe/infer")
+        );
+        assert_eq!(
+            spec.env
+                .get("MESH_LLM_FLASH_MOE_ARGS_JSON")
+                .map(String::as_str),
+            Some(r#"["--model","/models/qwen3.5","--weights","/models/experts.bin"]"#)
+        );
+        assert!(!spec.env.contains_key("MESH_LLM_FLASH_MOE_URL"));
+        assert_eq!(spec.url, None);
+    }
+
+    #[test]
+    fn flash_moe_can_attach_existing_endpoint() {
+        let config = MeshConfig {
+            plugins: vec![PluginConfigEntry {
+                name: FLASH_MOE_PLUGIN_ID.into(),
+                enabled: Some(true),
+                command: None,
+                args: Vec::new(),
+                url: Some(" http://127.0.0.1:8000/v1/ ".into()),
+            }],
+            ..MeshConfig::default()
+        };
+
+        let resolved = resolve_plugins(&config, private_host_mode()).unwrap();
+        let spec = resolved
+            .externals
+            .iter()
+            .find(|spec| spec.name == FLASH_MOE_PLUGIN_ID)
+            .expect("flash-moe spec");
+
+        assert_eq!(
+            spec.env.get("MESH_LLM_FLASH_MOE_URL").map(String::as_str),
+            Some("http://127.0.0.1:8000/v1/")
+        );
+        assert!(!spec.env.contains_key("MESH_LLM_FLASH_MOE_COMMAND"));
+        assert!(spec.args.contains(&FLASH_MOE_PLUGIN_ID.to_string()));
+    }
+
+    #[test]
+    fn flash_moe_rejects_missing_command_or_url() {
+        let config = MeshConfig {
+            plugins: vec![PluginConfigEntry {
+                name: FLASH_MOE_PLUGIN_ID.into(),
+                enabled: Some(true),
+                command: None,
+                args: Vec::new(),
+                url: None,
+            }],
+            ..MeshConfig::default()
+        };
+
+        let err = resolve_plugins(&config, private_host_mode()).unwrap_err();
+        assert!(err.to_string().contains("requires `command` or `url`"));
+    }
+
+    #[test]
+    fn flash_moe_rejects_user_supplied_serve_arg() {
+        let config = MeshConfig {
+            plugins: vec![PluginConfigEntry {
+                name: FLASH_MOE_PLUGIN_ID.into(),
+                enabled: Some(true),
+                command: Some("/opt/flash-moe/infer".into()),
+                args: vec!["--serve".into(), "9000".into()],
+                url: None,
+            }],
+            ..MeshConfig::default()
+        };
+
+        let err = resolve_plugins(&config, private_host_mode()).unwrap_err();
+        assert!(err.to_string().contains("owns the flash-moe `--serve`"));
+    }
+
     #[test]
     fn blackboard_is_resolved_on_public_meshes() {
         let resolved = resolve_plugins(

crates/mesh-llm-host-runtime/src/plugin/runtime.rs

@@ -189,6 +189,9 @@ impl ExternalPlugin {
         if let Some(ref url) = self.spec.url {
             child.env("MESH_LLM_OPENAI_ENDPOINT_URL", url);
         }
+        for (key, value) in &self.spec.env {
+            child.env(key, value);
+        }
         child.stdin(std::process::Stdio::null());
         child.stdout(std::process::Stdio::null());
         child.stderr(std::process::Stdio::inherit());

crates/mesh-llm-host-runtime/src/plugins/flash_moe/mod.rs

@@ -0,0 +1,415 @@
+use std::sync::Arc;
+use std::time::Duration;
+
+use anyhow::{bail, Context, Result};
+use mesh_llm_plugin::{
+    capability, plugin_server_info, PluginMetadata, PluginRuntime, PluginStartupPolicy,
+};
+use tokio::process::{Child, Command};
+use tokio::sync::Mutex;
+
+const ENV_COMMAND: &str = "MESH_LLM_FLASH_MOE_COMMAND";
+const ENV_ARGS_JSON: &str = "MESH_LLM_FLASH_MOE_ARGS_JSON";
+const ENV_URL: &str = "MESH_LLM_FLASH_MOE_URL";
+const ENDPOINT_ID: &str = "flash-moe";
+const HEALTH_TIMEOUT: Duration = Duration::from_secs(1);
+const INSTALL_HINT: &str = "Install Flash-MoE separately and set \
+                           MESH_LLM_FLASH_MOE_COMMAND to its infer binary, \
+                           or set MESH_LLM_FLASH_MOE_URL to an already-running \
+                           Flash-MoE /v1 endpoint.";
+
+#[derive(Clone, Debug, PartialEq, Eq)]
+enum FlashMoeSource {
+    Managed {
+        command: String,
+        args: Vec<String>,
+        port: u16,
+    },
+    External {
+        base_url: String,
+    },
+}
+
+#[derive(Clone, Debug, PartialEq, Eq)]
+struct FlashMoeConfig {
+    source: FlashMoeSource,
+}
+
+impl FlashMoeConfig {
+    fn from_env() -> Result<Self> {
+        Self::from_values(
+            std::env::var(ENV_COMMAND).ok(),
+            std::env::var(ENV_ARGS_JSON).ok(),
+            std::env::var(ENV_URL).ok(),
+        )
+    }
+
+    fn from_values(
+        command: Option<String>,
+        args_json: Option<String>,
+        url: Option<String>,
+    ) -> Result<Self> {
+        let command = command
+            .map(|value| value.trim().to_string())
+            .filter(|value| !value.is_empty());
+        let url = url
+            .map(|value| value.trim().to_string())
+            .filter(|value| !value.is_empty());
+        if command.is_some() && url.is_some() {
+            bail!("flash-moe plugin accepts either {ENV_COMMAND} or {ENV_URL}, not both");
+        }
+        if let Some(base_url) = url {
+            return Ok(Self {
+                source: FlashMoeSource::External {
+                    base_url: normalize_base_url(&base_url),
+                },
+            });
+        }
+        let command = command.with_context(|| {
+            format!("flash-moe plugin requires {ENV_COMMAND} or {ENV_URL}. {INSTALL_HINT}")
+        })?;
+        let args = match args_json {
+            Some(raw) if !raw.trim().is_empty() => serde_json::from_str::<Vec<String>>(&raw)
+                .with_context(|| format!("parse {ENV_ARGS_JSON}"))?,
+            _ => Vec::new(),
+        };
+        let port = allocate_local_port().context("allocate flash-moe endpoint port")?;
+        Ok(Self {
+            source: FlashMoeSource::Managed {
+                command,
+                args,
+                port,
+            },
+        })
+    }
+
+    fn endpoint_base_url(&self) -> String {
+        match &self.source {
+            FlashMoeSource::Managed { port, .. } => format!("http://127.0.0.1:{port}/v1"),
+            FlashMoeSource::External { base_url } => base_url.clone(),
+        }
+    }
+
+    fn managed(&self) -> bool {
+        matches!(self.source, FlashMoeSource::Managed { .. })
+    }
+}
+
+#[derive(Clone)]
+struct FlashMoeState {
+    config: FlashMoeConfig,
+    child: Arc<Mutex<Option<Child>>>,
+}
+
+impl FlashMoeState {
+    fn new(config: FlashMoeConfig) -> Self {
+        Self {
+            config,
+            child: Arc::new(Mutex::new(None)),
+        }
+    }
+
+    async fn ensure_started(&self) -> Result<()> {
+        let FlashMoeSource::Managed {
+            command,
+            args,
+            port,
+        } = &self.config.source
+        else {
+            return Ok(());
+        };
+
+        let mut child = self.child.lock().await;
+        if let Some(existing) = child.as_mut() {
+            match existing.try_wait().context("poll flash-moe process")? {
+                Some(status) => bail!("flash-moe exited before readiness: {status}"),
+                None => return Ok(()),
+            }
+        }
+
+        let managed_args = managed_command_args(args, *port)?;
+        let mut command_builder = Command::new(command);
+        command_builder.args(&managed_args);
+        command_builder.stdin(std::process::Stdio::null());
+        command_builder.stdout(std::process::Stdio::null());
+        command_builder.stderr(std::process::Stdio::inherit());
+        command_builder.kill_on_drop(true);
+        let spawned = command_builder
+            .spawn()
+            .with_context(|| format!("launch flash-moe backend via {command}. {INSTALL_HINT}"))?;
+        *child = Some(spawned);
+        Ok(())
+    }
+
+    async fn health(&self) -> Result<String> {
+        match &self.config.source {
+            FlashMoeSource::External { base_url } => Ok(format!("external_url={base_url}")),
+            FlashMoeSource::Managed { port, .. } => {
+                let mut child = self.child.lock().await;
+                match child.as_mut() {
+                    Some(process) => {
+                        if let Some(status) =
+                            process.try_wait().context("poll flash-moe process")?
+                        {
+                            *child = None;
+                            bail!("flash-moe exited with {status}");
+                        }
+                    }
+                    None => return Ok(format!("starting endpoint=http://127.0.0.1:{port}/v1")),
+                }
+                drop(child);
+
+                let health_url = format!("http://127.0.0.1:{port}/health");
+                match reqwest::Client::builder()
+                    .timeout(HEALTH_TIMEOUT)
+                    .build()
+                    .context("build flash-moe health client")?
+                    .get(&health_url)
+                    .send()
+                    .await
+                {
+                    Ok(response) if response.status().is_success() => {
+                        Ok(format!("endpoint=http://127.0.0.1:{port}/v1"))
+                    }
+                    Ok(response) => Ok(format!(
+                        "starting endpoint=http://127.0.0.1:{port}/v1 health_status={}",
+                        response.status()
+                    )),
+                    Err(error) => Ok(format!(
+                        "starting endpoint=http://127.0.0.1:{port}/v1 detail={error}"
+                    )),
+                }
+            }
+        }
+    }
+}
+
+fn normalize_base_url(value: &str) -> String {
+    value.trim().trim_end_matches('/').to_string()
+}
+
+fn allocate_local_port() -> Result<u16> {
+    let listener = std::net::TcpListener::bind("127.0.0.1:0")?;
+    let port = listener.local_addr()?.port();
+    drop(listener);
+    Ok(port)
+}
+
+fn managed_command_args(user_args: &[String], port: u16) -> Result<Vec<String>> {
+    if user_args
+        .iter()
+        .any(|arg| arg == "--serve" || arg.starts_with("--serve="))
+    {
+        bail!("mesh-llm owns the flash-moe --serve port; remove --serve from plugin args");
+    }
+    let mut args = user_args.to_vec();
+    args.push("--serve".to_string());
+    args.push(port.to_string());
+    Ok(args)
+}
+
+fn build_plugin_from_config(name: String, config: FlashMoeConfig) -> mesh_llm_plugin::SimplePlugin {
+    let endpoint_base = config.endpoint_base_url();
+    let state = FlashMoeState::new(config);
+    let startup_state = state.clone();
+    let health_state = state.clone();
+    let inference_endpoint = if state.config.managed() {
+        mesh_llm_plugin::inference::provider(ENDPOINT_ID, endpoint_base.clone())
+    } else {
+        mesh_llm_plugin::inference::openai_http(ENDPOINT_ID, endpoint_base.clone())
+    };
+
+    mesh_llm_plugin::plugin! {
+        metadata: PluginMetadata::new(
+            name,
+            crate::VERSION,
+            plugin_server_info(
+                "mesh-flash-moe",
+                crate::VERSION,
+                "Flash-MoE SSD Expert Streaming Provider",
+                "Registers a flash-moe OpenAI-compatible endpoint for single-node SSD expert streaming.",
+                Some(
+                    "Configure [[plugin]] name = \"flash-moe\" with either command/args \
+                     for a managed flash-moe process or url for an already-running endpoint. \
+                     Flash-MoE itself is installed separately.",
+                ),
+            ),
+        ),
+        startup_policy: PluginStartupPolicy::Any,
+        provides: [
+            capability("endpoint:inference"),
+            capability("endpoint:inference/openai_compatible"),
+            capability("backend:flash-moe"),
+            capability("backend:ssd-expert-streaming"),
+        ],
+        inference: [
+            inference_endpoint,
+        ],
+        health: move |_context| {
+            let health_state = health_state.clone();
+            Box::pin(async move { health_state.health().await })
+        },
+        on_initialized: move |_context| {
+            let startup_state = startup_state.clone();
+            Box::pin(async move { startup_state.ensure_started().await })
+        },
+    }
+}
+
+fn build_plugin(name: String) -> Result<mesh_llm_plugin::SimplePlugin> {
+    let config = FlashMoeConfig::from_env()?;
+    Ok(build_plugin_from_config(name, config))
+}
+
+pub(crate) async fn run_plugin(name: String) -> Result<()> {
+    PluginRuntime::run(build_plugin(name)?).await
+}
+
+#[cfg(test)]
+mod tests {
+    use mesh_llm_plugin::Plugin;
+
+    use super::*;
+
+    #[test]
+    fn managed_command_args_append_owned_serve_port() {
+        let args = managed_command_args(
+            &[
+                "--model".to_string(),
+                "/models/qwen3.5".to_string(),
+                "--tokens".to_string(),
+                "128".to_string(),
+            ],
+            8123,
+        )
+        .unwrap();
+
+        assert_eq!(
+            args,
+            vec![
+                "--model",
+                "/models/qwen3.5",
+                "--tokens",
+                "128",
+                "--serve",
+                "8123"
+            ]
+        );
+    }
+
+    #[test]
+    fn managed_command_args_reject_user_supplied_serve_port() {
+        let err = managed_command_args(&["--serve".to_string(), "9000".to_string()], 8123)
+            .expect_err("user-owned serve port must be rejected");
+        assert!(err.to_string().contains("--serve"));
+    }
+
+    #[test]
+    fn config_accepts_external_url_mode() {
+        let config = FlashMoeConfig::from_values(
+            None,
+            None,
+            Some(" http://127.0.0.1:8000/v1/ ".to_string()),
+        )
+        .unwrap();
+
+        assert_eq!(
+            config.source,
+            FlashMoeSource::External {
+                base_url: "http://127.0.0.1:8000/v1".to_string()
+            }
+        );
+        assert_eq!(config.endpoint_base_url(), "http://127.0.0.1:8000/v1");
+    }
+
+    #[test]
+    fn config_rejects_command_and_url_together() {
+        let err = FlashMoeConfig::from_values(
+            Some("/opt/flash-moe/infer".to_string()),
+            None,
+            Some("http://127.0.0.1:8000/v1".to_string()),
+        )
+        .expect_err("command and url must be mutually exclusive");
+        assert!(err.to_string().contains("not both"));
+    }
+
+    #[test]
+    fn config_missing_source_explains_external_install_boundary() {
+        let err = FlashMoeConfig::from_values(None, None, None)
+            .expect_err("flash-moe requires a managed command or attached endpoint");
+        let message = err.to_string();
+
+        assert!(message.contains("Install Flash-MoE separately"));
+        assert!(message.contains(ENV_COMMAND));
+        assert!(message.contains(ENV_URL));
+    }
+
+    #[tokio::test]
+    async fn managed_start_failure_explains_external_install_boundary() {
+        let state = FlashMoeState::new(FlashMoeConfig {
+            source: FlashMoeSource::Managed {
+                command: "/definitely/missing/flash-moe/infer".to_string(),
+                args: Vec::new(),
+                port: 8123,
+            },
+        });
+
+        let message = state
+            .ensure_started()
+            .await
+            .expect_err("missing external flash-moe binary must fail")
+            .to_string();
+
+        assert!(message.contains("Install Flash-MoE separately"));
+        assert!(message.contains("/definitely/missing/flash-moe/infer"));
+    }
+
+    #[test]
+    fn managed_manifest_declares_plugin_owned_openai_endpoint() {
+        let plugin = build_plugin_from_config(
+            "flash-moe".to_string(),
+            FlashMoeConfig {
+                source: FlashMoeSource::Managed {
+                    command: "/opt/flash-moe/infer".to_string(),
+                    args: vec!["--model".to_string(), "/models/qwen3.5".to_string()],
+                    port: 8123,
+                },
+            },
+        );
+        let manifest = plugin.manifest().expect("manifest");
+        let endpoint = manifest.endpoints.first().expect("endpoint");
+
+        assert_eq!(endpoint.endpoint_id, ENDPOINT_ID);
+        assert_eq!(
+            endpoint.address.as_deref(),
+            Some("http://127.0.0.1:8123/v1")
+        );
+        assert_eq!(endpoint.protocol.as_deref(), Some("openai_compatible"));
+        assert!(endpoint.supports_streaming);
+        assert!(endpoint.managed_by_plugin);
+        assert!(manifest
+            .capabilities
+            .iter()
+            .any(|capability| capability == "backend:ssd-expert-streaming"));
+    }
+
+    #[test]
+    fn external_manifest_declares_attached_openai_endpoint() {
+        let plugin = build_plugin_from_config(
+            "flash-moe".to_string(),
+            FlashMoeConfig {
+                source: FlashMoeSource::External {
+                    base_url: "http://127.0.0.1:8000/v1".to_string(),
+                },
+            },
+        );
+        let manifest = plugin.manifest().expect("manifest");
+        let endpoint = manifest.endpoints.first().expect("endpoint");
+
+        assert_eq!(
+            endpoint.address.as_deref(),
+            Some("http://127.0.0.1:8000/v1")
+        );
+        assert!(!endpoint.managed_by_plugin);
+    }
+}

crates/mesh-llm-host-runtime/src/plugins/mod.rs

@@ -1,4 +1,5 @@
 pub mod blackboard;
 pub mod blobstore;
+pub mod flash_moe;
 pub mod openai_endpoint;
 pub mod telemetry;

crates/mesh-llm/TODO.md

@@ -51,7 +51,9 @@ Run giant MoE models on a single node by streaming active experts from NVMe inst
 
 [flash-moe](https://github.com/danveloper/flash-moe) already does this — runs Qwen3.5-397B-A17B at 5.5 tok/s on a 48GB M3 Max with 6GB resident memory. See [ROADMAP.md](../../ROADMAP.md).
 
-**Plan:** Use flash-moe as an alternative backend. Mesh-llm spawns it like llama-server. Needs HTTP/SSE endpoint (currently CLI only) and OpenAI-compatible `/v1/chat/completions`.
+Initial adapter: mesh-llm registers Flash-MoE through the built-in `flash-moe` plugin, either by spawning the Flash-MoE `infer --serve` process or by attaching an already-running OpenAI-compatible endpoint. The adapter keeps Flash-MoE installation, source builds, and model artifact preparation outside mesh-llm.
+
+Remaining work: document the exact Flash-MoE artifact preparation flow, add real-machine smoke coverage for the target Qwen3.5-397B path, and revisit out-of-tree adapter packaging once the plugin SDK and crates.io surface are stable enough.
 
 ## MoE Expert Sharding
 

docs/plugins/README.md

@@ -8,6 +8,7 @@ As implementation lands, this document should be updated to match the intended e
 
 Plugin-specific documentation:
 
+- [Flash-MoE](flash-moe.md) - built-in OpenAI-compatible backend adapter for single-node SSD expert streaming
 - [Telemetry](telemetry.md) - built-in OTLP metrics-only runtime and routing telemetry
 
 The main goals are:

docs/plugins/flash-moe.md

@@ -0,0 +1,95 @@
+# Flash-MoE Plugin
+
+The built-in `flash-moe` plugin connects mesh-llm to a Flash-MoE OpenAI-compatible HTTP server.
+
+Use it for the SSD expert streaming roadmap path: a giant MoE model fits on one node's local NVMe, but not in RAM. Mesh-llm owns process lifecycle, plugin health, endpoint discovery, and request routing. Flash-MoE owns model execution.
+
+This is intentionally a single-node backend adapter. It does not change the mesh protocol, Skippy stage protocol, model-package format, or llama.cpp patch queue.
+
+## Prerequisites
+
+Flash-MoE is an external backend. Mesh-llm does not vendor, install, or build the Flash-MoE binary, model conversion tooling, or SSD-streaming artifacts.
+
+Install or build Flash-MoE separately from [danveloper/flash-moe](https://github.com/danveloper/flash-moe), prepare the model files with its tooling, then either:
+
+- set `command` to the local Flash-MoE `infer` binary for managed process mode; or
+- set `url` to an already-running Flash-MoE `/v1` endpoint.
+
+If upstream release artifacts are not available for your platform, use a source build or deployment-managed binary and point mesh-llm at that binary. This adapter intentionally keeps Flash-MoE ownership outside mesh-llm until the backend and packaging surface are mature enough to standardize.
+
+## Managed Process Mode
+
+Point `command` at the Flash-MoE `infer` binary and pass the normal model arguments in `args`.
+
+```toml
+[[plugin]]
+name = "flash-moe"
+command = "/opt/flash-moe/metal_infer/infer"
+args = [
+  "--model", "/models/qwen3.5-397b/model.gguf",
+  "--weights", "/models/qwen3.5-397b/experts.bin",
+  "--manifest", "/models/qwen3.5-397b/manifest.json",
+  "--vocab", "/models/qwen3.5-397b/vocab.json"
+]
+```
+
+Mesh-llm allocates a local port and appends:
+
+```text
+--serve <port>
+```
+
+Do not pass `--serve` in config. Keeping the port host-owned prevents collisions between plugins, external backends, and local llama.cpp serving.
+
+When the plugin starts, it registers an OpenAI-compatible inference endpoint like:
+
+```text
+http://127.0.0.1:<port>/v1
+```
+
+The host probes `GET /v1/models` through the normal plugin endpoint health path, so Flash-MoE models appear and disappear the same way other plugin-backed models do.
+
+## Existing Endpoint Mode
+
+If Flash-MoE is already running, attach the endpoint instead of letting mesh-llm spawn it:
+
+```toml
+[[plugin]]
+name = "flash-moe"
+url = "http://127.0.0.1:8000/v1"
+```
+
+This mode leaves process lifecycle outside mesh-llm and only registers the endpoint.
+
+## Config Rules
+
+- Configure either `command` or `url`, not both.
+- `args` require `command`.
+- `--serve` is host-owned and must not appear in `args`.
+- Model paths and weights stay local to the node running Flash-MoE.
+- No HuggingFace token or private credential is required by the plugin itself.
+
+## Packaging Boundary
+
+The first adapter stays in-tree because it depends on mesh-llm's host-runtime plugin lifecycle, local endpoint registration, environment handoff, process ownership, and health model. A separate adapter crate or repository is a better follow-up once the plugin SDK and crates.io surface are stable enough for out-of-tree inference providers.
+
+## Scope
+
+Included:
+
+- bundled `flash-moe` plugin entrypoint
+- managed Flash-MoE process launch
+- existing HTTP endpoint attachment
+- OpenAI-compatible endpoint registration
+- plugin health and lifecycle checks
+- model discovery via the existing `/v1/models` probe path
+
+Not included:
+
+- mesh-distributed SSD expert streaming
+- Skippy package slicing changes
+- Flash-MoE binary vendoring
+- Flash-MoE installation or source-build automation
+- Flash-MoE model conversion or download automation
+- out-of-tree adapter crate/repository packaging
+- changes to public mesh protocol fields