fluffos/docs/build-wasm.md
Claude 3bdbe9676e www: real ws backpressure fix, teardown leak fix, src/www docs, websocket smoke test in CI
The lws output wedge addressed in the previous commit was incompletely
understood: under genuine backpressure (peer slow or paused, kernel
send buffer full) a connection still froze permanently. Root cause,
established by tracing the writeable-request plumbing end to end:
lws_send_pipe_choked() is true not only when lws holds a truncated
send (lws re-arms the writeable callback itself then) but also when a
zero-timeout poll(POLLOUT) reports the socket simply full -- and in
that case every lws_write() has fully succeeded, lws has nothing
pending, and nobody re-arms anything. Fix, per lws README.coding.md:
whenever the drain loop exits with data still queued in pss->buffer,
request the next writeable callback. Queued data always has a callback
requested, so no exit path can strand output.

Also:
- LWS_CALLBACK_CLOSED frees the session evbuffer unconditionally: on
  driver-initiated closes (e.g. the mudlib destructing the interactive)
  close_user_websocket() nulls pss->user first, and the old early
  return leaked the buffer every time.
- src/www/README.md + src/www/AGENTS.md: architecture doc and agent
  checklist for the web terminal pages (xterm.js/telnet.js layering,
  vendor policy, packaging, testing, the wedge mechanism).
- tools/ws-smoke.js, wired into CI on the Clang Debug matrix entries
  (with and without sanitizers): a dependency-free node websocket
  client that boots the real driver and exercises the http mount,
  telnet + ascii subprotocols through the shared src/www/telnet.js,
  SGA char-mode switching, live TUI streaming, TLS, and -- the actual
  regression gate -- forced-backpressure bursts (paused socket, ~4.8MB)
  on the plain and TLS ports plus a destruct-while-choked teardown
  check. All three backpressure checks fail on the unfixed driver;
  neither GTest nor the LPC suite exercises any websocket client
  traffic.
- Fix stale src/www/wasm/vendor/ path references left from the vendor
  directory move (src/wasm/README.md, docs/build-wasm.md, release.yml).

Validated on the ASan Debug build: forced-backpressure repros recover
the full burst on both subprotocols, destruct-while-choked clean under
ASan, ws-smoke 17/17, GTest 312/312, LPC testsuite clean.
2026-07-14 09:53:26 -05:00

8.2 KiB

title
Build (WebAssembly)

Running FluffOS in the browser (WebAssembly)

FluffOS cross-compiles to WebAssembly with Emscripten: the whole compiler + VM + efun stack runs inside a webpage, boots a real mudlib from an in-memory filesystem, and speaks real telnet with the page — the page is the telnet client, JavaScript is the wire. The LPC testsuite passes in the browser and under node.

This page is the end-to-end workflow: build the toolchain deps once, build the driver, package your mudlib, serve it, embed it. For the architecture (event-loop inversion, the Transport interface, what is and isn't supported) see src/wasm/README.md.

Don't want to build? Every FluffOS release ships fluffos-<version>-wasm.zip — the prebuilt driver, the web terminal, and pack-mudlib.sh. Unzip it and skip straight to step 3 (the packer still needs emscripten's file_packager, i.e. emsdk on PATH).

0. Prerequisites

  • Emscripten (emcc, emcmake, emmake on PATH) — install via emsdk, or your distro package (apt install emscripten). Avoid emsdk 6.0.2: it defaulted GROWABLE_ARRAYBUFFERS=1, and in browsers with wasm resizable-ArrayBuffer support the driver fails on boot with getRandomValues ... must not be resizable (and jsbridge string passing breaks the same way). Use 6.0.3+ — the version CI pins is in .github/actions/build-wasm/action.yml.
  • A native toolchain plus the usual FluffOS build deps (OpenSSL/ICU headers): the codegen tools run on the build machine, and ICU's cross build needs a native ICU build first.
  • cmake ≥ 3.22, curl, python3.

1. Build the WASM dependencies (once)

The driver needs a static WASM build of ICU (Unicode: grapheme iteration, charset conversion) — the only cross-built dependency:

tools/wasm/build-deps.sh          # installs into /opt/wasm-deps
# or: PREFIX=$HOME/wasm-deps tools/wasm/build-deps.sh

This is fully scripted, including the ICU cross-compile quirks (the mh-unknown platform file, and generating the data archive as C source with the host genccode because pkgdata cannot emit wasm objects). It only runs once; re-runs are no-ops.

The ICU data archive is trimmed to break-iterator data so the driver stays small (fluffos.wasm is ~3.6MB raw, ~0.8MB brotli) — table charsets (GBK, Big5, …) are not available on this target unless you rebuild the deps with ICU_KEEP (see the header of build-deps.sh). UTF-8/UTF-16/Latin-1/ASCII always work.

2. Build the driver

Two CMake presets do the two stages — codegen tools natively, then the cross build:

cmake --preset native-tools && cmake --build --preset native-tools
emcmake cmake --preset wasm  && cmake --build --preset wasm

Non-default deps prefix: emcmake cmake --preset wasm -DFLUFFOS_WASM_DEPS=$HOME/wasm-deps.

Output: build-wasm/src/fluffos.js + fluffos.wasm.

tools/wasm/build.sh runs both presets and packages the bundled testsuite as a demo in one command.

3. Package your mudlib

tools/wasm/pack-mudlib.sh turns any mudlib + the driver into a static web bundle:

tools/wasm/pack-mudlib.sh \
    --mudlib /path/to/mylib \
    --config etc/config \
    --out dist/
  • --mudlib is packed with Emscripten's file_packager into mudlib.data + mudlib.js, mounted read-write in the page's memory filesystem at --mount (default /<basename>).
  • --config is the runtime config relative to the mudlib root, exactly like running driver etc/config natively from that directory. Ports/TLS entries in the config are ignored (there are no listening sockets; the page connects directly).
  • The bundle contains index.html (the web terminal: xterm.js under vendor/, telnet client in telnet.js — see src/www/README.md), fluffos.js/fluffos.wasm, mudlib.js/mudlib.data, and fluffos-boot.js (mount + config for the page).

4. Serve it

Any static HTTP server works (wasm cannot load from file://):

python3 -m http.server -d dist 8080
# open http://localhost:8080/

For production, serve fluffos.wasm and mudlib.data with gzip/brotli (fluffos.wasm is ~0.8MB brotli) and long cache lifetimes.

5. Run the LPC testsuite in the WASM driver

node tools/wasm/run-testsuite.js         # boots testsuite/, runs -ftest

Exit code 0 plus Checks succeeded. is the pass signal — the same gate CI uses (see the wasm job in .github/workflows/ci.yml). Tests for packages that don't exist on this target (sockets, external, db, ffi, pcre, crypto, async, compress) skip themselves via #ifdef __PACKAGE_*__ guards.

6. Embedding API (custom frontends)

index.html keeps the page glue small (xterm.js does the emulation); any custom frontend can drive the same module:

const M = await createFluffOS({ print, printErr, locateFile });
M.FS.chdir('/mylib');                          // mudlib mount point
M.fluffos = {
  onOutput:     (id, bytes) => { /* server->client telnet bytes */ },
  onDisconnect: (id)        => { /* connection closed */ },
};
M.ccall('fluffos_boot', 'number', ['string'], ['etc/config']);
setInterval(() => M.ccall('fluffos_tick', 'number', ['number'],
                          [performance.now()]), 50);
const id = M.ccall('fluffos_connect', 'number', [], []);   // telnet "dial"
M.ccall('fluffos_input', null, ['number','array','number'],
        [id, bytes, bytes.length]);                        // client->server
// also exported: fluffos_flag (master::flag, e.g. 'test'),
// fluffos_disconnect, fluffos_shutdown

The driver does real telnet negotiation on each connection (ECHO for password masking, SGA char mode, NAWS, GMCP…), so the page needs a telnet layer — reuse src/www/telnet.js (transport-agnostic; the bundled terminal wires it to both the wasm bridge and websockets).

7. Calling JavaScript from LPC (jsbridge)

The WASM driver ships the jsbridge package: LPC code can reach the page's JavaScript — fetch(), canvas/WebGL, storage, anything the page exposes — and get called back asynchronously.

// synchronous eval; result as a string
write(js_eval("navigator.userAgent"));

// async: call a handler the page registered; cb(result, success, id)
void got_body(string body, int success, int id) { write(body); }
js_call("fetch_text", ({ "https://example.com/data.json" }),
        (: got_body :));

The page registers handlers on the module (they may return values or Promises; structured data crosses the bridge as JSON strings):

M.fluffos.handlers = {
  fetch_text:  async (url) => (await fetch(url)).text(),
  canvas_draw: (op, ...args) => { /* draw on a <canvas> */ },
};

The reverse direction — the page calling into LPC — is js_export:

js_export("add", (: lpc_add :));   // mixed lpc_add(string *args, int id)
const sum = await M.fluffos.callLPC("add", "2", "3");  // Promise -> "5"

Demo: the bundled web terminal wires fetch_text and canvas_draw handlers; try jsdemo eval 1+2, jsdemo fetch /index.html, jsdemo canvas (LPC drawing on the page's canvas), and jsdemo export followed by await fluffos.callLPC("add", "2", "3") in the devtools console — implemented in testsuite/command/jsdemo.lpc + src/www/wasm/index.html. See the WASM driver cookbook for more recipes.

Notes & limits

  • No eval limit yet on this target: a while(1); in LPC blocks the tab.
  • resolve() raises "DNS resolver is not available"; connections report 127.0.0.1.
  • Mudlib writes live in page memory for the session; persistent storage (IDBFS) is on the roadmap in src/wasm/README.md.
  • No zlib on this target: compressed save_object degrades to a plain save, gzip'd write_file raises an error, and .gz files aren't transparently decompressed.
  • Only algorithmic charsets ship (UTF-8/UTF-16/UTF-32, Latin-1, ASCII); string_encode() to a table charset raises an error. LPC can adapt with #ifdef __WASM__ (predefined on this target).
  • Background tabs throttle timers; the driver catches up (capped) when the tab wakes.