fluffos/tools/ffi
Yucong Sun 4a9b2ed2d0
docs: rewrite FFI reference around buffer features, add worked libc examples (#1275)
* docs: rewrite FFI reference around buffer features, add worked libc examples

Update docs/driver/ffi-plan.md now that buffers carry the full byte
toolkit (string/int-array promotion, byte lvalues, range assignment,
foreach, concatenation):

- New "The buffer type does the heavy lifting" section mapping each
  buffer language feature to its FFI use.
- The C-string idiom is now the two-line promotion form
  (buffer b = s; b += ({ 0 })) instead of chained string_encode()
  calls; string_encode stays documented for non-UTF-8 encodings, and
  the doc notes that promotion never reaches inside ffi_call's args
  array (the byte boundary stays explicit).
- New "Worked examples -- calling libc" section: scalars (sqrt/pow/abs),
  strings in (strlen, incl. range assignment into an allocation),
  strings out (getenv + ffi_peek(addr, -1) + string_decode with a NULL
  check), out-parameters (frexp), structs (time/localtime via
  ffi_struct_layout), callbacks (qsort with an LPC comparator), and
  foreach over peeked bytes.
- Sync the reference with the implementation: add ffi_address() (was
  missing from the efun listing), correct ffi_free() (zeroes bytes; GC
  reclaims), ffi_peek(-1) (NUL-terminated read capped at the max
  buffer size config), FFI_POINTER args accepting int addresses
  (0 = NULL), pointer returns always being int addresses, the
  FFI_INT/FFI_LONG aliases, ffi_load("") semantics, the valid_ffi
  operation names with a sample master implementation, and the real
  build default (PACKAGE_FFI ships ON; runtime denies without a
  valid_ffi apply; WASM forces it off).

Every snippet is pinned verbatim by a new testsuite file,
testsuite/single/tests/efuns/ffi_doc_examples.lpc (guarded by
__PACKAGE_FFI__ and the ffi_probe availability fixture like the other
ffi tests). Verified: RelWithDebInfo build, ffi glob (21 files) and the
full LPC suite (600 files) pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SooGWcJbSWszCZ67k4iVQT

* docs: add a local build quickstart to AGENTS.md

Record the verified Debian/Ubuntu setup so future agents don't
rediscover it one configure failure at a time: the full apt package
list (CI's packages: lines assume a GitHub runner image that
preinstalls cmake/ninja/libicu-dev), the configure/build commands, the
build/src/driver binary path, -DPACKAGE_DB=OFF as the escape hatch
when no MySQL client dev package is available, that libevent is
vendored and GTest optional, and how to run the LPC suite without a
pipe masking the driver's exit status.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SooGWcJbSWszCZ67k4iVQT

* docs: rename driver/ffi-plan.md to driver/ffi.md

The page stopped being a plan and is the package reference; name it
accordingly (site URL /driver/ffi/, sidebar label "FFI Package").
Update every reference: source comments (ffi.spec, ffi.cc, both
include/ffi.h copies), tools/ffi (README, generate.py's module
docstring and the two comment lines it emits into generated bindings,
with the committed ffi_genmath.lpc fixture updated to match), the
testsuite doc-examples pin, sidebar_meta.json + regenerated
sidebars.generated.json, and the zh-CN sidebar-label key.

Note: the old /driver/ffi-plan/ URL is not redirected (the site has no
client-redirects plugin configured).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SooGWcJbSWszCZ67k4iVQT

* ffi: gate ffi_peek through valid_ffi("peek")

ffi_peek(address, nbytes) copies bytes from an arbitrary native
address, which makes it a process-memory disclosure primitive on its
own (and a driver crash on an unmapped address) -- yet it was the one
dangerous efun in the package not gated by valid_ffi, so any object
could call it whenever PACKAGE_FFI was compiled in, even under a
deny-all master. The other ungated efuns operate only on LPC-owned
buffers and handles (ffi_address reveals a buffer's own address, inert
without a peek/call grant) and stay ungated.

f_ffi_peek now calls check_valid_ffi("peek", address) before touching
any memory, exactly like load/symbol/prepare/callback. The testsuite
master denies a -0xDEAD sentinel address so the denial path is
testable without dereferencing anything; ffi_peek.lpc pins the exact
error. Docs updated: valid_ffi(4), ffi_peek(3), and the op list +
sample master in docs/driver/ffi.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SooGWcJbSWszCZ67k4iVQT

* stdlib: add /std/ffi_util, FFI-boundary helpers for package_ffi callers

A pure-LPC library packaging the recurring idioms at the C boundary so
binding code stops re-deriving them:

- cstr(s) / cstr_enc(s, enc): NUL-terminated C strings (buffer
  promotion for UTF-8, string_encode for other byte encodings)
- c_string(addr) / c_string_enc(addr, enc): read a returned char*,
  with the NULL -> 0 convention folded in
- c_out(type_code): a zeroed out-parameter block for T*
- c_argv(strings): a NULL-terminated char*[] plus the kept-alive
  per-string buffers (the lifetime footgun a library should own)
- c_field / c_field_set: struct-field access over ffi_struct_layout()
  layouts

Deliberately consumed via `inherit "/std/ffi_util"` rather than the
simul_efun object: c_string() calls ffi_peek(), gated by
valid_ffi("peek", addr, caller), and inheriting keeps the consuming
object as the security principal the master sees. The docs page
(docs/stdlib/ffi_util, "FFI Utilities" in the sidebar) spells this out.

Tests: testsuite/single/tests/std/ffi_util.lpc exercises every helper
against libc (strlen with both encodings, getenv through c_string,
frexp through c_out, strtol's char **endptr through
c_out(FFI_POINTER), and structural checks on c_argv's pointer array).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SooGWcJbSWszCZ67k4iVQT

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-15 22:09:25 -07:00
..
generate.py docs: rewrite FFI reference around buffer features, add worked libc examples (#1275) 2026-07-15 22:09:25 -07:00
README.md docs: rewrite FFI reference around buffer features, add worked libc examples (#1275) 2026-07-15 22:09:25 -07:00
test.py package_ffi: foreign function interface for LPC (libffi), with callbacks 2026-07-09 20:48:48 -04:00
test_sample.h package_ffi: foreign function interface for LPC (libffi), with callbacks 2026-07-09 20:48:48 -04:00

tools/ffi — LPC bindings generator for package_ffi

Generates ready-to-use LPC bindings for a native library from its C header, targeting the FFI package (src/packages/ffi, efuns declared in <ffi.h>). See the full design in docs/driver/ffi.md.

Usage

tools/ffi/generate.py <header.h> --lib <libpath> --out <basename> \
    [--string-convenience] [--emit-json]

Emits next to <basename>:

File Contents
<basename>.lpc One LPC wrapper per exported C function. Handles are prepared lazily and cached; the library is ffi_loaded on first use.
<basename>_structs.h ffi_struct_layout field-type arrays + symbolic field-offset #defines for each struct in the header.

Example:

tools/ffi/generate.py /usr/include/math.h --lib libm.so.6 --out mudlib/std/libm

The buffer / byte boundary

A C char* (or any data pointer) is emitted as an LPC buffer parameter, never string — LPC strings are UTF-8-native and the byte encoding must be explicit (string_encode/string_decode). With --string-convenience the generator also emits a <name>_s(string ...) overload that string_encodes to a NUL-terminated UTF-8 buffer for the common case, clearly named so the encoding is never hidden by default.

Supported C subset

Scalar types (charlong long, the intN_t/uintN_t family, size_t, float, double), any pointer (→ buffer), void returns, and plain struct { ... } bodies of those field types. Anything else — function-pointer parameters, unions, bitfields, varargs — is reported to stderr and skipped, never mis-bound. --emit-json dumps the parsed signature table (the shared contract for the bindings and tests).

Tests

python3 tools/ffi/test.py

Dependency-free; generates from test_sample.h and checks the emitted LPC/struct output and the JSON contract. The end-to-end path (generated bindings compiled and calling real native functions) is pinned by the driver testsuite: testsuite/single/tests/efuns/ffi_generated.lpc.