'<<'/'>>' with a shift count that's negative or >= 64 is UB for LPC's 64-bit int. Previously fixed by rejecting the count with a clean error(); switched instead to masking the count to its low 6 bits (`& 63`) before shifting, matching Java's `long` shift semantics (JLS 15.19). This keeps every LPC int a legal shift count with a deterministic result (e.g. `1 << 100` == `1 << 36`) rather than making an otherwise-ordinary script error out for shifting by a large or negative amount. Applied consistently across all three parallel sites so a folded and unfolded shift by the same count always agree: the runtime efuns (f_lsh/f_rsh/f_lsh_eq/f_rsh_eq, ops.cc), the compile-time constant folder (trees.cc's binary_int_op()), and the #if preprocessor evaluator (lexer_rules_pp.cc's ifexpr_binop()). Updated shift_overflow.lpc to assert the masked values instead of a caught error, and AGENTS.md's audit-checklist entry to describe masking rather than guard-and-error for shifts specifically (the INT_MIN/-1 divide/modulo guards are unchanged). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YTPLudxra3u1andNyWJEQG
59 KiB
FluffOS Agent Guide (AGENTS.md)
Welcome to FluffOS. This guide provides future AI coding agents with essential codebase information, architecture context, Package API standards, and Continuous Integration setup.
1. Codebase Architecture & Key Components
FluffOS is a high-performance LPMUD driver and game engine. Its codebase is structured as follows:
src/: Core driver and event dispatcher.mainlib.cc: Engine initialization.backend.cc: The gametick/event queues (event-loop-agnostic core);backend_libevent.ccis the native blocking loop,wasm/backend_wasm.ccthe host-driven WASM loop.comm.cc: Transport-agnostic connection handling (users, command queue, input_to, prompts, snoop). It talks to sockets only through theTransportinterface (net/transport.h).net/: The byte-transport layer --transport_libevent.cc(bufferevent/TLS/websocket transports + listening ports + socket reads),telnet.cc(protocol, portable),net_compat.h(platform type decls).user.cc: Connection session states.wasm/: WebAssembly target files (JS transport, exported entry points, crash-handler stub).
src/vm/: The execution VM and LPC bytecode interpreter.interpret.cc: The core bytecode interpreter loop (eval_instruction).simulate.cc: Object loading, cloning, destruction, and simulation event loops; alsorecompile_object()(in-place program update: recompiles a source file and swaps the program into the live master copy and all clones, carrying variables over by name -- see docs/concepts/general/hot_reload.md).vm.cc: VM startup and master object callbacks.
src/compiler/: The LPC parser, compiler, and code generator.grammar.y/grammar.autogen.cc: Bison rules compiling LPC scripts to bytecode.lexer.l/generate.cc: Lexer and VM instructions generator.
src/packages/: Modular packages implementing C++ external functions (efuns).packages/jsbridge/(WASM target only) is the LPC <-> page-JavaScript bridge:js_eval/js_calloutbound,js_export+ the page'sfluffos.callLPC()inbound; async deliveries run on the next tick and the pending tables hold/mark refs like the resolve() efun.testsuite/: Core LPC test cases, std library objects, and configurations (seetestsuite/README.md). LPC sources use the.lpcextension; the driver resolves an explicit.c/.lpcexactly and prefers.lpc(falling back to.c) for extension-less names — pinned bytestsuite/single/tests/efuns/dual_extension.lpc.
2. Package API & VM Stack Guidance
When creating or modifying packages (efuns) under src/packages/, you must follow these rules:
Unified Inclusion
- All package source files must include
"base/package_api.h"as their first header. Avoid importing VM, interpreter, or compiler headers directly.
Spec Files
- Package functions are declared in
[package_name].specfiles using LPC-like syntax, e.g.:int db_close(int);Themake_funcparser compiler compiles these specs into C++ glue code (efuns.autogen.cc).
VM Stack & Argument Unpacking
LPC arguments and return values are passed on the VM evaluation stack using svalue_t structures. The stack pointer is sp.
Navigating the Stack:
- The last argument is at
sp. The first argument is atsp - (num_args - 1). - For variable-argument functions, the number of arguments passed is available via the global
st_num_arg.
Accessing Arguments:
sp->u.numberor(sp - n)->u.number: Accessing integer values.sp->u.stringor(sp - n)->u.string: Accessing string values.sp->u.realor(sp - n)->u.real: Accessing float/real values.sp->u.arr: Accessing arrays (array_t *).sp->u.map: Accessing mappings (mapping_t *).sp->u.buf: Accessing buffers (buffer_t *).
Modifying the Stack & Returning Values:
- Use
pop_stack()orpop_n_elems(n)to clean up input arguments. - Push return values to the stack:
push_number(val)/push_real(val)push_malloced_string(str)/push_constant_string(str)put_malloced_string(str)/put_array(arr)/put_mapping(map)(these replace the top stack element directly).
Generic Argument Type-Checking (don't re-implement it)
- The VM's efun dispatch opcodes (
F_EFUN1/F_EFUN2/F_EFUN3for fixed arity,F_EFUNVwalkingefun_arg_etypes[]) callCHECK_TYPES()/bad_argument_multiple_types()before your efun's C function ever runs, checked against the type(s) declared for that argument in the.specfile. For any argument whose spec type is a scalar/reference type or a union of them (string,int,buffer,object,function,array,mapping, or an|of these), the top-levelsvalue_t.typetag is already validated -- a hand-writtenif (sp->type != T_STRING) error(...)duplicating that is dead code, not a fix, even though a mudlib call site used amixedvariable to dodge the compiler's staticstrict_typescheck. - This mechanism can only see the top-level type tag, so it cannot validate the ELEMENTS of a compound argument --
array_t/mapping_tcarry no per-element type info at the svalue level. A spec ofstring *namesgets zero element-type enforcement for free; per-element checks (each array item's.type) are a real, common gap (seematrix.cc's per-cell check,socket_write()'s T_ARRAY branch). When auditing a "missing type check" finding, test it empirically on the unfixed binary first (a probe LPC call with a bad-typed argument) -- if the generic dispatch already produces a cleanBad argument N to efun() Expected: ... Got: ...error, the finding is a false positive; only element-level or varargs-position checks are usually the real gap.
3. Memory Management & Reference Counting
LPC variables are dynamically typed and managed via the svalue_t structure. Memory safety is critical:
Reference Counting
- Compound types (
T_ARRAY,T_MAPPING,T_BUFFER,T_CLASS, and ref-countedT_STRINGstrings) are reference-counted. - If you copy a pointer to one of these types into another structure, you must increment its ref count (e.g.,
arr->ref++). - If you overwrite or destroy a reference, you must decrement its ref count or use
free_svalue(sval, tag)to clean it up safely.
Object Variable Block & Allocation Tags
- An object's global variables are a SEPARATE allocation from the
object_titself (ob->variables,TAG_OBJ_VARS, always >= 1 svalue, sized toprog->num_variables_total) -- this is what letsrecompile_object()swap a program with a different variable count onto a live object. Allocate viaallocate_object_variables();tot_alloc_object_sizeaccounting usessizeof(object_t) + (max(n,1)-1)*sizeof(svalue_t). - When introducing a new DMALLOC tag, wire it into the debug-build walkers in
packages/develop/checkmemory.cc(aDO_MARKat the owning object's mark site plus an orphan-report case), or every testsuite run on a Debug build will flag the blocks. Pick a freeTAG_PERMANENT + nslot inbase/internal/debugmalloc.h. destruct_object()SWEEPS the VM stack (remove_object_from_stack): any efun that runs arbitrary LPC (applies,__INIT) before cleaning up an object argument must free it withfree_svalue(sp, ...), neverfree_object(&sp->u.ob, ...)-- the slot may have become a plain number 0.
String Allocations
- Constant Strings (
STRING_CONSTANT): Not ref-counted or freed (point directly to literal read-only memory). - Malloced Strings (
STRING_MALLOC): Uniquely copied, allocated bystring_copy()/new_string()/int_new_string(). These allocations carry a block header before the char data, so thechar*you hold points into the middle of the allocation. Free them withFREE_MSTR(s)(=FREE(MSTR_BLOCK(s))), NEVER a plainFREE(s)on the char pointer -- that frees an interior pointer and ASan aborts with "attempted to free non-malloc'd pointer".free_string_svalue()/free_svalue()do this correctly; only reach forFREE_MSTRwhen you own a raw malloced string outside an svalue. - Shared Strings (
STRING_SHARED): Stored in a global string table, immutable, and ref-counted. Generated usingmake_shared_string(); released withfree_string().
The debug ref-count checker (checkmemory.cc) -- and off-graph / cross-thread refs
- On a Debug (
DEBUGMALLOC_EXTENSIONS) build the testsuite runscheck_memory()after every test file. It sweeps every live allocation, has each reference holder bump its target'sextra_ref, then flags any object/array/mapping/funptr/buffer/string whoseref != extra_refasBad ref count ... is <ref> - should be <extra_ref>. Read it as:is > should= a real reference nobody marked (or a genuine leak);is < should= you dropped a ref without decrementing (over-marking). It is a hard CI gate (gcc/clang Debug). - Any reference held OUTSIDE the normal object graph is invisible to the sweep -- a C++ container/global, a pending
call_out/async request, a cached dispatch slot. You MUST add a*_markfunction (invoked fromcheckmemory.cc's mark phase, e.g.async_mark_request) that bumpsextra_reffor the target and everything it transitively references -- a funptr also holds itshdr.ownerobject,hdr.argsarray, and program (mark_funp); an svalue holds its contents. Marking only the outer struct leaves the pointee under-counted. - If a worker thread touches the ref-holder, the mark function must take the SAME lock the worker uses, and must cover EVERY concurrent holder.
package/asyncoffloads I/O to worker threads and spawns a fresh one whenever its queue is momentarily empty, so more than one worker can run at once; a single "currently-processing" pointer read by the checker misses the others and yields a flakyBad ref count. Track all in-flight items in a lock-guarded container, not one pointer.
4. Error Handling & Stack Unwinding Safety
Caution
Error-Unwind Memory Leaks:
error()(src/vm/internal/simulate.cc) is a genuine C++throw, not a rawsetjmp/longjmp-- every path througherror_handler()/restore_context()ends in athrow(...), andsrc/CMakeLists.txt's own WASM-target comment confirms it: "The driver relies on C++ exceptions for LPC error handling." That distinction matters in practice: normal C++ stack unwinding runs, so RAII destructors DO fire reliably across anerror()unwind --f_ffi_struct_layout()'sstd::unique_ptr<array_t, ...>fix andf_ffi_callback()'s existing one both rely on this, and were verified to actually free on the unwind path. What does not run is any manually-written cleanup ("free this, then call error()") that sits on the normal-return path only --error()skips straight over it, same as it would with a raw longjmp.Prevention: Any raw heap pointer, held ref, or ad-hoc cleanup statement that appears after a call that might
error()(directly, or transitively through a nested helper) will never execute and will leak.Remedy: Always use RAII containers (
std::unique_ptr,std::shared_ptr) or C++ scope guards (DEFER/Neargye/scope_guard) to guarantee resource deallocation during unwinding -- they work here precisely because unwinding is real C++ exception unwinding, not because of any special-casing in the VM.
Three unwinding hazards that recur beyond raw heap pointers:
- An outstanding ref leaks like a raw pointer. A ref you took (
ob->ref++,fp->hdr.ref++,arr->ref++,add_ref(ob, ...)) is leaked iferror()fires before you store it somewhere that a later stage frees. Free it on the error path first. (Seen: an async callback funptr ref-bumped thenerror("permission denied")before hand-off; acall_out/input_to sentence + its object ref left dangling when a guardreturns early.) - Don't
error()/returnpast a half-initialized VM stack slot. If youSTACK_INC(orpush_*in pieces) and then bail before writing the slot, the eval stack holds an uninitializedsvalue_tthat a laterfree/sweep reads as garbage. Decide any early-out before theSTACK_INC. (Seen:input_to()to a#-prefixed apply.) - Reset module-global scratch state on the error path too. State kept in file-scope globals across a recursive operation and cleared only on the normal return will stay dirty when a nested
error()skips that cleanup, corrupting the next call. (Seen:restore_object'ssave_svalue_depth/sizessize pre-pass globals -- an OOM / mapping-too-largeerror()bypassedrestore_svalue's reset, so the next restore read stale sizes. Reset right before thoseerror()s.) - A deep call chain of C-style stack frames (not C++ objects) can't be cleaned up by
error()'s unwind at all. If a helper is called recursively many levels deep and each level frees its OWN temporary (a local buffer, anum_words---then-FREE_MSTRpattern) only on its normalreturn, anerror()/throwfired from deep inside skips every one of those pending frees on the way out -- unlike a ref held in aunique_ptr, there's no destructor to run for them. When a bounded search needs to bail out early for this reason, don't callerror()to escape it -- set a flag and have every recursive entry point check it andreturnimmediately instead, so the unwind is a normal chain of C++returns (running every frame's real cleanup) rather than a throw that jumps over all of them. From the LPC caller's point of view this is indistinguishable from "searched everything, found no match." (Seen:parser.cc'sparse_rule()step-budget abort, unwinding through however many pendingparse_recurse()frames were mid-backtrack.)
5. CMake & Compilation Constraints
- Header Probing Location: Do not move or reorder the
# System headersblock insrc/CMakeLists.txt. It must remain in place to preserve configure-time check ordering. - Header Guards: Do not modify
HAVE_*macro guards in C++ source files unless explicitly required. - Autogenerated Files:
grammar.autogen.cc/.his generated from Bisongrammar.y.efuns.autogen.cc/.his generated from spec files using themake_functool.applies_table.autogen.cc/.his generated using thebuild_appliestool.options.autogen.his generated using themake_options_defstool.
- Committed scanner/parser are version-pinned and hash-stamped. The committed
grammar.autogen.cc/.handlexer.autogen.ccpin the generator that produced them (the "made by GNU Bison X.Y.Z" banner /YY_FLEX_*_VERSIONdefines): a host with an OLDER bison/flex does not regenerate and builds from the committed copy; hosts at/above the pin regenerate normally (so CI still validatesgrammar.y/lexer.l). The post-build copy-back into the source tree is additionally gated on the sha256 of the INPUT, recorded in a trailing/* FluffOS generated-from <file> sha256=... */stamp — generator builds that emit cosmetically different code (distro-patched bison reporting the same version) can never churn the committed files; only a real.y/.ledit updates them. Do not hand-edit the stamp; do not commit locally-regenerated autogen files unless you actually changed the grammar/lexer. Machinery:cmake/util.cmake+ theFLUFFOS_PINNED_*/FLUFFOS_*_REGENlogic insrc/CMakeLists.txt. - WASM / Emscripten build: the driver cross-compiles to WebAssembly and runs a full mudlib in the browser (the page is the telnet client).
src/wasm/README.mdis the architecture doc (Transport interface, host-driven tick loop, per-target implementation files, package on/off matrix, roadmap);docs/build-wasm.mdis the user workflow (deps viatools/wasm/build-deps.sh, thenative-tools+wasmCMake presets, mudlib packaging viatools/wasm/pack-mudlib.sh). Per-connection byte transport is theTransportinterface (src/net/transport.h); per-target singletons (event loop, TLS, DNS resolver, crash handler) are selected at link time (*_libevent.ccvssrc/wasm/*.cc/*_stub.cc) -- do not add#ifdef __EMSCRIPTEN__to shared logic files. CI'swasmjob runs the LPC testsuite inside the wasm driver under node; testsuite files for optional packages must guard themselves with#ifdef __PACKAGE_*__.
6. Continuous Integration (CI) Environment
FluffOS uses GitHub Actions for CI on pull requests and pushes to master.
- Unified CI (
.github/workflows/ci.yml): All platforms and configurations are defined as matrix entries in a single workflow:- Ubuntu (
ubuntu-24.04): GCC and Clang, Debug and RelWithDebInfo builds. - Ubuntu + Sanitizers (
ubuntu-24.04): Clang with-DENABLE_SANITIZER=ON(Address & Undefined Behavior), Debug and RelWithDebInfo. - macOS (
macos-14, Apple Silicon): Default Clang, Debug and RelWithDebInfo. - Windows (
windows-latest): MSYS2/MinGW64 viamsys2/setup-msys2@v2, Debug and RelWithDebInfo. - WASM (
ubuntu-24.04, separatewasmjob): a pinned emsdk (.github/actions/build-wasm/action.yml'semsdk-verinput, deliberately not "latest" -- emsdk 6.0.2 defaultedGROWABLE_ARRAYBUFFERS=1, making everyALLOW_MEMORY_GROWTHbuild's heap a resizable ArrayBuffer in modern Chrome, which two unguardedHEAPU8.subarray()consumers then crash on:random_get()→crypto.getRandomValues()threw at boot andUTF8ToString()→TextDecoderbroke jsbridge. 6.0.3 reverted the default and fixed the string codegen, butrandom_get()is STILL unguarded upstream --src/www/wasm/index.htmlcarries a certifiedcrypto.getRandomValueswrapper for when the default returns; re-checklibwasi.jswhen bumping the pin) + cached wasm-built ICU (deps cache keyed on the resolvedemcc -dumpversion) → the two cross-build presets → the LPC testsuite running inside the wasm driver under node. The release workflow (release.yml) has a matchingbuild-wasmjob that shipsfluffos-<version>-wasm.zip(driver + web terminal +pack-mudlib.sh). - Flow (native platforms): Install dependencies → CMake configure → Build → GTest unit tests → LPC testsuite.
- Ubuntu (
- Docker CI (
.github/workflows/docker-publish.yml):- Builds a Docker image and pushes to
ghcr.ioon tagged releases and master merges.
- Builds a Docker image and pushes to
- Releases (
.github/workflows/release.yml):- Runs on a monthly schedule (09:00 UTC on the 1st), via manual
workflow_dispatch, by pushing av*tag, or by pushing arelease/vXbranch (the workflow creates tagvXat that commit and deletes the trigger branch -- the route for tooling that can push refs but lacksactions: write). Acheck-changesgate skips the scheduled run when master has no commits since the latest release tag; every explicit trigger always releases. Versions are date-based (vYYYY.MMDD.N, taken from the ref when one was pushed), release notes are auto-generated from merged PRs, and the build matrix ships Linux/Windows binaries plusfluffos-<version>-wasm.zip.
- Runs on a monthly schedule (09:00 UTC on the 1st), via manual
7. LPC Testsuite Conventions
- Efun Tests: Any new or modified external function must have matching LPC test scripts added to the
testsuite/directory undertestsuite/single/tests/efuns/. - Testing Targets:
driver-testsuite: Boots the local driver pointing to the test configuration.driver-fulltest/driver-autotest: Runs the LPC test suite and reports results before exiting.
- Harness facts that bite: LPC fixture files must live OUTSIDE
testsuite/single/tests/(use/clone/...or write them at runtime under/data/...) -- the runner executes everytests/**/*.lpcin RANDOMIZED order. Tests that register global state (e.g.master->set_compile_hooks()) must tear it down UNCONDITIONALLY: ASSERT macros record-and-continue, so wrap the body incatch(run_checks()), clean up, then re-error().master::flag()sits on the call stack for the entire run (so e.g. the master object can never be recompiled from inside a test; use acall_outthat fires post-run andshutdown(-1)s on failure -- call_outs only fire after the suite in FULL runs, never in single-file runs, which shut down synchronously). A plain-ftest:filter needs the FULL test path; suite runs dirtytestsuite/aw_test.txtandtestsuite/trace_test.json(restore before committing).file_name(ob)returns a leading slash. - The LPC suite is a real pass/fail gate, registered with ctest. the ctest test is named
testsuite—ctest -R testsuite(equivalently thedriver-autotestCMake target) runsdriver etc/config.test -ftest; CI runsctest -LE testsuite(GTest) andctest -L testsuite(this suite) as separate steps. The runner (testsuite/command/tests.lpc) prints gtest-style[ RUN ]/[ OK ]/[ FAILED ]blocks with per-file timing; failed checks are recorded and the run continues (one run reports every failure), then a recap lists the failed files and the driver exits nonzero. A clean run printsChecks succeeded.and exits 0 — that pair is the sound pass signal. Filter runs with-ftest:single/tests/efuns/foo.lpc(one file) or-ftest:efuns/dual*(glob). When touching the lexer/parser, run the suite 2–3× (it randomizes test file order) and prefer both a Debug ASan build and aRelWithDebInfobuild (some issues are release-only). - Regression tests for a memory-safety fix should fail (crash / ASan abort / leak / wrong output) on the UNFIXED binary -- verify that, not just that they pass now. Patterns that work from LPC: drive a compile-time bug by building source at runtime (
write_file("/gen.c", src); load_object("/gen")) -- e.g. a >4096-char local name to hit a compiler buffer overflow (single/tests/compiler/long_local_name.lpc); drive an efun bug by feeding the boundary argument (an out-of-range index, anINT_MINoperand, a crafted deep-nested save string) and asserting a cleancatch(...)error instead of a crash (efuns/sys_reload_tls.lpc,efuns/restore_variable.lpc). A flaky ref-count / threading bug (see §3) won't reproduce deterministically -- validate those with an ASan build + a repeated full-suite loop instead. A single flaky Debug failure at anasync_*test is almost always the ref-checker race, not your diff -- confirm the same suite passes on the other Debug configs before treating it as real.
8. LPC Language & Runtime Concepts for Agents
When writing or reviewing LPC code, consult docs/lpc/ first — it is the maintained language reference, kept in sync with driver behavior: source-file/extension resolution (docs/lpc/source-files.md), reading compiler diagnostics (docs/lpc/diagnostics.md), the full preprocessor reference (docs/lpc/preprocessor/ — define/include/conditionals/pragma), and the types/constructs pages. Testsuite conventions live in testsuite/README.md.
When editing compiler, VM, or package features, keep these structural mechanics in mind:
Mudlib vs Driver Separation
- The Driver (FluffOS): Written in C++, executes as the operating system, virtual machine, and compiler for LPC. It exposes built-in commands as "external functions" (efuns).
- The Mudlib: Written in LPC, contains the game logic, rooms, user logins, and rule definitions. It sits in a separate folder (e.g.
testsuite/or game folders) and is loaded by the driver.
Source File Extensions (.lpc / .c)
- LPC source files may use either
.lpc(preferred) or.c(legacy). Resolution, implemented inload_object()(src/vm/internal/simulate.cc) and pinned bytestsuite/single/tests/efuns/dual_extension.lpc:- An explicit extension is exact —
load_object("/foo.c")probes onlyfoo.c; no.lpclookup, and vice versa. On a miss the master'scompile_object()virtual hook gets the stripped name; if it declines, the load returns0. - Extension-less names prefer
.lpc, fall back to.c. - Object identity is extension-blind: object names carry no extension (
filename_to_obname()andotable.cc basename()strip either spelling), so any spelling of a loaded object's name finds it — the registry is consulted before the filesystem.prog->filenamekeeps the real extension of the compiled file.
- An explicit extension is exact —
- Do not append
".c"to names before loading (the old pattern); pass names through and letload_object()resolve. Suffix handling insave_object/restore_object,replace_program(),function_exists(), andchildren()must treat both spellings equivalently.
Pass-by-Reference (ref / &) and String-Char Lvalues
- The
refkeyword — and its sugar spelling&(both alternatives of the onerefnonterminal ingrammar.y) — appears in exactly three positions: parameter declarations (int ref x/int & x, viaarg_type), call-site arguments (f(ref x)/f(& x), viaexpr: ref lvalue, rejected outside an argument list byrule_expr_ref), andforeachloop variables. There is no ambiguity with binary&: ref is always prefix, bitwise AND always infix. - String-char lvalues are a SINGLE shared slot.
s[i]as an lvalue and aforeachref loop variable over a string both route through one file-scopeglobal_lvalue_codepoint(+ sentinelglobal_lvalue_codepoint_sv, typeT_LVALUE_CODEPOINT) ininterpret.cc. Consequences that must be preserved:- Loop-instance state lives on the VM stack, never in that global. A string
foreachkeeps its EGC cursor in the loop's own stack slot (theT_NUMBERslot under the loop variable) and re-arms the shared global per iteration (ref loops only). Nested string foreach used to segfault because the inner loop reset the shared iterator out from under the outer one — pinned bytests/operators/foreach.lpc. - Every operator that consumes
T_LVALUE_BYTE(buffer bytes) must also handleT_LVALUE_CODEPOINT(string chars) ors[i] <op>= xthrows "Bad Argument".++/--/=are handled in their opcodes;+=/-=go throughcodepoint_lvalue_add()(interpret.h). Writes rebuild the owner string (strings are immutable UTF-8; seeassign_lvalue_codepoint). s[i]lvalues and foreach ref loop chars share ONE arming path:aim_lvalue_codepoint()validates (out-of-bounds / multi-codepoint EGC error cleanly, same messages) and points the shared state at the target; don't hand-roll a second arming site. An EGC indexes as its FIRST codepoint when it fits in 4 bytes; wider EGCs (flag emoji, ZWJ sequences) are not char-lvalue-able -- pinned bytests/operators/ref.lpc.- Writes through a
foreachref loop variable over a STRING do not propagate to the iterated variable (the loop iterates a by-value stack copy) — behavior pinned bytests/operators/foreach.lpc("not suppose to change"). Arrays and mapping values DO mutate in place. - A string-char ref (
f(ref s[i])) is only valid until the next string-char lvalue is created anywhere; don't add code that holds one across arbitrary LPC execution. - Buffer bytes are the self-contained counterpart: every
T_LVALUE_BYTEconsumer readslval->u.lvalue_byte/lval->subtype(neverglobal_lvalue_bytedirectly -- the global is just the scratch instance thatb[i]arms). Buffers iterate inforeachlike arrays (each byte an int 0..255); a ref loop variable carries its OWN byte lvalue inref->sv, so nested buffer ref loops andb[i]in the body can't alias. Every byte write path (=,++,--,+=,-=) range-checks the result -- outside 0..255 errors ("Buffer byte value out of range"), never truncates. Strings (raw UTF-8 bytes) and arrays of ints 0..255 PROMOTE to buffers:do_promotions()wraps the rhs in ato_buffer()efun call (promote_to_buffer, same pattern as int<->float) forbuffer b = str,b += str, and initializers;+/+=/range-assignment also convert at RUNTIME (svalue_to_buffer_bytesin interpret.cc, shared with the efun) so untyped/mixedrhs works too. The compile-time type-check exemptions live inrule_expr_assign, the+type table, andgrammar_rules_decls.cc(buffer_promotable). All pinned bytests/operators/buffer_bytes.lpc. Aforeachref loop variable's staleT_REFis released on loop re-entry (free_svalueinF_FOREACH) -- reused inner-loop refs used to leak oneref_tper outer iteration.
- Loop-instance state lives on the VM stack, never in that global. A string
Simulated Efunctions (Simul_efuns)
- If an object makes a global function call (e.g.
foo()) that is not declared inside the object and is not a built-in driver efun, the compiler resolves it as a simulated efun call. - Simulated efuns are written in LPC and defined in a single file loaded at boot.
- Overriding built-in efuns is allowed in LPC by creating a simul_efun with the same name. To bypass a simul_efun override and invoke the driver efun directly, use the
efun::prefix (e.g.efun::move_object()).
Applies (Driver-to-LPC Callbacks)
- "Applies" are standard callbacks that the driver VM invokes on LPC objects during specific runtime events (e.g.,
createduring cloning,initwhen entering a room,clean_upduring sweep collections). - Applies are mapped using the
applies_table.autogen.cclookup tables. - Compile-time master applies
inherit_program(from, path, priv)andinclude_file(compiled, from, path)are consulted for every inherit statement / #include directive: string return redirects the path, array-of-strings return supplies the source text itself, any other return denies. They run MID-COMPILE (same precedent asvalid_override/get_include_path): implementations must never trigger another compile. Reference pages in docs/apply/master/; the dependency graph they expose powers the hot-reload daemon (testsuite/single/hot_reload.lpc).
Hot Reload (recompile_object())
recompile_object(master_copy)recompiles the source and swaps the program into the live master copy and every clone -- no destruct, identity preserved, variables carried by name (private included; new program's__INITruns first, then surviving names get old values). Works for the master object, the simul_efun object (their cached dispatch tablesmaster_applies/simulsare rebuilt against the new program BEFORE its__INITruns; simul indices are name-stable by design) and virtual objects (the backing program is what recompiles).- Invariants when touching this machinery: refuse while any live frame executes the old program (bytecode/variable indices are layout-relative); void pending
replace_program()entries at each target's swap point (cancel_pending_replace_program-- entries are computed against the program being replaced and old code can register one mid-update); function pointers that depend on the owner's layout (FP_LOCAL,FP_FUNCTIONAL) carry anowner_gensnapshot ofob->prog_generationand error cleanly when stale;FP_LOCALfunptrs store their creation program and accountfunc_refagainst IT (creation/destruction must stay symmetric -- decrementingowner->progcorrupts counts after a swap).
9. Documentation (docs/)
The FluffOS documentation site lives in the docs/ directory and is built with Docusaurus 3 (React-based).
Framework & Build
- Framework: Docusaurus 3 (
@docusaurus/preset-classic). Do not confuse with VitePress or Jekyll — the site was migrated from those frameworks and their leftover directories (.vitepress/,_layouts/,css/) have been removed. - Node: Use NVM. The active version is managed via
~/.nvm. Alwayssource "$NVM_DIR/nvm.sh"before running npm commands. - Key config files:
docs/docusaurus.config.ts— site config, navbar, footer, docs plugin pathdocs/sidebars.ts— hand-authored sidebar skeleton (DocusaurusSidebarsConfigformat, NOT VitePress format); importssidebars.generated.jsondocs/sidebars.generated.json+docs/sidebar_meta.json— generated sidebar trees and their curated metadata (see "Sidebar" below)docs/src/css/custom.css— Infima CSS variable overrides
- Build commands (run inside
docs/):npm run build # production build → docs/build/ npm run dev # dev server on 0.0.0.0 - Docs plugin config:
path: '.'androuteBasePath: '/'— markdown files live directly indocs/, not a subdirectory.
Markdown Compatibility
markdown.format: 'detect'is set indocusaurus.config.tsso.mdfiles use standard CommonMark (not MDX), while.mdxfiles get full MDX.- Even so, bare
{...}patterns in prose text (not inside fenced code blocks) are still parsed as JSX expressions and will cause SSG build failures. Escape them as\{...\}if they appear in non-code contexts. - Curated top-level pages use MDX (
docs/index.mdxlanding page with a card grid;docs/build.mdxwith per-platform<Tabs>); admonitions (:::note/:::tip/:::warning[Title]) work in both.mdand.mdx. In.mdx,<https://...>autolinks are NOT supported (parsed as JSX) — use[text](url).
Config Docs (auto-generated)
docs/driver/config.md is generated from src/base/internal/rc.cc. Do not edit it by hand. Regenerate with:
python3 docs/gen_config_docs.py
CI fails if the file is stale (.github/workflows/config-docs.yml).
Sidebar (partly generated)
The sidebar fully expands to every page. It has two layers:
docs/sidebars.ts— the hand-authored skeleton (Getting Started, thelpc/tree, Historical) which splices in the generated trees.docs/sidebars.generated.json— generated bydocs/gen_sidebar.pyfrom the reference doc trees (efun/,apply/,stdlib/,concepts/,driver/,cli/) plus the curated labels/descriptions/ordering indocs/sidebar_meta.json. Do not hand-edit the JSON.
After adding, removing, or moving any page in those trees, run:
python3 docs/gen_sidebar.py
CI fails if the file is stale (.github/workflows/docs-sidebar.yml).
Those trees have no index.md files (the old gen_index.py/update_index.sh machinery is gone): each category landing page is a Docusaurus generated-index card page whose title/description come from sidebar_meta.json. lpc/index.md is hand-written and stays.
Chinese docs (i18n)
The Chinese corpus is a proper Docusaurus locale, NOT a directory in the default docs tree: translations live under docs/i18n/zh-CN/docusaurus-plugin-content-docs/current/, mirroring the English tree layout (translation of docs/efun/arrays/allocate.md → .../current/efun/arrays/allocate.md). Served at /zh-CN/ via the navbar locale dropdown; untranslated pages fall back to English automatically. Both locales share one sidebar — Chinese category labels are translated in docs/i18n/zh-CN/docusaurus-plugin-content-docs/current.json (rescaffold new keys with npx docusaurus write-translations --locale zh-CN; generated sidebar items carry stable key fields — the directory/doc path — so translation keys survive label edits). npm run build builds both locales; the dev server takes --locale zh-CN for one at a time. Relative links in pages should be extension-less (../objects/clone_object, not clone_object.md) — .md-file links on an English page break the zh-CN build when the target has a translation.
Sidebar entry format notes:
- Links are doc IDs (relative file path without extension), not
.htmlURLs - Category with a hand-written landing page:
link: { type: 'doc', id: 'path/index' } - Category with an auto-generated card landing page:
link: { type: 'generated-index', title, description, slug }
10. Windows Environment Requirements (CRITICAL)
When working on a Windows host, FluffOS compiles and runs exclusively within the MSYS2 / MinGW64 environment.
Caution
MSYS2 Environment Constraint: Before proposing or executing any command (such as compilation, configuration, CMake, Make, Bison, or Git operations) on a Windows system, you must ensure that
MSYS2_ROOTis defined (e.g.,E:\msys64).All commands must be executed within that MSYS2 environment (by prepending MSYS2 MinGW64 binary directories to the environment
PATHor running through MSYS2 bash). Do NOT use standard Windows-installed binaries (like native Git, native CMake) or execute commands directly inside standard Windows Command Prompt or native PowerShell without MSYS2 paths active.If
MSYS2_ROOTis not configured, you must refuse to run any commands and prompt the user to configure the paths or launch viaopen-editor-msys2.bat(see README.md).
10. WSL Environment Requirements (CRITICAL)
WSL provides a native Linux environment. FluffOS runs natively under WSL distributions (Ubuntu, Alpine) using the standard Linux build toolchain — not MSYS2.
Caution
Mapped Directory Detection: If the workspace path starts with
/mnt/(inside WSL) or resides on a Windows drive letter path (e.g.E:\src\fluffos) being accessed via WSL, the codebase is on a mapped Windows filesystem. Cross-filesystem I/O across the Windows/Linux boundary is extremely slow.Agent action: If you detect the user is operating in a mapped directory, you must warn them and suggest they:
- Move or clone the repository into the WSL distribution's native Linux volume (e.g.
/home/user/fluffos).- Use
open-editor-wsl.bat(see README.md) to relaunch the editor from the native WSL path.- Access the native volume in Windows Explorer via
\\wsl.localhost\<distro>\home\user\fluffos.
11. Compiler Front-End (compiler/internal/)
Read src/compiler/internal/README.md first — it is the authoritative, maintained architecture document (module responsibilities, the data-flow diagram, header-ordering constraints). This section is the agent-facing summary of the load-bearing facts.
Single-scan, Flex-native lexer + preprocessor
- There is no separate preprocessor and no hand-rolled input buffer. Preprocessing is a set of Flex rule actions inside the driver's ONE scan over the source; every byte is read exactly once. The old standalone
preprocessor.ccengine and the oldoutp/ring-buffer are both gone. - Input rides on Flex's own buffer stack, and every input is an in-memory buffer. The main file is slurped and installed as the base buffer (
lpc_lex_set_source); macro expansions, pushbacks, and#includecontents are pushed as in-memory Flex buffers (lpc_lex_push_string_buffer), popped at their<<EOF>>. There is no byte-stream abstraction and noYY_INPUTrefill. Raw mid-rule reads (heredoc bodies, function-like macro-argument collection) go throughlpc_lex_getc()(a wrapper over the generatedyyinput()), which transparently pops a drained splice buffer into its parent. - Line/column tracking is native Flex state (
%option yylineno+ aYY_USER_ACTIONcolumn).current_lineis a macro over a reference to the innermost real buffer's per-buffer counter, so isolation across expansions and includes is automatic. - Macro expansion is rescan-driven:
lpc_lex_resolve_identifier()pushes the raw substituted body as a buffer and nested references expand when the rescan reaches them; self-reference termination is the set of live expansion-buffer frames.#if/#elifexpressions are evaluated over tokens pulled through the scanner (not a private character walk), so#define X 1+1+#if X*2is1+1*2with correct C precedence. Redefining a macro with a different body is a non-fatal warning, not an error.
The lexer.l purity rule (enforced)
lexer.lcontains ONLY Flex interactions — patterns, start-condition transitions (BEGIN), pushback (yyless), and the minimal trailer primitives that must dereference the generated scanner's private types (yyguts_t,yyinput(), the buffer-stack internals): buffer install/teardown,lpc_lex_getc(), the nested#if-expression pull, and the raw buffer accessors. ALL policy lives inlexer_rules.cc(token shaping),lexer_rules_pp.cc(preprocessing), orlexer_utils.cc(buffer/include bookkeeping, macro resolution, the tagged flex allocator, splice push/pop policy). When adding a rule, put anything that only needsyytext/yylval/yyget_extra()in alexer_rules*.cchelper; put anything expressible over the public reentrant API inlexer_utils.cc.
Grammar carries some lexical decisions (don't reintroduce lexer state for them)
- Array/mapping opens
({/([are ordinary'(' '{'/'(' '['token pairs the grammar pairs; the(: namefirst-class-function split is decided by LALR lookahead, not a start condition.grammar.y's%expectcounts the intentional (documented) conflicts — changing those productions means re-runningbison -Wcounterexamplesand updating the count. - Token inventory is deliberately minimal. Same-precedence operator families share one value-carrying token (opcode in
yylval:L_EQ_NE,L_SHIFT,L_INC_DEC,L_ORDER); single-char operators are plain char tokens ('!','.'). Add anL_*token only for a real new feature or a distinct grammar position.
Diagnostics
- Every
yyerror()/yywarn()captures a structuredDiagnostic(position + column, source snippet + caret,#includechain, macro-expansion chain, operand ranges via Bison%locations, fix-it hints) and reports it clang-style by default viareport_compile_diagnostic().lpcshellreadscompiler_diagsdirectly. - Never pass untrusted text as a printf format.
lexerror(s),yyerror,yywarn, and VMerror()are printf-style; source-derived text (#errorpayloads, macro/verb names, filenames) must be an ARGUMENT (yywarn("%s", s)), never the format string (a%in the text is a crash / CodeQLcpp/tainted-format-string).
The scratchpad arena (allocation during a compile)
- All transient compile strings live on the monotonic arena (
scratchpad.h): build them asScratchString/ScratchVector(allocation = pointer bump; individual deallocation = no-op;scratch_destroy()bulk-frees at compile end, soerror()unwinds leak nothing). Materialize a parser token withscratch_new_string()— the%union'sstringmember isScratchString *; SHARED strings on the value stack use the separateshared_stringmember (seegrammar.y'sfunctionproduction) so scratch-vs-shared lifetime is visible in the grammar. - Never put compile-surviving state on the arena: the macro table (
PpMacro/LpcMacroTable), predefines,Diagnosticrecords (lpcshell reads them after the arena resets), and program data staystd::string/heap — copy out at the boundary. Arena-backed members of objects that survive the compile (scanner-context accumulators) must be re-initialized per compile (lpc_lex_reset_context).
One compile state; staged outputs
g_compile(CompileState, compiler.h) is THE compiler state object: compile identity (filename, vm_context), the preprocessor state held DIRECTLY (g_compile.macros= user #defines only,g_compile.conds= the #if stack; predefines live in a shared version-cached table and never enter the per-compile table --start_new_file(..., keep_macros=true)retains #defines across REPL chunks with ZERO per-compile setup allocation), the diagnostics stream, and all one-shot diagnostic context. Legacy spellings (compiler_diags,compiler_vm_context,compiler_pending_*) are inline references into it -- do not add new loose compiler globals; add members.- Every compile stage has an output mode via
lpcc:-E(preprocessed source -- token-reconstructed, since the single-scan design has no textual pp artifact),--tokens(line:col kind spelling),--ast(parse trees via dump_tree before codegen),-O0(tree optimizer off -> dump_prog shows PRE-optimization bytecode), and the default full optimizeddump_progdisassembly. Pre-parse stages live incompiler/internal/stage_output.cc.
Utilities & testing
lexer_utils.cc/lexer_utils.halso own include-path management, path normalization, and the predefine registry (add_predefine,add_quoted_predefine,inc_open—std::string_viewinterfaces;inc_path/inc_listarestd::vector<std::string>, no manual GC marking).- Compiler front-end tests are
src/tests/test_compiler.cc(end-to-end through the real lexer) andsrc/tests/test_lexer.cc(token-level). When testing#undefof a predefined value, register the predefine (e.g.FLUFFOS) viaadd_predefinein setup — the full driver runtime is not initialized in unit tests. - The grammar ships as a machine contract, all under
tools/lpc-syntax/:grammar.ebnf(three layers: hand-authored Lexical + Preprocessor fromgrammar_lexical.ebnf.in, generated Syntax fromgrammar.y) andlpc-grammar.json(consumed by the JS tokenizer/highlighter/formatter and the VS Code extension intools/lpc-syntax/vscode/). Regenerate with thegenerate_ebnfCMake target whenevergrammar.yor the reserved-word/operator tables change — an uncategorized new token FAILS regeneration by design. JS tests:node tools/lpc-syntax/test.mjs. - CI regenerates the scanner and parser from
lexer.l/grammar.yon every platform (flex ≥ 2.6, bison ≥ 3.8 are installed in every CI environment), so alexer.l/grammar.ychange is validated by the toolchain, not just the checked-in*.autogen.*copies. Keep a clean regeneration warning-free.
12. Header Inclusion Guidelines
To maintain code health and consistency, FluffOS enforces the following global include rules:
- First Include ("base/std.h"): All code files in the driver (excluding files in
src/base/and package implementations insrc/packages/) must include"base/std.h"as their very first line, followed by a blank line to clearly separate it from other includes. - No Other Base Includes: Other headers inside
src/base/should not be included directly. Headers insidesrc/base/internal/must include necessary dependencies themselves. - Package Include ("base/package_api.h"): All package source files must include
"base/package_api.h"as their first header, and must not include"base/std.h"again.
13. Recurring Memory-Safety Bug Classes (audit checklist)
Repeated audits of the driver keep surfacing the same handful of defect shapes, almost always at the boundary where C++ handles mudlib-, database-, or network-supplied data. When writing or reviewing any efun / package / VM code that touches memory, check for each of these:
- Unbounded copy of variable-length data into a fixed buffer. LPC strings/arrays/buffers are arbitrary length -- use
SVALUE_STRLEN(sv),arr->size,buf->size(abuffer_t's payload isbuf->item, lengthbuf->size). Neverstrcpy/strcat/sprintf/memcpya mudlib/DB/network string into achar buf[N]without a bound; usesnprintfor a length-checkedmemcpy+ explicit NUL. Don't size a stack VLA from a config value (char buf[CONFIG_INT(...)]). (Codacy also flagsstrncpyas CWE-120; prefermemcpywith an explicit terminator.) - Integer overflow / truncation before an allocation,
memcpy, or index. LPC ints are 64-bit (LPC_INT); an efun argument stashed in anintoffset/length truncates or, if negative, becomes a hugesize_t. Range-check offsets/lengths in the non-truncating type, and clamplen * countagainst the limit before multiplying (post-multiply the product has already wrapped).-fwrapv(src/CMakeLists.txt) makes plain+/-/*/unary-negate overflow on signed ints DEFINED (silent two's-complement wraparound, not UB) -- LPC scripts can and do rely on this for+ - * += -= *=. This is consistent everywhere it's implemented: the runtime opcodes ininterpret.ccand the#ifevaluator inlexer_rules_pp.ccboth do plain wraparound arithmetic for these three operators; the compile-time constant folder intrees.ccdoesn't fold+/-/*at all (only| ^ & << >> %), so those always defer to the runtime path and there's no folded-vs-runtime discrepancy to check.- "Defined" does not mean "safe" for a size, offset, or count. A value that's allowed to wrap is still a memory-safety bug the instant it feeds a
DMALLOC/memcpy/array index -- a wrapped-negative length becomes a hugesize_t. Don't lean on wraparound to make a size computation safe; bound-check it explicitly before it can wrap (seef_uncompress()'s per-iterationlencap: checked every loop iteration, long beforelencould ever approach the type's range, not "after, and hope").
INT_MIN / -1/INT_MIN % -1, and an out-of-range shift count (negative, or>=the operand's bit width) are undefined behaviour that-fwrapvdoes NOT cover (it only defines overflow of+/-/*/unary-negate) -- both trip UBSan, and since the build sets-fno-sanitize-recover=allthat's a hard process abort, not a warning. Guard every signed divide/modulo of LPC ints explicitly (the-1divisor/dividend cases), erroring cleanly. For shift, don't error at all: mask the count to the low 6 bits (& 63) before shifting, matching Java'slongshift semantics (JLS 15.19) -- every LPC int becomes a legal shift count with a well-defined, deterministic result, instead of rejecting otherwise-ordinary scripts that happen to shift by a large or negative amount. Remember there are usually three parallel sites per operator that all need the identical treatment: the interpreter opcode / efun (F_DIVIDE/F_MOD,f_div_eq/f_mod_eq,f_lsh/f_rsh/f_lsh_eq/f_rsh_eq), the compile-time constant folder (trees.cc'sbinary_int_op()), and the#ifpreprocessor evaluator (lexer_rules_pp.cc'sifexpr_binop()) -- a folded and unfolded shift by the same count must agree.- Recursion with no depth cap on attacker-nested data. Deeply nested
({ ([ (:inrestore_objectdata, parser input, or compiled source overflows the C stack. Cap it -- the save AND restore paths shareMAX_SAVE_SVALUE_DEPTH; other callers of nested-array-walking helpers (e.g. the parser'senvironmentargument) need their own explicit depth parameter if they don't already have one. - Missing type / bounds checks on stack arguments. Spec-declared types cover only the first few fixed args;
varargs, index, and count args still need explicit validation (an array indexed by an LPC int; a matrix that must have 16 elements; a port index bounded by element count, notsizeof(array)which is the size in bytes). Note this is about element-level or varargs-position checks specifically -- see §2 "Generic Argument Type-Checking" for what the dispatcher already covers for free at the top level, so you don't file (or re-fix) a false positive. - Tainted format strings.
error(),debug_message(),debug(),yyerror/yywarn/lexerror, andtelnet_printfare printf-style. Any source-, mudlib-, DB-, or network-derived text (filenames, object paths, verbs, identifiers,#errorpayloads, DB error strings) must be a%sargument, never the format string -- a stray%is a crash / CodeQLcpp/tainted-format-string. Distinct but adjacent bug: a format specifier that doesn't match its argument's actual C++ type (e.g."%d"for anLPC_FLOAT/doublevararg) is also UB and prints garbage --error()has noformat(printf, ...)attribute, so the compiler won't catch this for you; make sure the value you pass is actually the type the specifier expects. - Leaks & dirty state on
error()paths -- see §4 (refs, half-filled VM stack slots, module-global scratch state, and the "deep C-style call chain" case where you must abort via plainreturns instead oferror()). - Off-graph & cross-thread references invisible to the debug ref-count checker -- see §3 (add a
*_markfunction; mark transitively; lock-guard and cover every concurrent worker). - Per-instance state parked in a shared global. The VM keeps a few singleton scratch slots (
global_lvalue_byte,global_lvalue_codepoint,global_lvalue_range); anything that can be active twice concurrently (a nested loop, re-entrant LPC via an apply, a second lvalue of the same kind) must keep its instance state on the VM stack and treat the global as per-use scratch, re-armed right before each use. Nested stringforeachsegfaulted for exactly this reason (inner loop reset the shared EGC iterator under the outer loop). Also: any operator handlingT_LVALUE_BYTEmust handleT_LVALUE_CODEPOINTtoo, or string-char compound assigns throw (see §8 "String-Char Lvalues"). - Signaling failure via an out-of-band sentinel value instead of a real out-parameter. Returning a special "impossible" value (e.g.
-INT_MAX+1) from a helper to mean "this failed," with the caller comparing the actual result against that sentinel, is unsound the moment the return type can legitimately produce that exact value -- a large-magnitude but perfectly correctLPC_FLOATresult silently collided with the sentinel and got misreported as an error. Use an explicit status out-parameter (bool*, or a smallintstatus/error code) so success can never be confused with a real computed value. (Seen:math.cc'snorm()/vector_op()/dotprod()/distance()/angle().) - A per-instance operation that mutates a shared/copied lvalue must update ALL of that value's state, not just the numeric payload.
svalue_tcarries side-channel state (subtype, e.g.T_UNDEFINED) alongsideu.number/u.real; a compound assign that doesargp->u.number OP= rhsbut leavesargp->subtypeuntouched leaves stale side-channel state on an otherwise-correctly-computed value (here: a variable that started "undefined" kept reportingundefinedp() == 1after a real arithmetic result was computed into it). When you touch an lvalue's payload, check whether its type also carries subtype/flag bits that need resetting to match the new, real value. - A driver-internal backtracking/recursive search with no step budget is invisible to the LPC eval-cost limiter.
max_eval_costonly meters LPC bytecode; a C++ search that tries every split/position combination for a crafted-but-legal input (e.g. a parser rule with several unbounded-span tokens against a long sentence) can run for a very long time before concluding "no match," completely ungoverned by the normal timeout. Add an explicit step counter with a generous-but-finite cap; see §4's "deep C-style call chain" entry for how to abort it without leaking.
Validate a fix on Debug + ASan/UBSan AND RelWithDebInfo (some bugs are release-only, some only trip the sanitizers), run the LPC suite 2–3× (randomized order), and add a regression under testsuite/single/tests/ that demonstrably fails on the unfixed binary (see §7).
14. Vendored Third-Party Dependencies (src/thirdparty/)
All third-party libraries are vendored as full source trees. Lessons from the 2026-07 full-fleet upgrade (argparse v3.2, scope_guard 0.9.4, backward-cpp v1.6, ghc/filesystem v1.5.14, nlohmann/json 3.12.0, fmt 12.2.0, utfcpp 4.1.1, libwebsockets v4.5.8):
Updating a dependency
- Vendor byte-exact from a
git cloneof the upstream release tag, then verify with a full-treediff -rqagainst the clone. Never reconstruct source through a lossy channel (an LLM fetch tool WILL silently truncate and even fabricate plausible C++ -- this was observed, and the resulting "fmt 12.2.0" compiled and passed every test while dynamic width/precision was silently non-functional). Passing tests do NOT prove a vendored tree is faithful. - Hunt for FluffOS-local patches FIRST:
git log --follow -- src/thirdparty/<dep>(unshallow the clone if needed) and diff the vendored tree against the pristine upstream tag it claims to be. Local patches must be re-applied on top of the new tag and disclosed in the commit message. If a local patch is obsolete upstream, say so in the commit message. - One dependency per commit, integration porting included in that dependency's commit. Commit messages disclose: version range, security-relevant fixes picked up, every local patch carried, and what was pruned.
- Prune policy: delete test/example/doc trees only after proving they are unreferenced -- read the vendor's own CMakeLists for guards. Known traps: lws
plugins/andlwsws/have UNCONDITIONALadd_subdirectory(internally guarded -- must stay); lwswin32port/is used by the WIN32 build; fmtsupport/cmake/is referenced unconditionally; deleting a dir whoseadd_subdirectoryis only transitively disabled leaves a landmine (force the guarding option OFF explicitly insrc/CMakeLists.txt). - Validate on BOTH gcc and clang locally before pushing: clang 18+/AppleClang hard-errors where gcc warns (
void main(void)in lws's configure probes silently failed EVERY header check under clang, includingLWS_HAVE_PTHREAD_H, breaking the whole build). Alpine/Docker are musl: glibc getsDl_infoetc. transitively, musl does not (backward-cpp needed an explicit<dlfcn.h>). Windows loader failures (0xc0000135= STATUS_DLL_NOT_FOUND) after a clean link mean a linked import library has no runtime DLL on CI -- never linkmsvcr90(d); those DLLs only exist where Visual Studio is installed (Windows builds here are always gcc/MinGW64, so_MSC_VER-guard MSVC-runtime-only calls instead). - Positional aggregate initializers of vendor structs are a trap across upgrades (lws inserted a bitfield mid-
lws_http_mount, silently shifting every later positional field). Use field-by-field init for vendor structs in FluffOS code.
Current FluffOS-local patches (keep across future updates unless obsolete upstream)
- backward-cpp
BackwardConfig.cmake: static-link setelf/dl/lzma/bz2/zstdalongsidedw(elfutils on modern distros), PUBLIC propagation through staticlibdriver, WIN32dbghelp/psapi(no msvcr!), cmake_minimum_required bump. - backward-cpp
backward.hpp:<dlfcn.h>under theBACKWARD_HAS_DWand backtrace branches (musl),_MSC_VERguard around_set_abort_behavior()(MinGW). - libwebsockets
CMakeLists.txt: configure probes spelledint main(void)(clang 18+; stillvoid mainupstream as of v4.5-stable).
libwebsockets integration facts (bitten once each)
- Since lws v4.3 an internal "system" vhost heads
context->vhost_list, solws_adopt_socket()(which adopts onto the list head) binds to a vhost with none of our ws protocols and every upgrade is rejected with "No supported protocol". Adopt explicitly:lws_adopt_socket_vhost(lws_get_vhost_by_name(context, "default"), fd)(src/net/websocket.cc). - New default-ON lws subsystems are forced OFF in
src/CMakeLists.txt-- see the commented block there (EVLIB_PLUGINSstatic evlib,HTTP_DIGEST_AUTH→GENCRYPTO→OpenSSL-3#error,LHP/UPNG/SECURE_STREAMS,MINIMAL_EXAMPLESpruned). Re-audit that option block on every lws bump; the implied-options file (CMakeLists-implied-options.txt) force-enables features transitively. - The test suites do not exercise a websocket client connection -- the vhost-adoption regression above passed the full GTest + LPC suites. After touching lws or
src/net/, run a real end-to-end smoke test: bootdriver etc/config.test, complete a 101 upgrade handshake on the plain and TLS ws ports with theascii/telnet/binarysubprotocols, and confirm the first data frame arrives — AND that a burst larger than one 2048-byte write window arrives completely (e.g.tuidemo chartsthroughsrc/www/index.html). - Never re-arm the writeable event from inside
LWS_CALLBACK_SERVER_WRITEABLE(with the libevent event lib): after the user callback returns, lws core clears POLLOUT and its pollfd bookkeeping desyncs from the evlib watcher — the request is dropped and every laterlws_callback_on_writable()no-ops, permanently freezing output on that connection. First bites on any burst larger than one window, which no test exercised until the browser e2e above. Thews_telnet.cc/ws_ascii.ccwriteable handlers instead drain the evbuffer in a loop gated onlws_send_pipe_choked(); a choked write is flushed by lws's own core-managed POLLOUT path, which fires the callback again.
Known state / deferred
crypt(musl 1.1.24 subset) andutf8_decoder_dfa(Hoehrmann) are intentionally frozen -- confirmed unchanged upstream through musl 1.2.6 / no upstream versioning.- libtelnet, libevent, widecharwidth: upgrades deliberately deferred (maintainer request, 2026-07); their unused test/doc trees are already pruned.