fluffos/testsuite/single/master.lpc
Yucong Sun 82d82f3beb
Add recompile_object() efun: in-place program update, state preserved (#1237)
* Add recompile_object() efun: in-place program update, state preserved

Recompiles a master copy's program from its source file and swaps the
fresh program into the LIVE master copy and every clone sharing it -
the "hot update" alternative to destruct+load_object: nothing is
destructed, so object identity (pointers held elsewhere, name,
inventory, shadows, interactive state, call_outs, heart_beat) is
untouched, and each object's global variables carry over BY NAME
inside the driver (private ones included): the new program's __INIT
runs first, then every surviving name gets its old value back. The
recompile behaves like a normal load - unloaded parents resolve
through the retry dance and the compile-time master applies are
consulted. Returns the number of objects updated.

Made possible by moving an object's variable block OUT of the object_t
allocation into its own (TAG_OBJ_VARS, always >= 1 svalue, wired into
the debug-malloc walkers): every access already went through
ob->variables[i], so a program with a different variable count can now
be swapped onto a live object.

Safety: refused while any object sharing the program is executing
anywhere on the call stack (live frames hold bytecode positions and
variable indices of the old layout), for clones (pass the master
copy), the simul_efun object, pending replace_program(), and nested
calls. Function pointers whose behavior depends on the owner's program
layout (FP_LOCAL, FP_FUNCTIONAL) go stale instead of corrupting:
objects carry a prog_generation stamp, funptrs snapshot it at
creation/bind, and call_function_pointer() errors cleanly on mismatch.

Fixing a latent asymmetry this exposed: make_lfun_funp incremented
func_ref on the creation-time program but dealloc_funp decremented the
owner's CURRENT program. FP_LOCAL pointers now store their program and
account against it symmetrically (checkmemory and %O formatting
updated to match) - caught by the debug-build memory checker in the
testsuite.

The hot-reload daemon's default (state-keeping) path now reloads
through recompile_object() - changed ancestors first, then the watched
program - so clones ride along automatically; a cooperative
hot_reload_state()/hot_reload_restore() pair takes the destruct+load
path with exactly the state it chooses, and watch(prog, 0) opts out
entirely. The daemon test demonstrates finding all live instances with
children()/clonep() and both clone behaviors (updated in place vs.
stragglers on the old program); single/tests/efuns/recompile_object.lpc
pins the efun semantics (master+clones count, per-object state incl.
private, initializers for new variables, removed variables, stale
funptrs, executing/clone/missing-source guards, call_out survival).

Docs: efun reference page, hot-reload guide step 5 rewritten around
the efun with the value-transfer technique kept as the manual
alternative, caveats updated (clone behavior per path, stale funptrs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* recompile_object: support master/simul_efun targets; review fixes

The master object and the simul_efun object can now be recompiled
live. Both subsystems dispatch through cached name->runtime-index
tables (master_applies / simuls) whose entries point into the old
program's function table, so recompile_object() rebuilds them against
the new program immediately after the swap and BEFORE the new
program's __INIT runs (an error inside it would already route through
those tables). Simul_efun indices are preserved by NAME across the
rebuild - that table is deliberately unsorted for exactly this reason
- so simul calls compiled into every other program keep working, and
a simul removed by the new source fails with the usual "no longer a
simul_efun" runtime error. set_master()/set_simul_efun() only
ref/assign when the object actually changes, keeping the classic
destruct-driven replacement path intact. %O of a function pointer to
a since-removed simul now prints a placeholder instead of derefing
the null table entry.

Testsuite: the efun test recompiles the live simul_efun object
mid-run (the very next ASSERT dispatches through the rebuilt table),
pins the currently-executing guard on the master (master::flag() sits
on the call stack for the whole run), and re-runs the master recompile
from a post-run call_out where the master is idle - state carry-over
and apply dispatch are enforced by exiting nonzero.

Also from this self-review round (multi-agent):

* f_recompile_object crashed when the target destructed itself from
  its new program's __INIT: destruct sweeps the VM stack, so the efun
  glue's stack slot held a plain 0 by the time it tried to
  free_object() it. Reproduced by a review agent's probe; the glue
  now uses free_svalue(), and the scenario is pinned in the efun test
  (destructed targets drop out of the updated count).

* hot_reload daemon: ancestors() now returns the inherit closure
  DEEPEST-first - recompiling a middle parent bakes in whatever
  grandparent program is live at that moment, so a >=3-level chain
  with two changed ancestors permanently embedded the stale
  grandparent (reproduced by a review agent; pinned by a new
  kid/mid/grand scenario).

* hot_reload daemon: dep records were map_delete'd before the
  recompile and rebuilt by the applies during it - but a throw BEFORE
  compiling (currently-executing guard, unreadable file) left the
  object loaded with no records, blinding closure_changed() to
  include-file edits forever. Records are now restored when the
  recompile throws (pinned by a new watched-object-drives-the-pass
  scenario).

* docs: inheritance wording ("copies code" -> the child links against
  the exact parent program it was compiled with), the
  currently-executing guard also covers inheritors running inherited
  code, and the cooperative-pair opt-out triggers on
  hot_reload_state() alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* recompile_object: void mid-update replace_program; cover virtuals

Two additions from the C++ review round:

* A replace_program() registered DURING the update slipped past the
  pre-flight check: an earlier target's __INIT can call into a
  not-yet-swapped clone, whose OLD code registers a pending entry -
  computed against the very program the update is replacing. The
  backend sweep then ran that entry's variable-offset shuffle against
  the fresh program's differently-sized variable block (negative
  num_fewer, heap corruption; reproduced under ASan by a review
  agent's probe). recompile_object() now voids any pending entry for
  each target at its swap point - an entry registered AFTER the swap
  is computed against the new program and survives. Pinned in the
  efun test; the rest of the suite run doubles as the sweep detector.

* Virtual objects (materialized through master::compile_object) are
  covered and pinned: the virtual object carries the BACKING file's
  program, so the recompile targets that source and swaps it in with
  the virtual name, identity, flag and state untouched. The testsuite
  master gains a /data/hu/virt* fixture mapping; docs note the
  behavior and that the hot-reload daemon keys its records by
  compiled program name (watch virtuals via their backing file).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* docs: capture hot-reload/recompile_object knowledge in README and AGENTS

README: the hot-reload language bullet now describes what actually
ships (recompile_object with state carried by name, clones included),
and Features gains a Hot Reload section linking the guide.

AGENTS.md, for future agents working on this machinery: the object
variable block is a separate allocation (TAG_OBJ_VARS) and what that
enables; the new-DMALLOC-tag checklist (checkmemory walkers); the
destruct-sweeps-the-VM-stack rule for efun glue; testsuite harness
facts (fixtures outside tests/, unconditional teardown, master::flag
on the stack all run + the post-run call_out pattern, full -ftest
paths, suite side-effect files); the compile-time master applies; and
the recompile_object invariants (executing-frame guard, dispatch-table
rebuild before __INIT, voiding mid-update replace_program entries,
funptr generation staleness, FP_LOCAL func_ref symmetry).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* recompile_object: pin shadow/catch_tell/add_action/heart_beat survival

New test recompile_object2.lpc verifies the object-attached runtime
state that dispatches by name keeps working across the swap:
catch_tell routes into the new program while accumulated state stays;
a shadow chain survives updating the SHADOWED object (still
intercepted, new code underneath) and updating the SHADOW itself while
attached; add_action sentences registered by the old code still fire
their verb into the new program; the heart_beat registration persists.

Also two doc wording fixes from the docs review: the executing-guard
bullet now covers both halves of the guard (frames executing the
program's code AND frames belonging to an object of the program
running inherited code), and the guide's mode summary matches the
daemon (hot_reload_state alone selects the cooperative path).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* recompile_object: fix simul_efun/__INIT edge cases; cover callback surface

Four defects from the C++ review round, each probe-verified under ASan:

* Recompiling the simul_efun object to a program that defines no simuls
  FREEd the live dispatch table (simul_names/simuls) while other
  compiled programs still carry F_SIMUL_EFUN opcodes and FP_SIMUL
  funptrs with baked indices -> use-after-free on the next simul call.
  Keep the tombstoned arrays instead (remove_simuls() already nulls
  every func, which yields the clean "no longer a simul_efun" error and
  preserves the name->index mapping for re-adds).

* The debug memory checker did not mark IHE_ORPHAN idents as permanent,
  so any run that removed a simul via an update tripped a spurious
  "orphan permanent identifier" leak and failed the testsuite gate.
  Add IHE_ORPHAN to the mark mask (it is part of IHE_PERMANENT).

* The disassembler dereferenced simuls[].func unguarded in two places;
  after a simul removal, dump_prog() on a program referencing it would
  null-deref. Guard both, matching the sprintf %O fix.

* An error() thrown from a target's __INIT during the swap leaked this
  loop's held references (the per-target snapshot ref, new_prog's
  compile ref, the old variable block) and left the update half-applied.
  Wrap call___INIT per target in save_context/try/restore: on error the
  object is left committed to the new program with fresh initializers
  (carried-over state dropped, like a create() that throws during load),
  sibling targets still update, and nothing leaks.

Test coverage:
* recompile_object.lpc: an __INIT that errors -- blueprint and clone
  both recompile, neither is immortalized, the object stays usable on
  the new program, no ref/variable leak (the per-file memory checker is
  the detector). (The simul zero-function / removal paths can't be
  exercised against the shared /single/simul_efun mid-suite; verified
  out-of-band with a throwaway ASan probe that reduces then restores the
  file.)
* recompile_object2.lpc: call_outs armed before the swap fire after it --
  a name-based call_out dispatches into the new program, a funptr
  call_out is stale and is refused cleanly (its target never runs, no
  crash), verified from a post-run call_out. Rounds out the by-name
  callback survivors already covered (catch_tell, add_action, heart_beat,
  shadows).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-11 11:24:30 -04:00

421 lines
11 KiB
Text

// file: /daemon/master.c
#include <globals.h>
// /inherit/master/valid.c contains all the valid_* functions
inherit "/inherit/master/valid";
nosave int has_error = 0;
nosave string last_error = "";
// Test-run statistics (gtest-style harness, see /command/tests.lpc):
// every assertion macro passes through clear_last_error(), so it doubles
// as the check counter; failed checks are RECORDED (the run continues to
// the next file) and the runner reports them all at the end.
nosave int num_checks = 0;
nosave string *test_failures = ({ });
public string clear_last_error() {
num_checks++;
last_error = "";
}
public void record_failure(string msg) {
test_failures += ({ msg });
}
public int query_num_checks() { return num_checks; }
public string *query_failures() { return test_failures; }
// Remove and return the most recently recorded failure (the harness
// self-test /single/tests/std/harness.lpc uses this to probe the
// recording machinery without failing the run).
public string pop_failure() {
string last;
if (!sizeof(test_failures)) {
return 0;
}
last = test_failures[<1];
test_failures = test_failures[0..<2];
return last;
}
public void reset_test_stats() {
num_checks = 0;
test_failures = ({ });
}
// find stack right before __assert
private mapping* trace_to_last_assert() {
mapping *trace = dump_trace();
for (int i = 0; i < sizeof(trace); i++) {
if (trace[i]["function"][0..7] == "__assert") {
return trace[0..i];
}
}
return trace;
}
public string get_last_error() {
if (last_error == "") {
return sprintf("%O", trace_to_last_assert());
}
return last_error;
}
void flag(string str) {
mixed error;
string cmd, arg;
if(sscanf(str, "%(test|speed):%s", cmd, arg) != 2)
cmd = str;
switch (cmd) {
case "test":
error = catch("/command/tests"->main(arg));
if(error) {
has_error = 1;
write(error);
}
// Backstop: recorded check failures fail the run even if the
// runner's own recap/shutdown path was skipped somehow.
if (sizeof(test_failures)) {
has_error = 1;
}
break;
case "speed":
error = catch("/command/speed"->main(arg));
if(error) {
has_error = 1;
write(error);
}
shutdown(0);
break;
default:
write("The only supported flag is 'test' and 'speed', got '" + str + "'.\n");
break;
}
if (has_error) { shutdown(-1); }
// otherwise wait for auto shutdown
}
void catch_tell(string str) {
has_error = 1;
}
object connect()
{
object login_ob;
mixed err;
err = catch(login_ob = new(LOGIN_OB));
if (err) {
write("It looks like someone is working on the player object.\n");
write(err);
destruct(this_object());
}
return login_ob;
}
// compile_object: This is used for loading MudOS "virtual" objects.
// It should return the object the mudlib wishes to associate with the
// filename named by 'file'. It should return 0 if no object is to be
// associated.
mixed compile_object(string file)
{
write("MASTER: compile_object is called, file : " + file + "\n");
if (file=="/test/virtual") {
return load_object("/single/void");
}
// Virtual-object fixture for the recompile_object test: any
// /data/hu/virt* name materializes from the runtime-written
// /data/hu/vbase source.
if (file[0..12] == "/data/hu/virt" && file_size("/data/hu/vbase.lpc") > 0) {
return load_object("/data/hu/vbase");
}
return 0;
}
// This is called when there is a driver segmentation fault or a bus error,
// etc. As it's static it can't be called by anything but the driver (and
// master).
staticf void crash(string, object, object)
{
foreach (object ob in users())
tell_object(ob, "Master object shouts: Damn!\nMaster object tells you: The game is crashing.\n");
#if 0
log_file("crashes", MUD_NAME + " crashed on: " + ctime(time()) +
", error: " + error + "\n");
if (command_giver) {
log_file("crashes", "this_player: " + file_name(command_giver) + "\n");
}
if (current_object) {
log_file("crashes", "this_object: " + file_name(current_object) + "\n");
}
#endif
}
// Function name: update_file
// Description: reads in a file, ignoring lines that begin with '#'
// Arguements: file: a string that shows what file to read in.
// Return: Array of nonblank lines that don't begin with '#'
// Note: must be declared static (else a security hole)
staticf string *update_file(string file)
{
string *arr;
string str;
int i;
str = read_file(file);
if (!str) {
return ({});
}
arr = explode(str, "\n");
for (i = 0; i < sizeof(arr); i++) {
if (arr[i][0] == '#') {
arr[i] = 0;
}
}
return arr;
}
// Function name: epilog
// Return: List of files to preload
string* epilog(int)
{
string *items;
items = update_file(CONFIG_DIR + "/preload");
return items;
}
// preload an object
void preload(string file)
{
int t1;
string err;
if (file_size(file + ".lpc") == -1 && file_size(file + ".c") == -1)
return;
t1 = time();
write("Preloading : " + file + "...");
err = catch(call_other(file, "??"));
if (err != 0) {
write("\nError " + err + " when loading " + file + "\n");
} else {
t1 = time() - t1;
write("(" + t1/60 + "." + t1 % 60 + ")\n");
}
}
// Write an error message into a log file. The error occured in the object
// 'file', giving the error message 'message'.
void log_error(string, string message)
{
write_file(LOG_DIR + "/compile", message);
}
// save_ed_setup and restore_ed_setup are called by the ed to maintain
// individual options settings. These functions are located in the master
// object so that the local admins can decide what strategy they want to use.
int save_ed_setup(object who, int code)
{
string file;
if (!intp(code)) {
return 0;
}
#ifdef __PACKAGE_UIDS__
file = user_path(getuid(who)) + ".edrc";
#else
file = "/.edrc";
#endif
rm(file);
return write_file(file, code + "");
}
// Retrieve the ed setup. No meaning to defend this file read from
// unauthorized access.
int retrieve_ed_setup(object who)
{
string file;
int code;
#ifdef __PACKAGE_UIDS__
file = user_path(getuid(who)) + ".edrc";
#else
file = "/.edrc";
#endif
if (file_size(file) <= 0) {
return 0;
}
sscanf(read_file(file), "%d", code);
return code;
}
// When an object is destructed, this function is called with every
// item in that room. We get the chance to save users from being destructed.
void destruct_environment_of(object ob)
{
if (!interactive(ob)) {
return;
}
tell_object(ob, "The object containing you was dested.\n");
ob->move(VOID_OB);
}
// make_path_absolute: This is called by the driver to resolve path names in ed.
string make_path_absolute(string file)
{
file = resolve_path((string)this_player()->query_cwd(), file);
return file;
}
string get_root_uid()
{
return ROOT_UID;
}
string get_bb_uid()
{
return BACKBONE_UID;
}
string creator_file(string str)
{
return (string)call_other(SINGLE_DIR + "/simul_efun", "creator_file", str);
}
string domain_file(string str)
{
return (string)call_other(SINGLE_DIR + "/simul_efun", "domain_file", str);
}
string author_file(string str)
{
return (string)call_other(SINGLE_DIR + "/simul_efun", "author_file", str);
}
string privs_file(string f) {
return f;
}
staticf void error_handler(mapping map, int flag) {
object ob;
string str;
/* Squelch the expected eval_cost errors thrown by the call_out tests */
if (map["program"] == "/single/tests/efuns/call_out.lpc" &&
map["error"] == "*Too long evaluation. Execution aborted.\n")
{
return;
}
ob = this_interactive() || this_player();
if (flag) str = "*Error caught\n";
else str = "";
str += sprintf("Error: %s\nCurrent object: %O\nCurrent program: %s\nFile: %O Line: %d\n%O\n",
map["error"], (map["object"] || "No current object"),
(map["program"] || "No current program"),
map["file"], map["line"],
implode(map_array(map["trace"],
(: sprintf("Line: %O File: %O Object: %O Program: %O", $1["line"], $1["file"], $1["object"] || "No object", $1["program"] ||
"No program") :)), "\n"));
last_error = str;
write_file("/log/log", str);
if (!flag && ob) tell_object(ob, str);
}
// ---------------------------------------------------------------------------
// Compile-time applies: inherit_program / include_file.
//
// The driver consults these for every `inherit "path";` statement and
// every #include directive. Returning the path unchanged keeps the
// default behavior; a different string redirects the inherit/include to
// that path; an array of strings supplies the inherited program's /
// included file's source text itself; any other value denies it.
//
// The testsuite master delegates to a registered hook object so
// individual tests (and the /single/hot_reload daemon) can observe or
// rewrite compiles while they run; with no hook registered the applies
// return the path unchanged, i.e. stock behavior. NOTE: these run in
// the middle of a compile -- a hook must never trigger another compile
// (no load_object/clone_object of unloaded files).
// ---------------------------------------------------------------------------
private object compile_hooks;
public void set_compile_hooks(object ob) { compile_hooks = ob; }
public object query_compile_hooks() { return compile_hooks; }
mixed inherit_program(string from, string path, int priv) {
if (compile_hooks) return compile_hooks->inherit_program(from, path, priv);
return path;
}
mixed include_file(string compiled, string from, string path) {
if (compile_hooks) return compile_hooks->include_file(compiled, from, path);
return path;
}
mixed get_include_path(string file)
{
switch(file)
{
case "/clone/mgip1":
case "/clone/mgip1.c":
case "/clone/mgip1.lpc":
return ({ "/include/m_gip1", "/include" });
case "/clone/mgip2":
case "/clone/mgip2.c":
case "/clone/mgip2.lpc":
return ({ "/include/m_gip2", "/include" });
case "/clone/mgip3":
case "/clone/mgip3.c":
case "/clone/mgip3.lpc":
return ({ "/include", "/include/m_gip1" });
case "/clone/mgip4":
case "/clone/mgip4.c":
case "/clone/mgip4.lpc":
return ({}); // should yield error message
default:
return ({ ":DEFAULT:" });;
}
}
int valid_database(object ob, string action, mixed *info) {
write("MASTER valid_database called: " + sprintf("ob:%O action:%O info:%O", ob, action, info) + "\n");
// Approve!
return 1;
}
// Security gate for package_ffi (op: "load"/"symbol"/"prepare"/
// "callback"). A missing apply denies by default; the testsuite allows
// everything so the ffi_* tests can exercise real native calls. A
// production master would restrict this tightly (it is a sandbox escape).
int valid_ffi(string op, mixed arg, object caller) {
// Deny a sentinel so the denial path is testable
// (single/tests/efuns/ffi_load.lpc).
if (stringp(arg) && strsrch(arg, "DENY_ME") != -1) {
return 0;
}
return 1;
}
string object_name(object ob) {
return ob->name();
}