mirror of
https://github.com/fluffos/fluffos
synced 2026-08-12 18:26:06 -04:00
reference loops: docs, runtime cycle efuns, orphan collector, copy() unwind fix (#1276)
A complete treatment of reference loops (cyclic data structures) in LPC:
the reference-counting VM has no cycle collector, so a value that reaches
itself leaks permanently -- and silently -- once the last outside
reference is dropped, and it cannot be saved, deep-copied, or fully
printed in the meantime. This change documents the problem, fixes a
driver memory-safety bug it exposes, and adds runtime tooling to detect,
locate, break, and (on debug builds) collect such loops.
Driver fixes:
- copy() on a cyclic structure always hits the MAX_SAVE_SVALUE_DEPTH
error(), and that unwind path leaked every partially-built container --
allocated with the _empty_ (uninitialized-svalue) allocators and holding
a borrowed, un-ref-counted pointer to the ORIGINAL value, which the
Debug memory checker then read after free (heap-use-after-free abort
under ASan, reproducible with just catch(copy(a)) on a[0] = a).
deep_copy_* now allocate zero-filled, hold the destination in a
unique_ptr whose deleter is the tag-symmetric free_*, and only write the
destination slot after the child copy fully succeeded (ffi.cc precedent,
AGENTS.md section 4).
- save_object/save_variable and copy()'s 'nested too deep' errors -- the
classic symptom of a loop -- now say so (the has_cycle() pointer is
gated on PACKAGE_CONTRIB so core never recommends an efun the build
lacks).
New efuns (contrib package, src/packages/contrib/cycles.cc):
- has_cycle(mixed): 1 if the value's reference graph contains a loop.
- find_cycles(mixed): one index-path string per loop-closing slot
("[3][\"peer\"].1" style).
- break_cycles(mixed): clears every loop in place and returns the number
of edges broken. Exactly the DFS back-edges are touched (a digraph is
acyclic iff its DFS has no back-edges): item/value slots are zeroed, a
loop closed in mapping-KEY position has its node deleted (hashed keys
cannot be overwritten), and a loop closing on the funptr->args edge
itself -- possible because bind() SHARES the args array between the old
and new funptr -- detaches the bound funptr's args list and replaces it
with a zero-filled one of the same size. DAG sharing is never touched;
one cut un-loops a whole ring; afterwards the value saves, copies,
prints, and frees normally.
All three share one ITERATIVE walk (explicit heap stack, white/grey/
black coloring): no C-stack recursion, no depth cap -- arbitrarily deep
acyclic values scan cleanly where save_variable() errors. Edges:
array/class items, mapping keys AND values, fp->hdr.args; objects are
deliberately leaves (loops through object variables are the
destruct()-managed kind: destruct2() zeroes the variable block).
break_cycles() records fixes during a mutation-free walk and applies
them in a post-pass that holds a reference on every touched container,
zeroes slots before deleting nodes (only node deletion can cascade
frees), and releases the holds last -- order-independent and safe
against shared/overlapping fixes.
Orphaned-loop collector (develop package, Debug/DEBUGMALLOC_EXTENSIONS):
- find_orphaned_cycles(int collect): finds -- and with any nonzero
argument reclaims -- data blocks that are unreachable because only a
reference loop keeps them alive: the case nothing LPC-level can reach
anymore. Detection is trial deletion (CPython-gc-style), implemented in
md_scan_orphaned_cycles (checkmemory.cc): count each array/class/
mapping/funptr's references held by OTHER data blocks; a block whose
real ref count exceeds that is externally held (object variables, VM
stack, call_out, any C++-side holder) and seeds liveness, which
propagates along data edges; the remainder is loop garbage. No root
enumeration to get wrong -- every legitimate holder shows up as an
external ref. Collection: hold a ref on every dead block, sever all
their child slots (releasing strings/objects/buffers/live values
normally), then release the holds -- each dead block deallocates with
nothing left to cascade into.
- check_all_blocks() runs the same scan (skippable via new flag bit 2,
value 4) and reports 'unreachable data block(s) kept alive only by
reference loop(s)', so the testsuite's per-file check_memory() gate
turns a dropped cycle into a hard, attributed failure. That immediately
caught a real pre-existing leak: tests/std/json.lpc's
test_encode_circular_references() dropped all four of its
deliberately-cyclic fixtures on every suite run since it was written.
Tests (testsuite/single/tests/):
- operators/reference_loop.lpc pins the driver contract around loops and
crashes the unfixed Debug/ASan driver (the copy() unwind UAF).
- efuns/has_cycle.lpc, find_cycles.lpc, break_cycles.lpc cover self/
mutual/ring loops across arrays, mappings (value and key position),
classes, funptr args (including the bind()-shared-args case, which was
unbreakable in an earlier revision of this change), DAG-sharing
preservation, save/copy working again after a break, idempotency, and a
5000-deep acyclic walk.
- efuns/find_orphaned_cycles.lpc pins baseline-relative detection of 6
orphans across three dropped loop shapes, idempotent detection, that
reachable loops are never classified as garbage, and that collection
reclaims everything while reachable data survives.
- Every cycle-building test has UNCONDITIONAL teardown (body in catch(),
find_orphaned_cycles(1) regardless, error re-raised) so a mid-test
regression stays one [ FAILED ] entry instead of cascading the
harness's LEAK gate into a suite-wide abort (AGENTS.md section 7).
Docs (Docusaurus, sidebar regenerated; full two-locale build verified):
- new concepts page docs/concepts/general/reference_loops.md: why loops
leak, what each recursive consumer does, the destruct() exception,
prevention patterns, the runtime tools, and the debug-build collector;
- efun pages for all four new efuns; check_memory.md documents the new
scan and flag bit.
Validated on Debug+ASan/UBSan (full LPC suite, randomized order, multiple
runs) and RelWithDebInfo (full suite), 313 GTest unit tests, plus an
8-angle adversarially-verified self-review.
Round-2 self-review (4 fresh angles, adversarially verified) additionally:
- break_cycles() post-pass releases its held container references via an
RAII guard: allocate_array() there can error() (set_config() can shrink
__MAX_ARRAY_SIZE__ at runtime below a shared args array's size), and the
old trailing release loop would have leaked every held ref on that
unwind (AGENTS.md section 4).
- documented the pre-existing map_delete()-class caveat: deleting a
key-closed loop's node while an outer unlocked foreach-ref variable is
aimed at it dangles that variable (not specific to this efun; noted in
code and doc).
- extended orphan-collector coverage from 6 to 10 blocks: class rings
(TAG_CLASS candidate/sever/free_class paths), mapping pairs closed in
KEY position (the collector's in-place key-zeroing sever path), and a
buffer payload riding an orphaned ring (sever must release it or the
Debug ref gate trips); added a destructed-object-in-walked-value test
(render_key + leaf handling).
- docs: refs.md and copy.md now link back to the cycle tooling; zh-CN
sidebar translation keys rescaffolded; AGENTS.md section 7 documents the
new hard gate and the catch + find_orphaned_cycles(1) teardown pattern.
- re-entrancy audit (foreach/MAP_LOCKED/locked_map_nodes/merge_arg_lists)
and LPC-test-semantics audit returned no code defects.
Claude-Session: https://claude.ai/code/session_01VaksxbPjc3hjzghoQPUHRo
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Yucong Sun
GitHub
parent
b040aae3c9
commit
86d13cdb83
27 changed files with 1860 additions and 48 deletions
|
|
@ -166,7 +166,7 @@ FluffOS uses GitHub Actions for CI on pull requests and pushes to `master`.
|
|||
* **Testing Targets**:
|
||||
- `driver-testsuite`: Boots the local driver pointing to the test configuration.
|
||||
- `driver-fulltest` / `driver-autotest`: Runs the LPC test suite and reports results before exiting.
|
||||
* **Harness facts that bite:** LPC fixture files must live OUTSIDE `testsuite/single/tests/` (use `/clone/...` or write them at runtime under `/data/...`) -- the runner executes every `tests/**/*.lpc` in RANDOMIZED order. Tests that register global state (e.g. `master->set_compile_hooks()`) must tear it down UNCONDITIONALLY: ASSERT macros record-and-continue, so wrap the body in `catch(run_checks())`, clean up, then re-`error()`. `master::flag()` sits on the call stack for the entire run (so e.g. the master object can never be recompiled from inside a test; use a `call_out` that fires post-run and `shutdown(-1)`s on failure -- call_outs only fire after the suite in FULL runs, never in single-file runs, which shut down synchronously). A plain `-ftest:` filter needs the FULL test path; suite runs dirty `testsuite/aw_test.txt` and `testsuite/trace_test.json` (restore before committing). `file_name(ob)` returns a leading slash.
|
||||
* **Harness facts that bite:** LPC fixture files must live OUTSIDE `testsuite/single/tests/` (use `/clone/...` or write them at runtime under `/data/...`) -- the runner executes every `tests/**/*.lpc` in RANDOMIZED order. A test that builds a **reference loop** (cyclic array/mapping/class/funptr-args structure) must break it before returning -- on Debug builds the post-file `check_memory()` hard-fails on dropped loops (`unreachable data block(s) kept alive only by reference loop(s)`); the robust teardown is `catch(run_checks())`, then an unconditional `#if efun_defined(find_orphaned_cycles) find_orphaned_cycles(1); #endif`, then re-`error()` (see `tests/efuns/break_cycles.lpc` and docs/concepts/general/reference_loops.md). Tests that register global state (e.g. `master->set_compile_hooks()`) must tear it down UNCONDITIONALLY: ASSERT macros record-and-continue, so wrap the body in `catch(run_checks())`, clean up, then re-`error()`. `master::flag()` sits on the call stack for the entire run (so e.g. the master object can never be recompiled from inside a test; use a `call_out` that fires post-run and `shutdown(-1)`s on failure -- call_outs only fire after the suite in FULL runs, never in single-file runs, which shut down synchronously). A plain `-ftest:` filter needs the FULL test path; suite runs dirty `testsuite/aw_test.txt` and `testsuite/trace_test.json` (restore before committing). `file_name(ob)` returns a leading slash.
|
||||
* **The LPC suite is a real pass/fail gate, registered with ctest.** the ctest test is named `testsuite` — `ctest -R testsuite` (equivalently the `driver-autotest` CMake target) runs `driver etc/config.test -ftest`; CI runs `ctest -LE testsuite` (GTest) and `ctest -L testsuite` (this suite) as separate steps. The runner (`testsuite/command/tests.lpc`) prints gtest-style `[ RUN ]/[ OK ]/[ FAILED ]` blocks with per-file timing; **failed checks are recorded and the run continues** (one run reports every failure), then a recap lists the failed files and the driver exits nonzero. A clean run prints `Checks succeeded.` and exits 0 — that pair is the sound pass signal. Filter runs with `-ftest:single/tests/efuns/foo.lpc` (one file) or `-ftest:efuns/dual*` (glob). When touching the lexer/parser, run the suite 2–3× (it randomizes test file order) and prefer both a Debug ASan build and a `RelWithDebInfo` build (some issues are release-only).
|
||||
* **Regression tests for a memory-safety fix should fail (crash / ASan abort / leak / wrong output) on the UNFIXED binary** -- verify that, not just that they pass now. Patterns that work from LPC: drive a **compile-time** bug by building source at runtime (`write_file("/gen.c", src); load_object("/gen")`) -- e.g. a >4096-char local name to hit a compiler buffer overflow (`single/tests/compiler/long_local_name.lpc`); drive an **efun** bug by feeding the boundary argument (an out-of-range index, an `INT_MIN` operand, a crafted deep-nested save string) and asserting a clean `catch(...)` error instead of a crash (`efuns/sys_reload_tls.lpc`, `efuns/restore_variable.lpc`). A flaky ref-count / threading bug (see §3) won't reproduce deterministically -- validate those with an ASan build + a repeated full-suite loop instead. **A single flaky Debug failure at an `async_*` test is almost always the ref-checker race, not your diff** -- confirm the same suite passes on the other Debug configs before treating it as real.
|
||||
|
||||
|
|
|
|||
240
docs/concepts/general/reference_loops.md
Normal file
240
docs/concepts/general/reference_loops.md
Normal file
|
|
@ -0,0 +1,240 @@
|
|||
---
|
||||
title: general / reference_loops
|
||||
---
|
||||
# Reference loops (cyclic data structures)
|
||||
|
||||
LPC's compound values — arrays, mappings, classes, buffers, and function
|
||||
pointers — are assigned **by reference**: `b = a` makes `b` point at the same
|
||||
array as `a`, it does not copy it. Nothing stops one of those references from
|
||||
pointing back at (or eventually reaching) its own container, and the result is
|
||||
a **reference loop**:
|
||||
|
||||
```c
|
||||
mixed *a = ({ 0 });
|
||||
a[0] = a; // a self-referential array
|
||||
a[0][0][0] == a; // true -- you can walk the loop forever
|
||||
|
||||
mapping m1 = ([]), m2 = ([]);
|
||||
m1["peer"] = m2; // a two-value loop
|
||||
m2["peer"] = m1;
|
||||
|
||||
class node { mixed next; }
|
||||
class node n1 = new(class node), n2 = new(class node);
|
||||
n1.next = n2; // a circular linked list
|
||||
n2.next = n1;
|
||||
```
|
||||
|
||||
All of this is **legal**. The loop behaves like any other data structure as
|
||||
long as you only take one step at a time: indexing, `foreach`, `==` (which is
|
||||
pointer identity for compound types), `sizeof`, `member_array`, and every
|
||||
other shallow operation work normally. What a loop changes is *memory
|
||||
management* and the behavior of the handful of operations that walk a value
|
||||
*recursively*.
|
||||
|
||||
## Why loops leak: reference counting has no cycle collector
|
||||
|
||||
The driver reclaims compound values by **pure reference counting**
|
||||
(`src/vm/internal/base/svalue.cc`): every svalue holding a pointer to an
|
||||
array/mapping/class counts as one reference, and the value is deallocated the
|
||||
moment its count reaches zero. There is **no mark-and-sweep pass, no cycle
|
||||
collector** — nothing ever asks "is this value still *reachable*?".
|
||||
|
||||
A reference loop keeps itself alive:
|
||||
|
||||
```c
|
||||
mixed *a = ({ 0 });
|
||||
a[0] = a; // the array's ref count is now 2: the variable + itself
|
||||
a = 0; // ref count drops to 1 -- and can never reach 0 again
|
||||
```
|
||||
|
||||
After `a = 0` the array is unreachable from any LPC variable, but its own
|
||||
back-reference holds its count at 1 forever. The memory is leaked until the
|
||||
driver shuts down. The same applies to a loop of any length: drop the last
|
||||
outside reference and the whole ring stays allocated, each member kept alive
|
||||
by the next.
|
||||
|
||||
Two properties make this leak unusually quiet:
|
||||
|
||||
* **The ref-count comparison can't see it.** A detached loop is perfectly
|
||||
self-consistent — every internal reference is accounted for by another
|
||||
member of the loop — so the classic "is X, should be Y" check in
|
||||
`check_memory()` passes. (On debug builds `check_memory()` now runs a
|
||||
*separate* trial-deletion scan for exactly this case — see
|
||||
[`find_orphaned_cycles()`](#after-the-fact-find_orphaned_cycles-debug-builds)
|
||||
below.)
|
||||
* **Leak detectors may not see it either.** On Debug builds the driver's own
|
||||
allocation tracker keeps a pointer to every block, so tools like
|
||||
LeakSanitizer consider the memory "still reachable".
|
||||
|
||||
On release builds the only visible symptom is the slow growth of the
|
||||
`Arrays` / `Mappings` counters in `mud_status(1)` (or `memory_summary()`)
|
||||
across a long uptime.
|
||||
|
||||
## What the recursive consumers do with a loop
|
||||
|
||||
Any operation that walks a value's contents recursively would run forever on
|
||||
a loop. The driver guards each of them with a depth cap instead of cycle
|
||||
detection, so what you get is a *clean, catchable error* (or truncation), not
|
||||
a crash:
|
||||
|
||||
| Operation | Behavior on a loop |
|
||||
| --- | --- |
|
||||
| `sprintf("%O", x)` | truncates with `...` at nesting depth 20 |
|
||||
| `save_object()` / `save_variable()` | throws `Mappings and/or arrays nested too deep (100) for save_object -- possibly a reference loop; see has_cycle()` |
|
||||
| `copy()` (deep copy) | throws `Mappings, arrays and/or classes nested too deep (100) for copy() -- possibly a reference loop; see has_cycle()` |
|
||||
| `restore_object()` / `restore_variable()` | not applicable — the save format cannot express a loop, so restore can never create one |
|
||||
| `==`, `foreach`, indexing, `sizeof`, ... | shallow; work normally |
|
||||
|
||||
These caps share `MAX_SAVE_SVALUE_DEPTH` (100, `options_internal.h`); the
|
||||
practical consequence is that **a value containing a loop cannot be saved,
|
||||
deep-copied, or fully printed**. If your object's `save_object()` suddenly
|
||||
errors with "nested too deep", a reference loop that crept into a saved
|
||||
variable is the usual culprit.
|
||||
|
||||
All of the above is pinned by the regression test
|
||||
`testsuite/single/tests/operators/reference_loop.lpc`.
|
||||
|
||||
## The one loop the driver *does* break: through an object
|
||||
|
||||
Objects are the exception, because they have an explicit lifetime. When an
|
||||
object is destructed, the driver **zeroes the object's global variables on
|
||||
the spot** (`destruct2()` in `src/vm/internal/simulate.cc`) — precisely so
|
||||
that "an object with a variable pointing to itself would never be freed"
|
||||
cannot happen. Every reference the object's variables held is released at
|
||||
destruct time, regardless of the object's own ref count.
|
||||
|
||||
So a loop is harmless **if an object is part of the ring**:
|
||||
|
||||
```c
|
||||
object ob = clone_object("/obj/container");
|
||||
mapping m = ([ "owner" : ob ]);
|
||||
ob->set_data(m); // ob.data -> m -> ob : a loop
|
||||
|
||||
destruct(ob); // driver clears ob's variables:
|
||||
// m's ref from ob is gone, m["owner"] reads 0
|
||||
```
|
||||
|
||||
`destruct()` cuts the ring, and plain reference counting reclaims the rest.
|
||||
|
||||
## How to prevent reference loops (and their leaks)
|
||||
|
||||
1. **Prefer trees over graphs in plain data.** If a child needs to find its
|
||||
parent, consider storing a lookup key (an object name, an index into a
|
||||
central mapping) instead of a direct back-reference.
|
||||
|
||||
2. **Route unavoidable back-references through an object.** Store the cyclic
|
||||
state in a (possibly invisible) daemon or data object's variables. The
|
||||
structure stays cyclic and convenient, but `destruct()` — including the
|
||||
driver's normal `clean_up`/swap-driven destruction — reliably reclaims
|
||||
it. This is the idiomatic LPC ownership pattern: the *object* owns the
|
||||
graph; the graph does not own itself.
|
||||
|
||||
3. **Break the loop before dropping the last reference.** If you build
|
||||
transient cyclic structures, null the back-edge when you are done:
|
||||
|
||||
```c
|
||||
a[0] = 0; // self-referential array: cut the self-edge
|
||||
map_delete(m1, "peer"); // mutual mappings: one deleted edge un-loops both
|
||||
n2.next = 0; // circular list: cut any one link
|
||||
```
|
||||
|
||||
One cut edge anywhere in the ring is enough — reference counting reclaims
|
||||
everything downstream of the cut.
|
||||
|
||||
4. **Never let a loop reach saved variables.** `save_object()` will error and
|
||||
your object's persistence silently stops working. Mark cache-like
|
||||
variables that might contain shared/cyclic data `nosave`.
|
||||
|
||||
5. **Audit with `refs()`.** The `refs(value)` efun (develop package) returns
|
||||
the value's reference count. A freshly built structure held by one
|
||||
variable reports 1; if it reports more and you can't account for the
|
||||
extras, something (possibly itself) is holding it:
|
||||
|
||||
```c
|
||||
mixed *a = ({ 0 });
|
||||
refs(a); // 1 -- just the variable
|
||||
a[0] = a;
|
||||
refs(a); // 2 -- the variable, plus the loop's own reference
|
||||
```
|
||||
|
||||
## Runtime cycle tools: `has_cycle()`, `find_cycles()`, `break_cycles()`
|
||||
|
||||
The contrib package ships three efuns built for exactly this problem. All
|
||||
three share one **iterative** graph walk (no C-stack recursion), so unlike
|
||||
`save_variable()`/`copy()` they have no nesting-depth limit, and they follow
|
||||
every kind of edge a loop can hide in: array/class items, mapping keys *and*
|
||||
values, and a function pointer's captured argument list.
|
||||
|
||||
```c
|
||||
mixed *a = ({ "x", 0 });
|
||||
a[1] = a;
|
||||
|
||||
has_cycle(a); // 1 -- there is a loop
|
||||
find_cycles(a); // ({ "[1]" }) -- one path per loop-closing slot
|
||||
break_cycles(a); // 1 -- cleared it, in place
|
||||
a[1]; // 0; a[0] is still "x", and refs(a) is back to 1
|
||||
```
|
||||
|
||||
* **`has_cycle(value)`** — cheap predicate. Ideal as a guard before
|
||||
`save_object()`, in a daemon's periodic self-check, or as an assertion in
|
||||
test code.
|
||||
* **`find_cycles(value)`** — one index path per back-edge
|
||||
(`"[3][\"peer\"].1"` style), pointing at the exact slot that closes each
|
||||
loop. Diagnostics for *where* the loop crept in.
|
||||
* **`break_cycles(value)`** — the safe-clearing primitive. It cuts exactly
|
||||
the loop-closing edges: item/value slots are zeroed in place, and a loop
|
||||
closed in mapping-*key* position has its node deleted (a hashed key cannot
|
||||
be overwritten). Deliberate sharing (DAG edges) is never touched, one cut
|
||||
un-loops a whole ring, and afterwards the value saves, copies, prints, and
|
||||
— crucially — *frees* normally. Call it before discarding any structure
|
||||
that might have become cyclic.
|
||||
|
||||
Because a broken loop is precisely the "break the back-edge" pattern from
|
||||
rule 3, `break_cycles()` is what you reach for when the structure was built
|
||||
by code you don't control (restored data run through mudlib mutators, a
|
||||
generic cache, user-scripted content).
|
||||
|
||||
## After the fact: `find_orphaned_cycles()` (debug builds)
|
||||
|
||||
The three efuns above need a variable that still reaches the loop. Once the
|
||||
last outside reference is dropped, the loop is unreachable — nothing in LPC
|
||||
can name it anymore. On debugging builds (`DEBUGMALLOC_EXTENSIONS`), the
|
||||
develop package adds the missing piece:
|
||||
|
||||
```c
|
||||
mixed *a = ({ 0 });
|
||||
a[0] = a;
|
||||
a = 0; // leaked -- invisibly, until now
|
||||
|
||||
find_orphaned_cycles(0); // 1 -- detected after the fact
|
||||
find_orphaned_cycles(1); // 1 -- detected AND reclaimed
|
||||
find_orphaned_cycles(0); // 0
|
||||
```
|
||||
|
||||
Detection is by **trial deletion** (the same idea CPython's garbage
|
||||
collector uses): a data block whose every reference comes from other data
|
||||
blocks, with no path from any externally held block — an object's
|
||||
variables, the VM stack, a `call_out`, a driver-internal holder — can only
|
||||
be loop garbage. Because the verdict is computed from real reference
|
||||
counts, reachable data is never misclassified. With the collect flag the
|
||||
garbage is reclaimed safely: values still reachable elsewhere are
|
||||
untouched, and everything the loop held (strings, buffers, object refs) is
|
||||
released normally.
|
||||
|
||||
`check_memory()` runs the same scan and reports
|
||||
`unreachable data block(s) kept alive only by reference loop(s)`, so on
|
||||
debug builds — including the driver's own testsuite, which calls
|
||||
`check_memory()` after every test file — **dropping a cycle is now a hard,
|
||||
attributed failure** instead of a silent leak.
|
||||
|
||||
Since `save_object()`, `save_variable()` and `copy()` failing with "nested
|
||||
too deep" is the classic *symptom* of a loop, those errors now say so:
|
||||
`... nested too deep (100) for save_object -- possibly a reference loop;
|
||||
see has_cycle()`.
|
||||
|
||||
For driver developers: `copy()`'s depth-cap error path must stay
|
||||
unwind-safe — the deep-copy helpers hold their half-built containers in
|
||||
RAII guards (`src/packages/contrib/contrib.cc`) because a cyclic argument
|
||||
*always* takes that error path. Any new efun that walks svalues recursively
|
||||
needs its own depth cap (audit checklist item — see `AGENTS.md` §13.4) and
|
||||
the same unwind discipline.
|
||||
75
docs/efun/contrib/break_cycles.md
Normal file
75
docs/efun/contrib/break_cycles.md
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
---
|
||||
title: contrib / break_cycles
|
||||
---
|
||||
# break_cycles
|
||||
|
||||
### NAME
|
||||
|
||||
break_cycles() - safely clear every reference loop in a value
|
||||
|
||||
### SYNOPSIS
|
||||
|
||||
int break_cycles(mixed value);
|
||||
|
||||
### DESCRIPTION
|
||||
|
||||
Clears every reference loop in `value` IN PLACE and returns the number
|
||||
of edges that were broken. Because compound values are passed by
|
||||
reference, the caller's value (and every other holder of it) sees the
|
||||
change.
|
||||
|
||||
Exactly the loop-closing back-edges are touched, nothing else:
|
||||
|
||||
- an array/class item or mapping VALUE that closes a loop is
|
||||
overwritten with 0;
|
||||
- a mapping KEY that closes a loop cannot be overwritten (the node is
|
||||
hashed by that key), so the whole key/value node is deleted, exactly
|
||||
as if map_delete() had been called;
|
||||
- a slot inside a function pointer's captured argument list that
|
||||
closes a loop is overwritten with 0 (the function pointer itself
|
||||
survives, its captured argument becomes 0); when the loop closes on
|
||||
the argument-list edge itself -- possible because bind() shares the
|
||||
argument list between the old and the new function pointer -- the
|
||||
bound function pointer's whole argument list is detached and
|
||||
replaced with a zero-filled one of the same size;
|
||||
- everything that is not part of a loop -- including deliberate
|
||||
sharing of one structure from several places -- is left untouched.
|
||||
|
||||
One broken edge un-loops an entire ring, so a mutual pair or a ring of
|
||||
N containers counts as 1, not N.
|
||||
|
||||
Afterwards has_cycle(value) is 0 and the value can be saved with
|
||||
save_object()/save_variable(), deep-copied with copy(), and printed
|
||||
with sprintf("%O") without hitting the nesting-depth errors that a
|
||||
loop otherwise causes -- and dropping the last reference actually
|
||||
frees the memory instead of leaking it.
|
||||
|
||||
Call it before discarding any structure that might have become
|
||||
cyclic, e.g. from a generic cache daemon's clean-up path:
|
||||
|
||||
void flush() {
|
||||
break_cycles(cache);
|
||||
cache = ([]);
|
||||
}
|
||||
|
||||
The traversal is iterative and has no nesting-depth limit.
|
||||
|
||||
Caveat (shared with map_delete()): if a loop is closed in mapping-KEY
|
||||
position, the node deletion carries the same restriction as deleting
|
||||
a mapping entry from inside a `foreach (key, ref value in m)` loop
|
||||
over that same mapping -- do not call break_cycles(m) from inside
|
||||
such a loop while the ref variable is aimed at the entry being
|
||||
removed.
|
||||
|
||||
### EXAMPLE
|
||||
|
||||
mixed *a = ({ "keep", 0 });
|
||||
a[1] = a; // loop
|
||||
break_cycles(a); // 1
|
||||
a[0]; // "keep" -- untouched
|
||||
a[1]; // 0 -- back-edge cleared
|
||||
save_variable(a); // works again
|
||||
|
||||
### SEE ALSO
|
||||
|
||||
has_cycle(3), find_cycles(3), refs(3), map_delete(3)
|
||||
|
|
@ -25,3 +25,14 @@ title: contrib / copy.pre
|
|||
|
||||
This is particularly useful when you wish to have data that is passed
|
||||
by reference, but do not want to alter the original.
|
||||
|
||||
### ERRORS
|
||||
|
||||
Values nested deeper than 100 levels throw "Mappings, arrays and/or
|
||||
classes nested too deep (100) for copy() -- possibly a reference loop;
|
||||
see has_cycle()". A value containing a reference loop always exceeds
|
||||
the cap; test with has_cycle(3) and clear with break_cycles(3).
|
||||
|
||||
### SEE ALSO
|
||||
|
||||
has_cycle(3), break_cycles(3), save_object(3), restore_object(3)
|
||||
|
|
|
|||
50
docs/efun/contrib/find_cycles.md
Normal file
50
docs/efun/contrib/find_cycles.md
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
---
|
||||
title: contrib / find_cycles
|
||||
---
|
||||
# find_cycles
|
||||
|
||||
### NAME
|
||||
|
||||
find_cycles() - locate every reference loop in a value
|
||||
|
||||
### SYNOPSIS
|
||||
|
||||
string *find_cycles(mixed value);
|
||||
|
||||
### DESCRIPTION
|
||||
|
||||
Returns one index path per back-edge -- per slot that closes a
|
||||
reference loop in `value`. An empty array means the value is acyclic.
|
||||
Breaking (or deleting) exactly the returned slots would make the value
|
||||
loop-free; break_cycles() does that in one call.
|
||||
|
||||
Path syntax, concatenated from the outermost container inward:
|
||||
|
||||
[3] array item 3
|
||||
.2 class field 2 (declaration order)
|
||||
["name"] mapping value under key "name"
|
||||
[key <map>] a mapping KEY (the key itself continues the path)
|
||||
(args) a function pointer's captured argument list
|
||||
|
||||
Which slot of a loop is reported depends on traversal order (array
|
||||
index order, mapping table order), so treat the paths as diagnostics,
|
||||
not as a stable contract. Long string keys are truncated in the
|
||||
rendering.
|
||||
|
||||
The traversal is iterative and has no nesting-depth limit.
|
||||
|
||||
### EXAMPLE
|
||||
|
||||
mixed *a = ({ "x", 0 });
|
||||
a[1] = a;
|
||||
find_cycles(a); // ({ "[1]" })
|
||||
|
||||
mapping m = ([]);
|
||||
m["self"] = m;
|
||||
find_cycles(m); // ({ "[\"self\"]" })
|
||||
|
||||
find_cycles(({ 1, ({ 2 }) })); // ({ }) -- acyclic
|
||||
|
||||
### SEE ALSO
|
||||
|
||||
has_cycle(3), break_cycles(3), refs(3)
|
||||
46
docs/efun/contrib/has_cycle.md
Normal file
46
docs/efun/contrib/has_cycle.md
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
title: contrib / has_cycle
|
||||
---
|
||||
# has_cycle
|
||||
|
||||
### NAME
|
||||
|
||||
has_cycle() - test whether a value contains a reference loop
|
||||
|
||||
### SYNOPSIS
|
||||
|
||||
int has_cycle(mixed value);
|
||||
|
||||
### DESCRIPTION
|
||||
|
||||
Returns 1 if the reference graph of `value` contains a loop (a
|
||||
structure that can reach itself), 0 otherwise.
|
||||
|
||||
The walk follows array and class items, mapping keys and values, and a
|
||||
function pointer's captured argument list. Objects are leaves: a loop
|
||||
routed through an object's global variables is reclaimed by destruct()
|
||||
and is not reported here.
|
||||
|
||||
Sharing without a loop (the same array referenced from two places) is
|
||||
NOT a cycle and returns 0.
|
||||
|
||||
The traversal is iterative, so unlike save_variable() or copy() it has
|
||||
no nesting-depth limit: arbitrarily deep acyclic values scan cleanly.
|
||||
|
||||
Values containing a loop cannot be saved, deep-copied, or fully
|
||||
printed, and once the last outside reference is dropped they leak
|
||||
permanently (the driver reclaims values by reference counting and has
|
||||
no cycle collector). See the "Reference Loops" concepts page for the
|
||||
full story.
|
||||
|
||||
### EXAMPLE
|
||||
|
||||
mixed *a = ({ 0 });
|
||||
a[0] = a;
|
||||
has_cycle(a); // 1
|
||||
a[0] = 0;
|
||||
has_cycle(a); // 0
|
||||
|
||||
### SEE ALSO
|
||||
|
||||
find_cycles(3), break_cycles(3), refs(3), copy(3)
|
||||
|
|
@ -24,10 +24,20 @@ title: internals / check_memory
|
|||
down by allocation source.
|
||||
Bit 1 (value 2) Runs silently -- suppresses the warnings and report
|
||||
and returns 0 instead of the report string.
|
||||
Bit 2 (value 4) Skips the orphaned-reference-loop scan (see below).
|
||||
|
||||
Unless bit 1 or bit 2 is set, the report also includes a scan for
|
||||
data blocks that are unreachable because only a reference loop keeps
|
||||
them alive (`unreachable data block(s) kept alive only by reference
|
||||
loop(s)`) -- garbage that pure reference counting can never reclaim
|
||||
and that the plain ref-count comparison cannot see. Reclaim such
|
||||
blocks with find_orphaned_cycles(1). The driver testsuite calls
|
||||
check_memory() after every test file, so a test that drops a cyclic
|
||||
structure without breaking it first now fails with that warning.
|
||||
|
||||
This efun is available only in DEBUGMALLOC builds (compiled with
|
||||
DEBUGMALLOC_EXTENSIONS); it does not exist in ordinary builds.
|
||||
|
||||
### SEE ALSO
|
||||
|
||||
dump_stralloc(3), reclaim_objects(3)
|
||||
find_orphaned_cycles(3), dump_stralloc(3), reclaim_objects(3)
|
||||
|
|
|
|||
55
docs/efun/internals/find_orphaned_cycles.md
Normal file
55
docs/efun/internals/find_orphaned_cycles.md
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
---
|
||||
title: internals / find_orphaned_cycles
|
||||
---
|
||||
# find_orphaned_cycles
|
||||
|
||||
### NAME
|
||||
|
||||
find_orphaned_cycles() - detect (and reclaim) memory leaked by dropped
|
||||
reference loops
|
||||
|
||||
### SYNOPSIS
|
||||
|
||||
int find_orphaned_cycles(int collect);
|
||||
|
||||
### DESCRIPTION
|
||||
|
||||
Scans every live array, class, mapping, and function pointer in the
|
||||
driver and returns the number of blocks that are UNREACHABLE -- kept
|
||||
alive only by a reference loop after the last outside reference was
|
||||
dropped. Pure reference counting can never reclaim such blocks, and no
|
||||
LPC-level tool (refs(), break_cycles()) can reach them anymore, because
|
||||
by definition no variable leads to them.
|
||||
|
||||
With collect != 0 the orphaned blocks are also reclaimed, safely:
|
||||
strings, buffers, and objects they referenced are released normally,
|
||||
values still reachable elsewhere are untouched, and the loop members
|
||||
themselves are freed. This is a true (if manual) cycle collector.
|
||||
|
||||
Detection is by trial deletion: a data block whose every reference is
|
||||
accounted for by other data blocks, with no path from any externally
|
||||
held block (an object's variables, the VM stack, a call_out, a driver-
|
||||
internal holder), is garbage. Reachable loops are NOT reported -- use
|
||||
has_cycle() / find_cycles() / break_cycles() while you still hold a
|
||||
reference.
|
||||
|
||||
Only available on debugging builds (DEBUGMALLOC_EXTENSIONS), because it
|
||||
needs the debug allocator's registry of every live block; guard call
|
||||
sites with `#if efun_defined(find_orphaned_cycles)`. On those builds,
|
||||
check_memory() also reports orphaned loops (`unreachable data
|
||||
block(s) kept alive only by reference loop(s)`), which makes a dropped
|
||||
cycle a hard failure in the driver testsuite.
|
||||
|
||||
### EXAMPLE
|
||||
|
||||
mixed *a = ({ 0 });
|
||||
a[0] = a; // build a loop...
|
||||
a = 0; // ...and drop it: leaked, invisibly
|
||||
|
||||
find_orphaned_cycles(0); // 1 -- found it
|
||||
find_orphaned_cycles(1); // 1 -- found it and reclaimed it
|
||||
find_orphaned_cycles(0); // 0
|
||||
|
||||
### SEE ALSO
|
||||
|
||||
has_cycle(3), find_cycles(3), break_cycles(3), check_memory(3), refs(3)
|
||||
|
|
@ -17,7 +17,14 @@ title: internals / refs
|
|||
useful for deciding whether or not to make a copy of a data structure
|
||||
before returning it.
|
||||
|
||||
### NOTES
|
||||
|
||||
A structure held by a single variable reports 1; a higher count you
|
||||
cannot account for often means the structure references itself -- see
|
||||
has_cycle(3) and the "Reference Loops" concepts page.
|
||||
|
||||
### SEE ALSO
|
||||
|
||||
children(3), inherit_list(3), deep_inherit_list(3), objects(3)
|
||||
children(3), inherit_list(3), deep_inherit_list(3), objects(3),
|
||||
has_cycle(3), find_cycles(3), break_cycles(3), find_orphaned_cycles(3)
|
||||
|
||||
|
|
|
|||
|
|
@ -2678,5 +2678,33 @@
|
|||
"sidebar.docs.doc.Build (v2017, legacy)": {
|
||||
"message": "构建（v2017，旧版）",
|
||||
"description": "The label for the doc item 'Build (v2017, legacy)' in sidebar 'docs', linking to the doc build_v2017"
|
||||
},
|
||||
"sidebar.docs.doc.efun/buffers/to_buffer": {
|
||||
"message": "to_buffer",
|
||||
"description": "The label for the doc item 'to_buffer' in sidebar 'docs', linking to the doc efun/buffers/to_buffer"
|
||||
},
|
||||
"sidebar.docs.doc.efun/contrib/break_cycles": {
|
||||
"message": "break_cycles",
|
||||
"description": "The label for the doc item 'break_cycles' in sidebar 'docs', linking to the doc efun/contrib/break_cycles"
|
||||
},
|
||||
"sidebar.docs.doc.efun/contrib/find_cycles": {
|
||||
"message": "find_cycles",
|
||||
"description": "The label for the doc item 'find_cycles' in sidebar 'docs', linking to the doc efun/contrib/find_cycles"
|
||||
},
|
||||
"sidebar.docs.doc.efun/contrib/has_cycle": {
|
||||
"message": "has_cycle",
|
||||
"description": "The label for the doc item 'has_cycle' in sidebar 'docs', linking to the doc efun/contrib/has_cycle"
|
||||
},
|
||||
"sidebar.docs.doc.efun/internals/find_orphaned_cycles": {
|
||||
"message": "find_orphaned_cycles",
|
||||
"description": "The label for the doc item 'find_orphaned_cycles' in sidebar 'docs', linking to the doc efun/internals/find_orphaned_cycles"
|
||||
},
|
||||
"sidebar.docs.doc.apply/master/valid_ffi": {
|
||||
"message": "valid_ffi",
|
||||
"description": "The label for the doc item 'valid_ffi' in sidebar 'docs', linking to the doc apply/master/valid_ffi"
|
||||
},
|
||||
"sidebar.docs.doc.concepts/general/reference_loops": {
|
||||
"message": "Reference Loops",
|
||||
"description": "The label for the doc item 'Reference Loops' in sidebar 'docs', linking to the doc concepts/general/reference_loops"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -176,6 +176,7 @@
|
|||
"fluffos_driver",
|
||||
"simul_efun",
|
||||
"message_doc",
|
||||
"reference_loops",
|
||||
"hot_reload",
|
||||
"socket_efuns",
|
||||
"tls",
|
||||
|
|
@ -191,6 +192,7 @@
|
|||
"fluffos_driver": "The FluffOS Driver",
|
||||
"simul_efun": "Simulated Efuns",
|
||||
"message_doc": "The message() System",
|
||||
"reference_loops": "Reference Loops",
|
||||
"hot_reload": "Hot Reload",
|
||||
"socket_efuns": "Socket Efuns",
|
||||
"tls": "TLS",
|
||||
|
|
|
|||
|
|
@ -295,6 +295,12 @@
|
|||
"key": "efun/contrib/base_name",
|
||||
"label": "base_name"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "efun/contrib/break_cycles",
|
||||
"key": "efun/contrib/break_cycles",
|
||||
"label": "break_cycles"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "efun/contrib/classes",
|
||||
|
|
@ -349,6 +355,12 @@
|
|||
"key": "efun/contrib/file_length",
|
||||
"label": "file_length"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "efun/contrib/find_cycles",
|
||||
"key": "efun/contrib/find_cycles",
|
||||
"label": "find_cycles"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "efun/contrib/function_owner",
|
||||
|
|
@ -373,6 +385,12 @@
|
|||
"key": "efun/contrib/get_os_env",
|
||||
"label": "get_os_env"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "efun/contrib/has_cycle",
|
||||
"key": "efun/contrib/has_cycle",
|
||||
"label": "has_cycle"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "efun/contrib/heart_beats",
|
||||
|
|
@ -1728,6 +1746,12 @@
|
|||
"key": "efun/internals/dumpallobj",
|
||||
"label": "dumpallobj"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "efun/internals/find_orphaned_cycles",
|
||||
"key": "efun/internals/find_orphaned_cycles",
|
||||
"label": "find_orphaned_cycles"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "efun/internals/get_config",
|
||||
|
|
@ -3593,6 +3617,12 @@
|
|||
"key": "concepts/general/message_doc",
|
||||
"label": "The message() System"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "concepts/general/reference_loops",
|
||||
"key": "concepts/general/reference_loops",
|
||||
"label": "Reference Loops"
|
||||
},
|
||||
{
|
||||
"type": "doc",
|
||||
"id": "concepts/general/hot_reload",
|
||||
|
|
|
|||
|
|
@ -31,6 +31,12 @@ typedef struct md_node_s {
|
|||
#define MD_OVERHEAD (sizeof(md_node_t))
|
||||
#endif
|
||||
void check_all_blocks(int);
|
||||
#ifdef DEBUGMALLOC_EXTENSIONS
|
||||
// Orphaned reference-loop scan (trial deletion); see checkmemory.cc.
|
||||
// Returns the number of unreachable array/class/mapping/funptr blocks;
|
||||
// reclaims them when collect is nonzero. ob (nullable) receives detail.
|
||||
int md_scan_orphaned_cycles(int collect, outbuffer_t* ob);
|
||||
#endif
|
||||
|
||||
#define MD_TABLE_BITS 14u
|
||||
#define MD_TABLE_SIZE (1u << MD_TABLE_BITS)
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
if(${PACKAGE_CONTRIB})
|
||||
add_library(package_contrib STATIC
|
||||
"contrib.cc"
|
||||
"cycles.cc"
|
||||
)
|
||||
endif()
|
||||
|
|
|
|||
|
|
@ -2,6 +2,8 @@
|
|||
|
||||
#include <sys/stat.h> // for struct stat
|
||||
|
||||
#include <memory> // for std::unique_ptr (deep_copy_* error-unwind guards)
|
||||
|
||||
#include "packages/core/heartbeat.h"
|
||||
#include "packages/core/add_action.h"
|
||||
#include "packages/core/file.h"
|
||||
|
|
@ -220,28 +222,29 @@ static int depth;
|
|||
|
||||
static void deep_copy_svalue(svalue_t* /*from*/, svalue_t* /*to*/);
|
||||
|
||||
// The deep_copy_* helpers must survive an error() unwind from a nested
|
||||
// deep_copy_svalue() -- the MAX_SAVE_SVALUE_DEPTH cap fires on every
|
||||
// self-referential (cyclic) structure, and mapping_too_large() can fire from
|
||||
// doCopy(). Two rules keep that safe: allocate the destination ZEROED (the
|
||||
// _empty_ allocators leave svalues uninitialized, which the Debug memory
|
||||
// checker would then walk), and hold it in a unique_ptr so the half-built
|
||||
// copy is freed instead of leaking with borrowed pointers inside.
|
||||
static array_t* deep_copy_array(array_t* arg) {
|
||||
array_t* vec;
|
||||
int i;
|
||||
|
||||
vec = allocate_empty_array(arg->size);
|
||||
for (i = 0; i < arg->size; i++) {
|
||||
std::unique_ptr<array_t, void (*)(array_t*)> vec(allocate_array(arg->size), free_array);
|
||||
for (int i = 0; i < arg->size; i++) {
|
||||
deep_copy_svalue(&arg->item[i], &vec->item[i]);
|
||||
}
|
||||
|
||||
return vec;
|
||||
return vec.release();
|
||||
}
|
||||
|
||||
static array_t* deep_copy_class(array_t* arg) {
|
||||
array_t* vec;
|
||||
int i;
|
||||
|
||||
vec = allocate_empty_class_by_size(arg->size);
|
||||
for (i = 0; i < arg->size; i++) {
|
||||
std::unique_ptr<array_t, void (*)(array_t*)> vec(allocate_class_by_size(arg->size), free_class);
|
||||
for (int i = 0; i < arg->size; i++) {
|
||||
deep_copy_svalue(&arg->item[i], &vec->item[i]);
|
||||
}
|
||||
|
||||
return vec;
|
||||
return vec.release();
|
||||
}
|
||||
|
||||
static int doCopy(mapping_t* /*map*/, mapping_node_t* elt, void* dest) {
|
||||
|
|
@ -258,48 +261,44 @@ static int doCopy(mapping_t* /*map*/, mapping_node_t* elt, void* dest) {
|
|||
}
|
||||
|
||||
static mapping_t* deep_copy_mapping(mapping_t* arg) {
|
||||
mapping_t* map;
|
||||
|
||||
map = allocate_mapping(0); /* this should be fixed. -Beek */
|
||||
mapTraverse(arg, doCopy, map); /* Not horridly efficient either */
|
||||
return map;
|
||||
std::unique_ptr<mapping_t, void (*)(mapping_t*)> map(allocate_mapping(0), free_mapping);
|
||||
mapTraverse(arg, doCopy, map.get()); /* Not horridly efficient either */
|
||||
return map.release();
|
||||
}
|
||||
|
||||
// Copy the child FIRST, then write the destination slot in one go: `to`
|
||||
// points into a container that outlives an error() unwind (freed by the
|
||||
// guards above), so it must never hold `from`'s pointer without owning a
|
||||
// reference while a nested copy can still throw.
|
||||
static void deep_copy_svalue(svalue_t* from, svalue_t* to) {
|
||||
switch (from->type) {
|
||||
case T_ARRAY:
|
||||
depth++;
|
||||
if (depth > MAX_SAVE_SVALUE_DEPTH) {
|
||||
depth = 0;
|
||||
error("Mappings, arrays and/or classes nested too deep (%d) for copy()\n",
|
||||
MAX_SAVE_SVALUE_DEPTH);
|
||||
}
|
||||
*to = *from;
|
||||
to->u.arr = deep_copy_array(from->u.arr);
|
||||
depth--;
|
||||
break;
|
||||
case T_CLASS:
|
||||
case T_MAPPING: {
|
||||
depth++;
|
||||
if (depth > MAX_SAVE_SVALUE_DEPTH) {
|
||||
depth = 0;
|
||||
error("Mappings, arrays and/or classes nested too deep (%d) for copy()\n",
|
||||
MAX_SAVE_SVALUE_DEPTH);
|
||||
error(
|
||||
"Mappings, arrays and/or classes nested too deep (%d) for copy() "
|
||||
"-- possibly a reference loop; see has_cycle()\n",
|
||||
MAX_SAVE_SVALUE_DEPTH);
|
||||
}
|
||||
*to = *from;
|
||||
to->u.arr = deep_copy_class(from->u.arr);
|
||||
depth--;
|
||||
break;
|
||||
case T_MAPPING:
|
||||
depth++;
|
||||
if (depth > MAX_SAVE_SVALUE_DEPTH) {
|
||||
depth = 0;
|
||||
error("Mappings, arrays and/or classes nested too deep (%d) for copy()\n",
|
||||
MAX_SAVE_SVALUE_DEPTH);
|
||||
}
|
||||
*to = *from;
|
||||
to->u.map = deep_copy_mapping(from->u.map);
|
||||
svalue_t nv = *from;
|
||||
switch (from->type) {
|
||||
case T_ARRAY:
|
||||
nv.u.arr = deep_copy_array(from->u.arr);
|
||||
break;
|
||||
case T_CLASS:
|
||||
nv.u.arr = deep_copy_class(from->u.arr);
|
||||
break;
|
||||
case T_MAPPING:
|
||||
nv.u.map = deep_copy_mapping(from->u.map);
|
||||
break;
|
||||
}
|
||||
*to = nv;
|
||||
depth--;
|
||||
break;
|
||||
}
|
||||
case T_BUFFER:
|
||||
*to = *from;
|
||||
to->u.buf = allocate_buffer(from->u.buf->size);
|
||||
|
|
|
|||
|
|
@ -59,3 +59,11 @@ mixed *classes(object, int default : 0);
|
|||
int test_load(string);
|
||||
string get_os_env(string);
|
||||
int set_os_env(string, string | void);
|
||||
|
||||
/*
|
||||
* Reference-loop (cycle) introspection -- see cycles.cc and
|
||||
* docs/concepts/general/reference_loops.md.
|
||||
*/
|
||||
int has_cycle(mixed);
|
||||
string *find_cycles(mixed);
|
||||
int break_cycles(mixed);
|
||||
|
|
|
|||
377
src/packages/contrib/cycles.cc
Normal file
377
src/packages/contrib/cycles.cc
Normal file
|
|
@ -0,0 +1,377 @@
|
|||
// Reference-loop (cycle) introspection efuns: has_cycle(), find_cycles(),
|
||||
// break_cycles().
|
||||
//
|
||||
// The VM reclaims compound values by pure reference counting with no cycle
|
||||
// collector, so a structure that reaches itself leaks permanently once the
|
||||
// last outside reference is dropped (see docs/concepts/general/
|
||||
// reference_loops.md). These efuns give the mudlib tools to find such loops
|
||||
// and to break them safely.
|
||||
//
|
||||
// All three share one traversal: an ITERATIVE depth-first search (explicit
|
||||
// heap stack -- mudlib data can nest arbitrarily deep, so no C-stack
|
||||
// recursion and no MAX_SAVE_SVALUE_DEPTH cap) over the value graph, with the
|
||||
// classic white/grey/black coloring. An edge whose target is grey (still
|
||||
// open on the current DFS path) is a back-edge; a directed graph is acyclic
|
||||
// iff its DFS finds no back-edges, so breaking exactly the back-edges is the
|
||||
// minimal edit that makes a value loop-free while leaving all other sharing
|
||||
// (DAG edges to already-finished nodes) untouched.
|
||||
//
|
||||
// Traversed edges: array/class items, mapping keys AND values, and a
|
||||
// function pointer's captured argument list (fp->hdr.args). Objects are
|
||||
// deliberately leaves: a loop routed through an object's global variables is
|
||||
// the destruct()-managed kind (destruct2() zeroes the variable block), and a
|
||||
// value walk has no business reading other objects' variables.
|
||||
// NOTE: this edge set must stay in sync with each_child() in
|
||||
// src/packages/develop/checkmemory.cc (md_scan_orphaned_cycles), or these
|
||||
// efuns and the Debug orphan scan will disagree about what a loop is.
|
||||
//
|
||||
// The walk itself never mutates the graph and never frees anything, so the
|
||||
// raw pointers used as color-map keys stay valid for the whole walk.
|
||||
// break_cycles() records the slots to clear during the walk and applies them
|
||||
// in a post-pass; see the ordering comments in f_break_cycles().
|
||||
|
||||
#include "base/package_api.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <string>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
|
||||
#if defined(F_HAS_CYCLE) || defined(F_FIND_CYCLES) || defined(F_BREAK_CYCLES)
|
||||
|
||||
namespace {
|
||||
|
||||
enum : char { COLOR_GREY = 1, COLOR_BLACK = 2 };
|
||||
|
||||
enum WalkMode { WALK_DETECT, WALK_FIND, WALK_BREAK };
|
||||
|
||||
// Which kind of slot a back-edge sits in decides how break_cycles() clears
|
||||
// it: item/value slots are overwritten with 0 in place; a back-edge in
|
||||
// mapping-KEY position cannot be overwritten (the node is hashed by the
|
||||
// key's pointer, and a second zero key could collide), so the whole node is
|
||||
// deleted instead; a back-edge on the funptr->args edge (possible because
|
||||
// bind() SHARES the args array between the old and new funptr) has no
|
||||
// svalue slot at all, so the funptr's args list is replaced with a
|
||||
// zero-filled one of the same size.
|
||||
enum SlotKind { SLOT_ITEM, SLOT_MAP_VALUE, SLOT_MAP_KEY, SLOT_FP_ARGS };
|
||||
|
||||
struct PendingFix {
|
||||
SlotKind kind;
|
||||
svalue_t container; // owns a reference until the post-pass is done
|
||||
svalue_t* slot; // ITEM/MAP_VALUE: slot to zero; MAP_KEY: &node->values[0];
|
||||
// FP_ARGS: null (container is the owning funptr)
|
||||
};
|
||||
|
||||
struct Frame {
|
||||
unsigned short type; // T_ARRAY, T_CLASS, T_MAPPING, or T_FUNCTION
|
||||
union {
|
||||
array_t* arr;
|
||||
mapping_t* map;
|
||||
funptr_t* fp;
|
||||
void* ptr;
|
||||
} u;
|
||||
int idx; // next item (array/class), current bucket (mapping),
|
||||
// 0 = args pending / 1 = done (function)
|
||||
mapping_node_t* node; // current node within the bucket (mapping)
|
||||
int phase; // 0 = key next, 1 = value next (mapping)
|
||||
};
|
||||
|
||||
struct WalkResult {
|
||||
bool found = false;
|
||||
std::vector<std::string> paths; // WALK_FIND
|
||||
std::vector<PendingFix> fixes; // WALK_BREAK
|
||||
};
|
||||
|
||||
void* compound_ptr(const svalue_t* sv) {
|
||||
switch (sv->type) {
|
||||
case T_ARRAY:
|
||||
case T_CLASS:
|
||||
return reinterpret_cast<void*>(sv->u.arr);
|
||||
case T_MAPPING:
|
||||
return reinterpret_cast<void*>(sv->u.map);
|
||||
case T_FUNCTION:
|
||||
return reinterpret_cast<void*>(sv->u.fp);
|
||||
}
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
// Human-readable rendering of a mapping key for find_cycles() paths.
|
||||
std::string render_key(const svalue_t* key) {
|
||||
switch (key->type) {
|
||||
case T_NUMBER:
|
||||
return std::to_string(key->u.number);
|
||||
case T_REAL: {
|
||||
char buf[64];
|
||||
snprintf(buf, sizeof(buf), "%g", key->u.real);
|
||||
return buf;
|
||||
}
|
||||
case T_STRING: {
|
||||
size_t len = SVALUE_STRLEN(key);
|
||||
if (len > 32) {
|
||||
len = u8_truncate(reinterpret_cast<const uint8_t*>(key->u.string), 32);
|
||||
}
|
||||
return "\"" + std::string(key->u.string, len) + "\"";
|
||||
}
|
||||
case T_OBJECT:
|
||||
return std::string("OBJ(") + key->u.ob->obname + ")";
|
||||
case T_ARRAY:
|
||||
return "<array>";
|
||||
case T_CLASS:
|
||||
return "<class>";
|
||||
case T_MAPPING:
|
||||
return "<mapping>";
|
||||
case T_FUNCTION:
|
||||
return "<function>";
|
||||
default:
|
||||
return "<...>";
|
||||
}
|
||||
}
|
||||
|
||||
Frame make_frame(unsigned short type, void* ptr) {
|
||||
Frame f{};
|
||||
f.type = type;
|
||||
f.u.ptr = ptr;
|
||||
if (type == T_MAPPING) {
|
||||
f.node = reinterpret_cast<mapping_t*>(ptr)->table[0];
|
||||
}
|
||||
return f;
|
||||
}
|
||||
|
||||
void cycle_walk(svalue_t* root, WalkMode mode, WalkResult* res) {
|
||||
void* rptr = compound_ptr(root);
|
||||
if (rptr == nullptr) {
|
||||
return;
|
||||
}
|
||||
|
||||
std::unordered_map<void*, char> color;
|
||||
std::vector<Frame> stack;
|
||||
std::vector<std::string> labels; // edge label per open frame below the root
|
||||
|
||||
color[rptr] = COLOR_GREY;
|
||||
stack.push_back(make_frame(root->type, rptr));
|
||||
|
||||
// Examine one outgoing edge of the frame on top of the stack. `slot` is
|
||||
// null only for the funptr->args edge (kind SLOT_FP_ARGS), which is not an
|
||||
// svalue slot; it CAN be a back-edge target, because bind() shares the
|
||||
// args array between the old and the new funptr (f_bind in
|
||||
// packages/core/efuns_main.cc bumps args->ref instead of copying).
|
||||
auto handle_edge = [&](unsigned short ttype, void* tptr, svalue_t* slot, SlotKind kind,
|
||||
std::string&& label) {
|
||||
auto ins = color.try_emplace(tptr, COLOR_GREY);
|
||||
if (ins.second) { // white: tree edge, descend
|
||||
Frame child = make_frame(ttype, tptr);
|
||||
stack.push_back(child); // may invalidate refs into `stack`
|
||||
if (mode == WALK_FIND) {
|
||||
labels.push_back(std::move(label));
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (ins.first->second == COLOR_GREY) { // back-edge: a cycle
|
||||
res->found = true;
|
||||
if (mode == WALK_FIND) {
|
||||
std::string path;
|
||||
for (const auto& l : labels) {
|
||||
path += l;
|
||||
}
|
||||
path += label;
|
||||
res->paths.push_back(std::move(path));
|
||||
} else if (mode == WALK_BREAK) {
|
||||
PendingFix fix;
|
||||
fix.kind = kind;
|
||||
fix.slot = slot;
|
||||
// Hold a reference on the slot's container (for SLOT_FP_ARGS, the
|
||||
// owning funptr) so the post-pass can touch it no matter what
|
||||
// earlier fixes deallocated.
|
||||
const Frame& owner = stack.back();
|
||||
svalue_t tmp;
|
||||
tmp.type = owner.type;
|
||||
tmp.subtype = 0;
|
||||
tmp.u.arr = reinterpret_cast<array_t*>(owner.u.ptr);
|
||||
assign_svalue_no_free(&fix.container, &tmp);
|
||||
res->fixes.push_back(fix);
|
||||
}
|
||||
}
|
||||
// black: already fully explored, nothing new reachable
|
||||
};
|
||||
|
||||
while (!stack.empty()) {
|
||||
if (mode == WALK_DETECT && res->found) {
|
||||
return;
|
||||
}
|
||||
|
||||
Frame& f = stack.back();
|
||||
svalue_t* slot = nullptr;
|
||||
SlotKind kind = SLOT_ITEM;
|
||||
std::string label;
|
||||
bool have_edge = false;
|
||||
|
||||
switch (f.type) {
|
||||
case T_ARRAY:
|
||||
case T_CLASS: {
|
||||
array_t* arr = f.u.arr;
|
||||
if (f.idx < arr->size) {
|
||||
slot = &arr->item[f.idx];
|
||||
kind = SLOT_ITEM;
|
||||
if (mode == WALK_FIND) {
|
||||
label = (f.type == T_ARRAY) ? "[" + std::to_string(f.idx) + "]"
|
||||
: "." + std::to_string(f.idx);
|
||||
}
|
||||
f.idx++;
|
||||
have_edge = true;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case T_MAPPING: {
|
||||
mapping_t* map = f.u.map;
|
||||
// table_size is a mask: valid buckets are 0 .. table_size inclusive
|
||||
while (f.node == nullptr && f.idx < static_cast<int>(map->table_size)) {
|
||||
f.idx++;
|
||||
f.node = map->table[f.idx];
|
||||
}
|
||||
if (f.node != nullptr) {
|
||||
if (f.phase == 0) {
|
||||
slot = &f.node->values[0];
|
||||
kind = SLOT_MAP_KEY;
|
||||
if (mode == WALK_FIND) {
|
||||
label = "[key " + render_key(slot) + "]";
|
||||
}
|
||||
f.phase = 1;
|
||||
} else {
|
||||
slot = &f.node->values[1];
|
||||
kind = SLOT_MAP_VALUE;
|
||||
if (mode == WALK_FIND) {
|
||||
label = "[" + render_key(&f.node->values[0]) + "]";
|
||||
}
|
||||
f.node = f.node->next;
|
||||
f.phase = 0;
|
||||
}
|
||||
have_edge = true;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case T_FUNCTION: {
|
||||
funptr_t* fp = f.u.fp;
|
||||
if (f.idx == 0) {
|
||||
f.idx = 1;
|
||||
if (fp->hdr.args != nullptr) {
|
||||
handle_edge(T_ARRAY, reinterpret_cast<void*>(fp->hdr.args), nullptr, SLOT_FP_ARGS,
|
||||
mode == WALK_FIND ? std::string("(args)") : std::string());
|
||||
continue; // `f` may be a stale reference now
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!have_edge) { // frame exhausted: close it
|
||||
color[f.u.ptr] = COLOR_BLACK;
|
||||
stack.pop_back();
|
||||
if (mode == WALK_FIND && !stack.empty()) {
|
||||
labels.pop_back(); // the root frame never pushed a label
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
void* tptr = compound_ptr(slot);
|
||||
if (tptr != nullptr) {
|
||||
handle_edge(slot->type, tptr, slot, kind, std::move(label));
|
||||
// `f` may be a stale reference now; loop back around
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
#endif // any of the three efuns
|
||||
|
||||
#ifdef F_HAS_CYCLE
|
||||
void f_has_cycle() {
|
||||
WalkResult res;
|
||||
cycle_walk(sp, WALK_DETECT, &res);
|
||||
free_svalue(sp, "f_has_cycle");
|
||||
put_number(res.found ? 1 : 0);
|
||||
}
|
||||
#endif
|
||||
|
||||
#ifdef F_FIND_CYCLES
|
||||
void f_find_cycles() {
|
||||
WalkResult res;
|
||||
cycle_walk(sp, WALK_FIND, &res);
|
||||
// May error("Illegal array size") if there are more back-edges than
|
||||
// max_array_size; at this point nothing is held, so that unwinds cleanly
|
||||
// (the std::vectors are reclaimed by their destructors).
|
||||
array_t* ret = allocate_empty_array(static_cast<int>(res.paths.size()));
|
||||
for (size_t i = 0; i < res.paths.size(); i++) {
|
||||
ret->item[i].type = T_STRING;
|
||||
ret->item[i].subtype = STRING_MALLOC;
|
||||
ret->item[i].u.string = string_copy(res.paths[i].c_str(), "f_find_cycles");
|
||||
}
|
||||
free_svalue(sp, "f_find_cycles");
|
||||
put_array(ret);
|
||||
}
|
||||
#endif
|
||||
|
||||
#ifdef F_BREAK_CYCLES
|
||||
void f_break_cycles() {
|
||||
WalkResult res;
|
||||
cycle_walk(sp, WALK_BREAK, &res);
|
||||
|
||||
// The temporary container references taken during the walk are released
|
||||
// by RAII, not a trailing loop: allocate_array() below can error() (LPC
|
||||
// can shrink __MAX_ARRAY_SIZE__ at runtime via set_config(), making a
|
||||
// same-size replacement illegal), and an unwind that skipped a manual
|
||||
// release loop would leak every held reference (AGENTS.md section 4).
|
||||
struct ContainerRelease {
|
||||
std::vector<PendingFix>* fixes;
|
||||
~ContainerRelease() {
|
||||
for (auto& fix : *fixes) {
|
||||
free_svalue(&fix.container, "f_break_cycles");
|
||||
}
|
||||
}
|
||||
} release_guard{&res.fixes};
|
||||
|
||||
// Apply the recorded fixes. Order matters:
|
||||
//
|
||||
// 1. Overwrite item/value back-edge slots with 0 first, and detach the
|
||||
// args list of any funptr whose args EDGE is the back-edge (possible
|
||||
// via bind()'s args sharing) -- replaced with a zero-filled list of
|
||||
// the same size so the funptr keeps its call arity. Neither can ever
|
||||
// cascade a deallocation: every back-edge target is an ancestor on the
|
||||
// walk's tree path, and that path -- made of tree edges, which we never
|
||||
// break -- anchors it to the root the caller still holds on the VM
|
||||
// stack.
|
||||
LPC_INT broken = 0;
|
||||
for (auto& fix : res.fixes) {
|
||||
if (fix.kind == SLOT_FP_ARGS) {
|
||||
funptr_t* fp = fix.container.u.fp;
|
||||
array_t* old = fp->hdr.args;
|
||||
if (old != nullptr) {
|
||||
fp->hdr.args = allocate_array(old->size);
|
||||
free_array(old);
|
||||
}
|
||||
broken++;
|
||||
} else if (fix.kind != SLOT_MAP_KEY) {
|
||||
free_svalue(fix.slot, "f_break_cycles");
|
||||
*fix.slot = const0;
|
||||
broken++;
|
||||
}
|
||||
}
|
||||
// 2. Then delete mapping nodes whose KEY is the back-edge. Deleting a node
|
||||
// frees its value subtree, which CAN cascade; every other pending fix
|
||||
// holds its own reference on its container (taken during the walk), so
|
||||
// the deletions are order-independent and never touch freed memory.
|
||||
// (Same caveat as map_delete(): a `foreach (k, ref v in m)` loop
|
||||
// variable in an OUTER frame aimed at the deleted node's value slot is
|
||||
// left dangling -- a pre-existing hazard of node deletion during an
|
||||
// unlocked mapping foreach, not specific to this efun.)
|
||||
for (auto& fix : res.fixes) {
|
||||
if (fix.kind == SLOT_MAP_KEY) {
|
||||
mapping_delete(fix.container.u.map, fix.slot);
|
||||
broken++;
|
||||
}
|
||||
}
|
||||
|
||||
free_svalue(sp, "f_break_cycles");
|
||||
put_number(broken);
|
||||
}
|
||||
#endif
|
||||
|
|
@ -38,6 +38,10 @@
|
|||
#include "packages/jsbridge/jsbridge.h"
|
||||
#endif
|
||||
|
||||
#include <functional>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
|
||||
#if (defined(DEBUGMALLOC) && defined(DEBUGMALLOC_EXTENSIONS))
|
||||
|
||||
void mark_svalue(struct svalue_t*);
|
||||
|
|
@ -1070,6 +1074,20 @@ void check_all_blocks(int flag) {
|
|||
}
|
||||
}
|
||||
if (!(flag & 2)) {
|
||||
// A detached reference loop is invisible to the ref-count comparison
|
||||
// above (a cycle is perfectly self-consistent), so hunt for it
|
||||
// separately -- skippable via flag bit 2 (value 4) since it is a full
|
||||
// extra pass over every compound block. Report only; collection is the
|
||||
// mudlib's explicit call via find_orphaned_cycles(1).
|
||||
if (!(flag & 4)) {
|
||||
int orphans = md_scan_orphaned_cycles(0, &out);
|
||||
if (orphans) {
|
||||
outbuf_addv(&out,
|
||||
"WARNING: %d unreachable data block(s) kept alive only by "
|
||||
"reference loop(s); reclaim with find_orphaned_cycles(1)\n",
|
||||
orphans);
|
||||
}
|
||||
}
|
||||
outbuf_push(&out);
|
||||
} else {
|
||||
FREE_MSTR(out.buffer);
|
||||
|
|
@ -1077,4 +1095,268 @@ void check_all_blocks(int flag) {
|
|||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* Orphaned reference-loop detection and collection, via trial deletion
|
||||
* (the same idea CPython's gc uses):
|
||||
*
|
||||
* For every cycle-capable data block (array, class, mapping, funptr),
|
||||
* count how many of its references come from OTHER data blocks
|
||||
* ("internal"). external = ref - internal. A block with external > 0 is
|
||||
* held by something that is not plain data -- an object's variables, the
|
||||
* VM stack, a call_out, a C++-side holder -- and is therefore a live
|
||||
* root; liveness then propagates along data edges. Whatever remains is
|
||||
* reachable only from itself: garbage that pure reference counting can
|
||||
* never reclaim, which (in a finite graph) always contains at least one
|
||||
* reference loop.
|
||||
*
|
||||
* Because the verdict is computed from real ref counts, there is no root
|
||||
* enumeration to get wrong: any holder that took a legitimate reference --
|
||||
* including package internals invisible to the mark phase -- shows up as an
|
||||
* external ref and keeps its data alive.
|
||||
*
|
||||
* Collection mirrors what free-ing the loop by hand would do, in an order
|
||||
* that can never touch freed memory:
|
||||
* 1. take a reference on every dead block (so nothing deallocates early),
|
||||
* 2. sever: clear every dead block's child slots (releasing strings,
|
||||
* objects, buffers, and live values normally, and unlinking the dead
|
||||
* blocks from each other; a dead funptr just drops its args array),
|
||||
* 3. release the held references -- each dead block is now at ref 1 and
|
||||
* deallocates cleanly with nothing left to cascade into.
|
||||
*/
|
||||
int md_scan_orphaned_cycles(int collect, outbuffer_t* ob) {
|
||||
struct Cand {
|
||||
int tag;
|
||||
int internal = 0;
|
||||
bool live = false;
|
||||
};
|
||||
std::unordered_map<void*, Cand> cands;
|
||||
cands.reserve(blocks[TAG_ARRAY & 0xff] + blocks[TAG_CLASS & 0xff] +
|
||||
blocks[TAG_MAPPING & 0xff] + blocks[TAG_FUNP & 0xff]);
|
||||
|
||||
for (int hsh = 0; hsh < MD_TABLE_SIZE; hsh++) {
|
||||
for (md_node_t* entry = table[hsh]; entry; entry = entry->next) {
|
||||
switch (entry->tag) {
|
||||
case TAG_ARRAY:
|
||||
case TAG_CLASS:
|
||||
cands[NODET_TO_PTR(entry, void*)] = Cand{entry->tag};
|
||||
break;
|
||||
case TAG_MAPPING:
|
||||
cands[NODET_TO_PTR(entry, void*)] = Cand{entry->tag};
|
||||
break;
|
||||
case TAG_FUNP:
|
||||
cands[NODET_TO_PTR(entry, void*)] = Cand{entry->tag};
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
auto ref_of = [](void* p, int tag) -> LPC_INT {
|
||||
switch (tag) {
|
||||
case TAG_ARRAY:
|
||||
case TAG_CLASS:
|
||||
return reinterpret_cast<array_t*>(p)->ref;
|
||||
case TAG_MAPPING:
|
||||
return reinterpret_cast<mapping_t*>(p)->ref;
|
||||
case TAG_FUNP:
|
||||
return reinterpret_cast<funptr_t*>(p)->hdr.ref;
|
||||
}
|
||||
return 0;
|
||||
};
|
||||
|
||||
auto data_child = [](svalue_t* sv) -> void* {
|
||||
switch (sv->type) {
|
||||
case T_ARRAY:
|
||||
case T_CLASS:
|
||||
return reinterpret_cast<void*>(sv->u.arr);
|
||||
case T_MAPPING:
|
||||
return reinterpret_cast<void*>(sv->u.map);
|
||||
case T_FUNCTION:
|
||||
return reinterpret_cast<void*>(sv->u.fp);
|
||||
}
|
||||
return nullptr;
|
||||
};
|
||||
|
||||
// cb receives each data-block pointer this block holds a reference to.
|
||||
// (Generic lambda so the per-element callback inlines -- this runs over
|
||||
// every slot of every compound block in the heap, twice.)
|
||||
// NOTE: this edge set (array/class items, mapping keys AND values,
|
||||
// fp->hdr.args; objects are leaves) must stay in sync with the walker in
|
||||
// src/packages/contrib/cycles.cc (cycle_walk), or has_cycle()/
|
||||
// break_cycles() and this scan will disagree about what a loop is.
|
||||
auto each_child = [&](void* p, int tag, auto&& cb) {
|
||||
switch (tag) {
|
||||
case TAG_ARRAY:
|
||||
case TAG_CLASS: {
|
||||
auto* arr = reinterpret_cast<array_t*>(p);
|
||||
for (int i = 0; i < arr->size; i++) {
|
||||
if (void* c = data_child(&arr->item[i])) {
|
||||
cb(c);
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TAG_MAPPING: {
|
||||
auto* map = reinterpret_cast<mapping_t*>(p);
|
||||
for (int i = 0; i <= static_cast<int>(map->table_size); i++) {
|
||||
for (mapping_node_t* node = map->table[i]; node; node = node->next) {
|
||||
if (void* c = data_child(&node->values[0])) {
|
||||
cb(c);
|
||||
}
|
||||
if (void* c = data_child(&node->values[1])) {
|
||||
cb(c);
|
||||
}
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TAG_FUNP: {
|
||||
auto* fp = reinterpret_cast<funptr_t*>(p);
|
||||
if (fp->hdr.args) {
|
||||
cb(reinterpret_cast<void*>(fp->hdr.args));
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// internal = references held by other data blocks
|
||||
for (auto& kv : cands) {
|
||||
each_child(kv.first, kv.second.tag, [&](void* c) {
|
||||
auto it = cands.find(c);
|
||||
if (it != cands.end()) {
|
||||
it->second.internal++;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// seeds: any block some non-data holder references (conservatively
|
||||
// including anything whose counts look inconsistent), then propagate
|
||||
std::vector<std::pair<void*, int>> work; // (block, tag) -- avoids a re-lookup per pop
|
||||
for (auto& kv : cands) {
|
||||
if (ref_of(kv.first, kv.second.tag) != kv.second.internal) {
|
||||
kv.second.live = true;
|
||||
work.emplace_back(kv.first, kv.second.tag);
|
||||
}
|
||||
}
|
||||
while (!work.empty()) {
|
||||
auto [p, tag] = work.back();
|
||||
work.pop_back();
|
||||
each_child(p, tag, [&](void* c) {
|
||||
auto it = cands.find(c);
|
||||
if (it != cands.end() && !it->second.live) {
|
||||
it->second.live = true;
|
||||
work.emplace_back(c, it->second.tag);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
int dead = 0, n_arr = 0, n_cls = 0, n_map = 0, n_fp = 0;
|
||||
for (auto& kv : cands) {
|
||||
if (!kv.second.live) {
|
||||
dead++;
|
||||
switch (kv.second.tag) {
|
||||
case TAG_ARRAY:
|
||||
n_arr++;
|
||||
break;
|
||||
case TAG_CLASS:
|
||||
n_cls++;
|
||||
break;
|
||||
case TAG_MAPPING:
|
||||
n_map++;
|
||||
break;
|
||||
case TAG_FUNP:
|
||||
n_fp++;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (dead && ob) {
|
||||
outbuf_addv(ob, "orphaned by reference loops: %d array(s), %d class(es), %d mapping(s), %d function pointer(s)\n",
|
||||
n_arr, n_cls, n_map, n_fp);
|
||||
}
|
||||
|
||||
if (dead && collect) {
|
||||
// 1. hold
|
||||
for (auto& kv : cands) {
|
||||
if (kv.second.live) {
|
||||
continue;
|
||||
}
|
||||
switch (kv.second.tag) {
|
||||
case TAG_ARRAY:
|
||||
case TAG_CLASS:
|
||||
reinterpret_cast<array_t*>(kv.first)->ref++;
|
||||
break;
|
||||
case TAG_MAPPING:
|
||||
reinterpret_cast<mapping_t*>(kv.first)->ref++;
|
||||
break;
|
||||
case TAG_FUNP:
|
||||
reinterpret_cast<funptr_t*>(kv.first)->hdr.ref++;
|
||||
break;
|
||||
}
|
||||
}
|
||||
// 2. sever
|
||||
for (auto& kv : cands) {
|
||||
if (kv.second.live) {
|
||||
continue;
|
||||
}
|
||||
switch (kv.second.tag) {
|
||||
case TAG_ARRAY:
|
||||
case TAG_CLASS: {
|
||||
auto* arr = reinterpret_cast<array_t*>(kv.first);
|
||||
for (int i = 0; i < arr->size; i++) {
|
||||
free_svalue(&arr->item[i], "collect_cycles");
|
||||
arr->item[i] = const0;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TAG_MAPPING: {
|
||||
// The mapping is garbage; nobody can look anything up in it, so
|
||||
// zeroing keys in place (hash invariants be damned) is fine --
|
||||
// dealloc_mapping just walks the table.
|
||||
auto* map = reinterpret_cast<mapping_t*>(kv.first);
|
||||
for (int i = 0; i <= static_cast<int>(map->table_size); i++) {
|
||||
for (mapping_node_t* node = map->table[i]; node; node = node->next) {
|
||||
free_svalue(&node->values[0], "collect_cycles");
|
||||
node->values[0] = const0;
|
||||
free_svalue(&node->values[1], "collect_cycles");
|
||||
node->values[1] = const0;
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TAG_FUNP: {
|
||||
auto* fp = reinterpret_cast<funptr_t*>(kv.first);
|
||||
if (fp->hdr.args) {
|
||||
free_array(fp->hdr.args);
|
||||
fp->hdr.args = nullptr;
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
// 3. release
|
||||
for (auto& kv : cands) {
|
||||
if (kv.second.live) {
|
||||
continue;
|
||||
}
|
||||
switch (kv.second.tag) {
|
||||
case TAG_ARRAY:
|
||||
free_array(reinterpret_cast<array_t*>(kv.first));
|
||||
break;
|
||||
case TAG_CLASS:
|
||||
free_class(reinterpret_cast<array_t*>(kv.first));
|
||||
break;
|
||||
case TAG_MAPPING:
|
||||
free_mapping(reinterpret_cast<mapping_t*>(kv.first));
|
||||
break;
|
||||
case TAG_FUNP:
|
||||
free_funp(reinterpret_cast<funptr_t*>(kv.first));
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return dead;
|
||||
}
|
||||
|
||||
#endif /* DEBUGMALLOC_EXTENSIONS */
|
||||
|
|
|
|||
|
|
@ -246,6 +246,14 @@ void f_set_malloc_mask() { set_malloc_mask((sp--)->u.number); }
|
|||
#ifdef F_CHECK_MEMORY
|
||||
void f_check_memory() { check_all_blocks((sp--)->u.number); }
|
||||
#endif
|
||||
|
||||
#ifdef F_FIND_ORPHANED_CYCLES
|
||||
void f_find_orphaned_cycles() {
|
||||
// any nonzero LPC int means collect (don't truncate the 64-bit argument)
|
||||
int const collect = (sp--)->u.number != 0;
|
||||
push_number(md_scan_orphaned_cycles(collect, nullptr));
|
||||
}
|
||||
#endif
|
||||
#endif /* (defined(DEBUGMALLOC) && \
|
||||
* defined(DEBUGMALLOC_EXTENSIONS)) */
|
||||
|
||||
|
|
|
|||
|
|
@ -18,6 +18,9 @@
|
|||
string debugmalloc(string, int default: 0);
|
||||
void set_malloc_mask(int);
|
||||
string check_memory(int default: 0);
|
||||
/* count (and with arg 1, reclaim) data blocks that are unreachable because
|
||||
only a reference loop keeps them alive -- see checkmemory.cc */
|
||||
int find_orphaned_cycles(int default: 0);
|
||||
#endif
|
||||
string dump_stralloc(string);
|
||||
#ifdef DEBUG
|
||||
|
|
|
|||
|
|
@ -41,8 +41,15 @@ namespace fs = ghc::filesystem;
|
|||
#include "packages/sockets/socket_efuns.h" // for check_valid_path
|
||||
#endif
|
||||
|
||||
#define too_deep_save_error() \
|
||||
error("Mappings and/or arrays nested too deep (%d) for save_object\n", MAX_SAVE_SVALUE_DEPTH);
|
||||
// Only point at has_cycle() when the contrib package actually provides it.
|
||||
#ifdef PACKAGE_CONTRIB
|
||||
#define TOO_DEEP_SAVE_HINT " -- possibly a reference loop; see has_cycle()"
|
||||
#else
|
||||
#define TOO_DEEP_SAVE_HINT " -- possibly a reference loop"
|
||||
#endif
|
||||
#define too_deep_save_error() \
|
||||
error("Mappings and/or arrays nested too deep (%d) for save_object" TOO_DEEP_SAVE_HINT "\n", \
|
||||
MAX_SAVE_SVALUE_DEPTH);
|
||||
|
||||
object_t* previous_ob;
|
||||
|
||||
|
|
|
|||
144
testsuite/single/tests/efuns/break_cycles.lpc
Normal file
144
testsuite/single/tests/efuns/break_cycles.lpc
Normal file
|
|
@ -0,0 +1,144 @@
|
|||
// break_cycles(mixed): clears every reference loop in the value, in place,
|
||||
// by zeroing the back-edge slots (deleting the node when the back-edge is a
|
||||
// mapping KEY). Returns the number of edges broken. Everything that is not
|
||||
// part of a loop -- including DAG sharing -- must be left untouched, and the
|
||||
// value must afterwards be save/copy/print-safe again.
|
||||
|
||||
class Node {
|
||||
mixed data;
|
||||
mixed next;
|
||||
}
|
||||
|
||||
private void run_checks() {
|
||||
#if efun_defined(break_cycles)
|
||||
mixed *a, *ring1, *ring2, *ring3, *shared, *dag;
|
||||
mapping m1, m2;
|
||||
class Node n1, n2;
|
||||
function f, g;
|
||||
object clone;
|
||||
|
||||
// non-compound and acyclic values: nothing to do
|
||||
ASSERT_EQ(0, break_cycles(42));
|
||||
ASSERT_EQ(0, break_cycles(({ 1, ({ 2 }), ([ "k": 3 ]) })));
|
||||
|
||||
// self-referential array
|
||||
a = ({ "keep", 0 });
|
||||
a[1] = a;
|
||||
ASSERT_EQ(1, break_cycles(a));
|
||||
ASSERT_EQ("keep", a[0]); // non-loop content untouched
|
||||
ASSERT_EQ(0, a[1]); // the back-edge is now 0
|
||||
ASSERT_EQ(0, has_cycle(a));
|
||||
#if efun_defined(refs)
|
||||
ASSERT_EQ(1, refs(a)); // only the local variable holds it
|
||||
#endif
|
||||
ASSERT_EQ(0, catch(save_variable(a))); // saving works again
|
||||
ASSERT_EQ(0, catch(copy(a))); // deep copy works again
|
||||
|
||||
// DAG sharing is preserved, not treated as a loop
|
||||
shared = ({ 1 });
|
||||
dag = ({ shared, shared });
|
||||
ASSERT_EQ(0, break_cycles(dag));
|
||||
ASSERT(dag[0] == shared && dag[1] == shared); // identity intact
|
||||
|
||||
// mutual mapping cycle: exactly one edge cut, the rest intact
|
||||
m1 = ([ "name": "m1" ]);
|
||||
m2 = ([ "name": "m2" ]);
|
||||
m1["peer"] = m2;
|
||||
m2["peer"] = m1;
|
||||
ASSERT_EQ(1, break_cycles(m1));
|
||||
ASSERT_EQ(0, has_cycle(m1));
|
||||
ASSERT_EQ("m1", m1["name"]);
|
||||
ASSERT_EQ("m2", m2["name"]);
|
||||
// one of the two peer edges survives, the other is 0
|
||||
ASSERT((m1["peer"] == m2 && !m2["peer"]) || (m2["peer"] == m1 && !m1["peer"]));
|
||||
|
||||
// ring of three arrays: one cut un-loops the whole ring
|
||||
ring1 = ({ 0 });
|
||||
ring2 = ({ 0 });
|
||||
ring3 = ({ 0 });
|
||||
ring1[0] = ring2;
|
||||
ring2[0] = ring3;
|
||||
ring3[0] = ring1;
|
||||
ASSERT_EQ(1, break_cycles(ring1));
|
||||
ASSERT_EQ(0, has_cycle(ring1));
|
||||
|
||||
// circular class list
|
||||
n1 = new(class Node, data: 1);
|
||||
n2 = new(class Node, data: 2);
|
||||
n1.next = n2;
|
||||
n2.next = n1;
|
||||
ASSERT_EQ(1, break_cycles(n1));
|
||||
ASSERT_EQ(0, has_cycle(n1));
|
||||
ASSERT(n1.next == n2); // tree edge kept, back-edge cut
|
||||
ASSERT_EQ(0, n2.next);
|
||||
|
||||
// cycle closed in mapping-KEY position: the node is deleted
|
||||
m1 = ([]);
|
||||
m2 = ([]);
|
||||
m1[m2] = "v1";
|
||||
m2[m1] = "v2";
|
||||
ASSERT_EQ(1, break_cycles(m1));
|
||||
ASSERT_EQ(0, has_cycle(m1));
|
||||
ASSERT_EQ(1, sizeof(m1) + sizeof(m2)); // exactly one node was deleted
|
||||
|
||||
// cycle through a function pointer's captured arguments: the captured
|
||||
// slot inside the args list is zeroed, the funptr itself survives
|
||||
a = ({ 0 });
|
||||
f = (: member_array, 0, a :);
|
||||
a[0] = f;
|
||||
ASSERT_EQ(1, break_cycles(a));
|
||||
ASSERT_EQ(0, has_cycle(a));
|
||||
ASSERT(functionp(a[0]));
|
||||
|
||||
// bind() to a DIFFERENT owner creates a new funptr that SHARES the old
|
||||
// one's args array, so a loop can close on the funptr->args edge itself
|
||||
// (no svalue slot to zero): the bound funptr's args list is detached and
|
||||
// replaced with a zero-filled one instead. Regression: this used to be
|
||||
// silently unbreakable (break_cycles() returned 0, has_cycle() stayed 1).
|
||||
// (bind() to the SAME owner is a no-op that returns the original funptr,
|
||||
// which is why the clone is needed to construct the shared-args pair.)
|
||||
a = ({ 0 });
|
||||
f = (: member_array, 0, a :); // f.args = ({ 0, a })
|
||||
clone = new(__FILE__);
|
||||
g = bind(f, clone); // g is a new funptr sharing f's args
|
||||
ASSERT(g != f);
|
||||
a[0] = g; // args -> a -> g -> args : the loop
|
||||
ASSERT_EQ(1, has_cycle(f));
|
||||
ASSERT_EQ(1, break_cycles(f));
|
||||
ASSERT_EQ(0, has_cycle(f));
|
||||
ASSERT_EQ(0, has_cycle(g));
|
||||
ASSERT(functionp(a[0])); // g itself survives, args detached
|
||||
destruct(clone);
|
||||
|
||||
// values containing DESTRUCTED objects: walked as leaves and rendered
|
||||
// in paths without touching freed memory, freed normally on break
|
||||
clone = new(__FILE__);
|
||||
m1 = ([ clone: ({ 0 }) ]);
|
||||
m1[clone][0] = m1; // loop through the value under an object key
|
||||
destruct(clone);
|
||||
ASSERT_EQ(1, has_cycle(m1));
|
||||
#if efun_defined(find_cycles)
|
||||
ASSERT_EQ(1, sizeof(find_cycles(m1))); // renders the (dead) object key
|
||||
#endif
|
||||
ASSERT_EQ(1, break_cycles(m1));
|
||||
ASSERT_EQ(0, has_cycle(m1));
|
||||
|
||||
// idempotent: nothing left to break
|
||||
ASSERT_EQ(0, break_cycles(a));
|
||||
ASSERT_EQ(0, break_cycles(m1));
|
||||
#endif
|
||||
ASSERT(1);
|
||||
}
|
||||
|
||||
void do_tests() {
|
||||
// unconditional teardown (AGENTS.md section 7): if anything above errors
|
||||
// uncaught mid-test, locally-built cycles unwind into orphans that the
|
||||
// harness's post-file check_memory() now hard-fails on -- collect them
|
||||
// so one regression stays one [ FAILED ] entry instead of aborting the
|
||||
// whole randomized run
|
||||
string err = catch(run_checks());
|
||||
#if efun_defined(find_orphaned_cycles)
|
||||
find_orphaned_cycles(1);
|
||||
#endif
|
||||
if (err) error(err);
|
||||
}
|
||||
73
testsuite/single/tests/efuns/find_cycles.lpc
Normal file
73
testsuite/single/tests/efuns/find_cycles.lpc
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
// find_cycles(mixed): index paths (one string per back-edge) locating every
|
||||
// slot that closes a reference loop. Array/class paths are deterministic;
|
||||
// mapping paths depend on internal table order, so those are checked by
|
||||
// count and prefix only.
|
||||
|
||||
class Node {
|
||||
mixed data;
|
||||
mixed next;
|
||||
}
|
||||
|
||||
private void run_checks() {
|
||||
#if efun_defined(find_cycles)
|
||||
mixed *a, *b, *two;
|
||||
mapping m;
|
||||
class Node n;
|
||||
string *paths;
|
||||
|
||||
// acyclic values report nothing
|
||||
ASSERT_EQ(({}), find_cycles(0));
|
||||
ASSERT_EQ(({}), find_cycles(({ 1, ({ 2 }) })));
|
||||
|
||||
// self-referential array: the back-edge is the slot itself
|
||||
a = ({ "x", 0 });
|
||||
a[1] = a;
|
||||
ASSERT_EQ(({ "[1]" }), find_cycles(a));
|
||||
a[1] = 0; // break before reassigning, or the old array is leaked
|
||||
|
||||
// nested: path walks through the outer array
|
||||
b = ({ ({ 0 }) });
|
||||
b[0][0] = b;
|
||||
ASSERT_EQ(({ "[0][0]" }), find_cycles(b));
|
||||
b[0][0] = 0;
|
||||
|
||||
// two independent loops: one path each
|
||||
a = ({ 0 });
|
||||
a[0] = a;
|
||||
b = ({ 0 });
|
||||
b[0] = b;
|
||||
two = ({ a, b });
|
||||
ASSERT_EQ(({ "[0][0]", "[1][0]" }), find_cycles(two));
|
||||
|
||||
// class ring: field index in declaration order (next == field 1)
|
||||
n = new(class Node, data: 1);
|
||||
n.next = n;
|
||||
ASSERT_EQ(({ ".1" }), find_cycles(n));
|
||||
|
||||
// mapping value cycle: exactly one back-edge, key rendered in the path
|
||||
m = ([]);
|
||||
m["self"] = m;
|
||||
paths = find_cycles(m);
|
||||
ASSERT_EQ(({ "[\"self\"]" }), paths);
|
||||
|
||||
// break everything before returning
|
||||
a[0] = 0;
|
||||
b[0] = 0;
|
||||
n.next = 0;
|
||||
map_delete(m, "self");
|
||||
#endif
|
||||
ASSERT(1);
|
||||
}
|
||||
|
||||
void do_tests() {
|
||||
// unconditional teardown (AGENTS.md section 7): if anything above errors
|
||||
// uncaught mid-test, locally-built cycles unwind into orphans that the
|
||||
// harness's post-file check_memory() now hard-fails on -- collect them
|
||||
// so one regression stays one [ FAILED ] entry instead of aborting the
|
||||
// whole randomized run
|
||||
string err = catch(run_checks());
|
||||
#if efun_defined(find_orphaned_cycles)
|
||||
find_orphaned_cycles(1);
|
||||
#endif
|
||||
if (err) error(err);
|
||||
}
|
||||
111
testsuite/single/tests/efuns/find_orphaned_cycles.lpc
Normal file
111
testsuite/single/tests/efuns/find_orphaned_cycles.lpc
Normal file
|
|
@ -0,0 +1,111 @@
|
|||
// find_orphaned_cycles(int collect): Debug-build (DEBUGMALLOC_EXTENSIONS)
|
||||
// garbage detector for reference loops that have already been dropped --
|
||||
// data blocks kept alive only by a loop, unreachable from any variable, so
|
||||
// no LPC-level tool (refs(), break_cycles()) can reach them anymore.
|
||||
// Detection is trial deletion: a block whose every reference comes from
|
||||
// other data blocks, with no path from an externally-held block, is
|
||||
// garbage. With collect=1 the garbage is reclaimed safely.
|
||||
//
|
||||
// Counts are asserted relative to a baseline snapshot so the test is
|
||||
// insensitive to whatever the surrounding suite state holds. If this test
|
||||
// itself failed to clean up, the harness's post-file check_memory() would
|
||||
// now also flag the orphans ("unreachable data block(s)") -- that report
|
||||
// line is new alongside this efun.
|
||||
|
||||
class Ring {
|
||||
mixed next;
|
||||
}
|
||||
|
||||
void make_orphans() {
|
||||
// in a helper so its locals are provably off the stack when we scan
|
||||
mixed *a;
|
||||
mapping m1, m2;
|
||||
function f;
|
||||
class Ring c1, c2;
|
||||
|
||||
// self-referential array, dropped: 1 orphaned block. The buffer riding
|
||||
// inside is NOT cycle-capable (not counted), but collection must release
|
||||
// it via the sever pass -- a buffer leak here trips the harness's
|
||||
// post-file ref-count gate.
|
||||
a = ({ 0, allocate_buffer(64) });
|
||||
a[0] = a;
|
||||
a = 0;
|
||||
|
||||
// mutual mapping pair (value edges), dropped: 2 orphaned blocks
|
||||
m1 = ([]);
|
||||
m2 = ([]);
|
||||
m1["peer"] = m2;
|
||||
m2["peer"] = m1;
|
||||
m1 = 0;
|
||||
m2 = 0;
|
||||
|
||||
// mutual mapping pair closed in KEY position, dropped: 2 orphaned
|
||||
// blocks -- exercises the collector's key-slot sever path (zeroing a
|
||||
// node's key in place), which break_cycles() never uses
|
||||
m1 = ([]);
|
||||
m2 = ([]);
|
||||
m1[m2] = "v1";
|
||||
m2[m1] = "v2";
|
||||
m1 = 0;
|
||||
m2 = 0;
|
||||
|
||||
// circular class list, dropped: 2 orphaned blocks (TAG_CLASS candidate,
|
||||
// hold, sever, and free_class release paths)
|
||||
c1 = new(class Ring);
|
||||
c2 = new(class Ring);
|
||||
c1.next = c2;
|
||||
c2.next = c1;
|
||||
c1 = 0;
|
||||
c2 = 0;
|
||||
|
||||
// array -> funptr -> captured-args -> array ring, dropped: 3 blocks
|
||||
// (the array, the function pointer, and its args array)
|
||||
a = ({ 0 });
|
||||
f = (: member_array, 0, a :);
|
||||
a[0] = f;
|
||||
a = 0;
|
||||
f = 0;
|
||||
}
|
||||
|
||||
private void run_checks() {
|
||||
#if efun_defined(find_orphaned_cycles)
|
||||
int base, n;
|
||||
mixed *live;
|
||||
|
||||
base = find_orphaned_cycles(0); // whatever the suite already leaked
|
||||
|
||||
make_orphans();
|
||||
n = find_orphaned_cycles(0); // detect only, nothing reclaimed
|
||||
ASSERT_EQ(base + 10, n);
|
||||
ASSERT_EQ(base + 10, find_orphaned_cycles(0)); // detection is idempotent
|
||||
|
||||
// a loop still reachable from a variable is NOT orphaned garbage
|
||||
live = ({ 0 });
|
||||
live[0] = live;
|
||||
ASSERT_EQ(base + 10, find_orphaned_cycles(0));
|
||||
|
||||
// collect: our 6 blocks (plus any pre-existing orphans) are reclaimed
|
||||
ASSERT_EQ(base + 10, find_orphaned_cycles(1));
|
||||
ASSERT_EQ(0, find_orphaned_cycles(0));
|
||||
|
||||
// the reachable loop survived the collection intact
|
||||
ASSERT(live[0] == live);
|
||||
#if efun_defined(break_cycles)
|
||||
break_cycles(live);
|
||||
#else
|
||||
live[0] = 0;
|
||||
#endif
|
||||
#endif
|
||||
ASSERT(1);
|
||||
}
|
||||
|
||||
void do_tests() {
|
||||
// unconditional teardown (AGENTS.md section 7): an uncaught error after
|
||||
// make_orphans() would otherwise strand the orphans, and the harness's
|
||||
// post-file check_memory() now hard-fails on them
|
||||
string err = catch(run_checks());
|
||||
#if efun_defined(find_orphaned_cycles)
|
||||
find_orphaned_cycles(1);
|
||||
#endif
|
||||
if (err) error(err);
|
||||
}
|
||||
97
testsuite/single/tests/efuns/has_cycle.lpc
Normal file
97
testsuite/single/tests/efuns/has_cycle.lpc
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
// has_cycle(mixed): 1 if the value's reference graph contains a loop.
|
||||
// The walker is iterative, so unlike save_variable()/copy() it has no
|
||||
// depth cap -- deep acyclic nesting must scan clean, not error.
|
||||
|
||||
class Node {
|
||||
mixed data;
|
||||
mixed next;
|
||||
}
|
||||
|
||||
private void run_checks() {
|
||||
#if efun_defined(has_cycle)
|
||||
mixed *a, *deep, *shared;
|
||||
mapping m1, m2;
|
||||
class Node n1, n2;
|
||||
function f;
|
||||
int i;
|
||||
|
||||
// leaves and acyclic structures
|
||||
ASSERT_EQ(0, has_cycle(0));
|
||||
ASSERT_EQ(0, has_cycle("string"));
|
||||
ASSERT_EQ(0, has_cycle(({ 1, ({ 2, ({ 3 }) }), ([ "k": ({ 4 }) ]) })));
|
||||
|
||||
// DAG sharing (the same array referenced twice) is NOT a cycle
|
||||
shared = ({ 1 });
|
||||
ASSERT_EQ(0, has_cycle(({ shared, shared, ([ "k": shared ]) })));
|
||||
|
||||
// self-referential array, directly and one level down
|
||||
a = ({ 0 });
|
||||
a[0] = a;
|
||||
ASSERT_EQ(1, has_cycle(a));
|
||||
ASSERT_EQ(1, has_cycle(({ "wrapper", a })));
|
||||
a[0] = 0;
|
||||
ASSERT_EQ(0, has_cycle(a));
|
||||
|
||||
// mutual mapping cycle
|
||||
m1 = ([]);
|
||||
m2 = ([]);
|
||||
m1["peer"] = m2;
|
||||
m2["peer"] = m1;
|
||||
ASSERT_EQ(1, has_cycle(m1));
|
||||
ASSERT_EQ(1, has_cycle(m2));
|
||||
map_delete(m1, "peer");
|
||||
ASSERT_EQ(0, has_cycle(m1));
|
||||
ASSERT_EQ(0, has_cycle(m2));
|
||||
|
||||
// cycle where a mapping is used as a KEY of another mapping
|
||||
m1 = ([]);
|
||||
m2 = ([]);
|
||||
m1[m2] = 1;
|
||||
m2[m1] = 1;
|
||||
ASSERT_EQ(1, has_cycle(m1));
|
||||
map_delete(m1, m2);
|
||||
ASSERT_EQ(0, has_cycle(m2));
|
||||
map_delete(m2, m1);
|
||||
|
||||
// circular class list
|
||||
n1 = new(class Node, data: 1);
|
||||
n2 = new(class Node, data: 2);
|
||||
n1.next = n2;
|
||||
n2.next = n1;
|
||||
ASSERT_EQ(1, has_cycle(n1));
|
||||
n2.next = 0;
|
||||
ASSERT_EQ(0, has_cycle(n1));
|
||||
|
||||
// cycle through a function pointer's captured arguments
|
||||
a = ({ 0 });
|
||||
f = (: member_array, 0, a :);
|
||||
a[0] = f;
|
||||
ASSERT_EQ(1, has_cycle(a));
|
||||
ASSERT_EQ(1, has_cycle(f));
|
||||
a[0] = 0;
|
||||
ASSERT_EQ(0, has_cycle(f));
|
||||
|
||||
// deep acyclic nesting far beyond MAX_SAVE_SVALUE_DEPTH: the iterative
|
||||
// walker must return 0 cleanly where save_variable() errors out
|
||||
deep = ({ 0 });
|
||||
for (i = 0; i < 5000; i++) {
|
||||
deep = ({ deep });
|
||||
}
|
||||
ASSERT_EQ(0, has_cycle(deep));
|
||||
ASSERT(catch(save_variable(deep)));
|
||||
#endif
|
||||
ASSERT(1);
|
||||
}
|
||||
|
||||
void do_tests() {
|
||||
// unconditional teardown (AGENTS.md section 7): if anything above errors
|
||||
// uncaught mid-test, locally-built cycles unwind into orphans that the
|
||||
// harness's post-file check_memory() now hard-fails on -- collect them
|
||||
// so one regression stays one [ FAILED ] entry instead of aborting the
|
||||
// whole randomized run
|
||||
string err = catch(run_checks());
|
||||
#if efun_defined(find_orphaned_cycles)
|
||||
find_orphaned_cycles(1);
|
||||
#endif
|
||||
if (err) error(err);
|
||||
}
|
||||
128
testsuite/single/tests/operators/reference_loop.lpc
Normal file
128
testsuite/single/tests/operators/reference_loop.lpc
Normal file
|
|
@ -0,0 +1,128 @@
|
|||
// Reference loops (cyclic data structures) and the reference-counting VM.
|
||||
//
|
||||
// FluffOS manages arrays / mappings / classes / buffers / function pointers
|
||||
// purely by reference counting (free_svalue -> dealloc_* at ref 0, see
|
||||
// src/vm/internal/base/svalue.cc); there is NO cycle collector. A structure
|
||||
// that reaches itself keeps its own ref >= 1 forever, so the moment the last
|
||||
// OUTSIDE reference is dropped the whole loop becomes unreachable-but-alive:
|
||||
// memory that can never be reclaimed until the driver shuts down. The leak is
|
||||
// even invisible to the Debug ref-count checker (check_memory), because a
|
||||
// detached cycle is perfectly self-consistent -- every internal reference is
|
||||
// marked by another member of the loop.
|
||||
//
|
||||
// This test pins what IS guaranteed around cycles:
|
||||
// 1. building and traversing them is legal (assignment is pointer copy),
|
||||
// 2. refs() exposes the extra self-reference (the leak-detection tool),
|
||||
// 3. the recursion-guarded consumers survive them: sprintf("%O") truncates
|
||||
// with "...", save_variable()/save_object() and copy() error cleanly at
|
||||
// MAX_SAVE_SVALUE_DEPTH instead of overflowing the C stack,
|
||||
// 4. destruct() breaks any loop that passes through an object's global
|
||||
// variables (destruct2() zeroes the variable block for exactly this
|
||||
// reason -- see src/vm/internal/simulate.cc), and
|
||||
// 5. explicitly overwriting the back-reference really releases the loop.
|
||||
//
|
||||
// Every cycle built below is broken before the test returns: that is the
|
||||
// documented prevention pattern, and it keeps Debug/ASan suite runs
|
||||
// leak-free.
|
||||
|
||||
#include <lpctypes.h>
|
||||
|
||||
class Node {
|
||||
mixed data;
|
||||
mixed next;
|
||||
}
|
||||
|
||||
// fixture state for the destruct demo (only used on clones of this file)
|
||||
mixed stash;
|
||||
void set_stash(mixed v) { stash = v; }
|
||||
mixed query_stash() { return stash; }
|
||||
|
||||
private void run_checks() {
|
||||
mixed *a, *plain;
|
||||
mapping m1, m2;
|
||||
class Node n1, n2;
|
||||
mapping outer;
|
||||
object ob;
|
||||
string err, dump;
|
||||
|
||||
// --- 1. cycles are legal to build and traverse ------------------------
|
||||
a = ({ 0 });
|
||||
a[0] = a; // self-referential array
|
||||
ASSERT(a[0] == a); // == on arrays is pointer identity
|
||||
ASSERT(a[0][0][0] == a); // traversal never ends, one hop at a time
|
||||
|
||||
m1 = ([]);
|
||||
m2 = ([]);
|
||||
m1["peer"] = m2; // mutual mapping cycle
|
||||
m2["peer"] = m1;
|
||||
ASSERT(m1["peer"]["peer"] == m1);
|
||||
|
||||
n1 = new(class Node, data: 1);
|
||||
n2 = new(class Node, data: 2);
|
||||
n1.next = n2; // two-node circular list
|
||||
n2.next = n1;
|
||||
ASSERT(n1.next.next == n1);
|
||||
|
||||
// --- 2. refs() shows the extra self-reference -------------------------
|
||||
#if efun_defined(refs)
|
||||
plain = ({ 0 });
|
||||
ASSERT_EQ(1, refs(plain)); // held only by the local variable
|
||||
ASSERT_EQ(2, refs(a)); // local variable + the cycle's own ref:
|
||||
// dropping `a` leaves ref at 1 forever
|
||||
#endif
|
||||
|
||||
// --- 3. recursion-guarded consumers survive cycles ---------------------
|
||||
// sprintf("%O") truncates self-referential structures with "..."
|
||||
dump = sprintf("%O", a);
|
||||
ASSERT(strsrch(dump, "...") != -1);
|
||||
dump = sprintf("%O", m1);
|
||||
ASSERT(strsrch(dump, "...") != -1);
|
||||
|
||||
// save_variable() hits the MAX_SAVE_SVALUE_DEPTH cap and errors cleanly
|
||||
err = catch(save_variable(a));
|
||||
ASSERT(err && strsrch(err, "too deep") != -1);
|
||||
err = catch(save_variable(m1));
|
||||
ASSERT(err && strsrch(err, "too deep") != -1);
|
||||
|
||||
// copy() (deep copy) also errors at the depth cap instead of recursing
|
||||
// the C stack into the ground
|
||||
err = catch(copy(a));
|
||||
ASSERT(err && strsrch(err, "too deep") != -1);
|
||||
err = catch(copy(n1));
|
||||
ASSERT(err && strsrch(err, "too deep") != -1);
|
||||
|
||||
// --- 4. destruct() breaks loops through object variables ---------------
|
||||
// The driver zeroes an object's global variables at destruct time
|
||||
// precisely so that "an object with a variable pointing to itself would
|
||||
// never be freed" cannot happen. A loop routed through an object is
|
||||
// therefore reclaimable: destruct the object and the loop falls apart.
|
||||
ob = new(__FILE__);
|
||||
outer = ([ "owner": ob ]);
|
||||
ob->set_stash(outer); // ob.stash -> outer -> ob : a cycle
|
||||
ASSERT(ob->query_stash()["owner"] == ob);
|
||||
destruct(ob);
|
||||
ASSERT(!ob); // the object really died...
|
||||
ASSERT(!outer["owner"]); // ...and the loop edge in `outer` is now 0
|
||||
map_delete(outer, "owner");
|
||||
|
||||
// --- 5. prevention: break the loop before dropping it ------------------
|
||||
a[0] = 0; // overwrite the back-reference
|
||||
#if efun_defined(refs)
|
||||
ASSERT_EQ(1, refs(a)); // only the local variable holds it again
|
||||
#endif
|
||||
map_delete(m1, "peer"); // one deleted edge un-loops both mappings
|
||||
n2.next = 0; // same for the circular list
|
||||
}
|
||||
|
||||
void do_tests() {
|
||||
// unconditional teardown (AGENTS.md section 7): if anything above errors
|
||||
// uncaught mid-test, locally-built cycles unwind into orphans that the
|
||||
// harness's post-file check_memory() now hard-fails on -- collect them
|
||||
// so one regression stays one [ FAILED ] entry instead of aborting the
|
||||
// whole randomized run
|
||||
string err = catch(run_checks());
|
||||
#if efun_defined(find_orphaned_cycles)
|
||||
find_orphaned_cycles(1);
|
||||
#endif
|
||||
if (err) error(err);
|
||||
}
|
||||
|
|
@ -334,6 +334,7 @@ void test_encode_circular_references() {
|
|||
m1 = (["key":"value"]);
|
||||
m1["self"] = m1;
|
||||
ASSERT(strsrch(json_encode(m1), "\"self\":null") != -1);
|
||||
map_delete(m1, "self"); // break the loop before dropping m1, or it leaks
|
||||
|
||||
// Circular array reference. Note: `arr1 += ({arr1})` would NOT be
|
||||
// circular (LPC array '+' builds a new array, so the element would
|
||||
|
|
@ -351,6 +352,11 @@ void test_encode_circular_references() {
|
|||
m1["ref"] = m2;
|
||||
// Should encode with null for circular part
|
||||
ASSERT(json_encode(m1));
|
||||
|
||||
// break the loops before the locals go out of scope -- a dropped
|
||||
// reference loop is unreclaimable and now fails check_memory()
|
||||
arr1[3] = 0;
|
||||
map_delete(m1, "ref");
|
||||
}
|
||||
|
||||
void test_roundtrip() {
|
||||
|
|
@ -534,6 +540,7 @@ void test_performance_large_strings() {
|
|||
}
|
||||
|
||||
void do_tests() {
|
||||
string err;
|
||||
if(!find_object("/std/json")) {
|
||||
write("json not loaded, skipped");
|
||||
return;
|
||||
|
|
@ -577,7 +584,14 @@ void do_tests() {
|
|||
test_encode_non_string_keys();
|
||||
write("✓ Non-string key tests passed\n");
|
||||
|
||||
test_encode_circular_references();
|
||||
// catch + unconditional collect: an uncaught error inside would strand
|
||||
// the deliberately-cyclic fixtures as orphans, which the harness's
|
||||
// post-file check_memory() now hard-fails on
|
||||
err = catch(test_encode_circular_references());
|
||||
#if efun_defined(find_orphaned_cycles)
|
||||
find_orphaned_cycles(1);
|
||||
#endif
|
||||
if (err) error(err);
|
||||
write("✓ Circular reference tests passed\n");
|
||||
|
||||
write("\n=== Round-trip Tests ===\n");
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue