fun_u evaluates the attribute body with executor = the object holding
the attribute and caller = the previous executor; the Tier 3 inline
compiled the body into the caller's program without that swap, so
everything executor-derived inside u(#obj/attr) resolved against the
CALLER: %! (SUBST slot), %va-%vz and %=<name> (xget against that slot),
and every ECALL's ec->executor — v(), bare-name u(), name(me), and
permission checks, in both directions. The MAP/FILTER/FOLD inline arms
documented this exact hole and guard it with a runtime executor==thing
STRCMP fallback ("the u()-inline lives with this hole"); #2092's arm
un-reversal then made the u() hole reachable for the first time.
Fix: _PUSH_UEXEC/_POP_UEXEC helpers bracket the inlined body. Push
saves ec->executor/ec->caller to a handle stack, sets caller = old
executor and executor = thing (mirroring fun_u's mux_exec call), and
rewrites the guest %! substitution slot; pop restores all three. Both
executor-derived channels — the ECALL context and the SUBST slot — go
through exactly this pair, so v(), bare-name u() with parent
inheritance, name(me), %!, %va-%vz, %=<name> and See_attr checks all
compose, including across nested inlines (each pair strictly brackets
its body). A failed push (stack exhausted) returns -1 and the lowering
BRANCHES it to the existing fun_u ECALL fallback, which establishes its
own context — exhaustion costs the inline, never correctness.
The helper save stacks (this one and the existing CARGS stack) leaked
their slot when a program was abandoned between save and restore (DBT
decline, error unwind); a top-level run_cached_program entry proves no
outer program holds a live handle, so both stacks reset there. Nested
entries never reset: both entry points have claimed the depth counter
since #2106.
Verified against the interpreter oracle (the [num(#obj)] spelling that
never reaches the inline gate): the issue's repro plus nested inline
(#2 -> #3 -> back), executor restored after the call, ulocal, and the
real-world iter(u(bare-name)) shape all byte-match, jit_handled=5/5.
jit_route_parity_fn TC012 pins the shape in smoke — its trigger context
runs as the test object, so caller != target (#1) makes the
discrimination live on every run. Full suite 35/0.
The farm's 2x +jobs regression (wrong-executor denials flooding the
ECALL error-format path) should collapse with this; if utf8_strlen
under mux_vsnprintf stays hot afterwards it deserves its own issue.
MAP/FILTER/FOLD can now adopt the same pair and drop their conservative
executor==thing gate to open inlining to executor != thing — filed as a
perf follow-up in the arm's comment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Compiled iter() levels are now published in a guest-side loop-context
table (rv_compiler::LOOPCTX_BASE): depth, plus each live level's element
buffer address and 1-based iteration number. At ECALL time an RAII
GuestLoopContext pushes those levels onto the interpreter's
itext[]/inum[]/in_loop stack around the callee, so anything that
evaluates softcode — fun_u's mux_exec, fun_itext, fun_ilev — sees the
composed stack instead of an empty one. %i0 inside u() called from a
compiled iter now answers, and nested programs compose correctly
because each nesting depth owns its own s_vm buffer while the caller's
RAII push stays active for the callee's whole run.
The marshal also retires #2170's remaining compile bails: ilev() and
dynamic-depth itext()/inum() inside compiled levels now lower to the
plain ECALL and are correct at runtime, and constant depths naming
enclosing interpreted iters no longer need the depth adjusted down by
the compiled levels. Constant depths naming levels in THIS program
still resolve at compile time. A nest deeper than the table
(LOOPCTX_MAX_LEVELS = 10) declines the compile rather than publish a
partial stack.
Two codegen subtleties found by live probes, not review:
- The element's table payload must go through rv_load_guest_addr: its
buffer can be an output-frame slot, and storing the tagged constant
raw handed the host an out-of-range guest address (the push then
aborted, and ilev() read an empty stack).
- HIR_LCTX_KEEP (emits no code) references the element at the END of
the level's body, extending its live interval so the slot allocator
cannot recycle the buffer for an inner loop's element or a body
temporary while a callee could still read it through the table.
run_cached_program zeroes the table's depth each run — the VM buffer is
shared across programs, so a loop-free program must publish no levels.
Stale-cache compatibility is not a concern: the code cache is flushed
after a rebuild because the program hashes regenerate.
Verified live on direct-#1 stdin (the JIT-gated context),
jit_handled=5/5: u() callees see %i0/%i1/itext/inum across one and two
compiled levels; ilev() composes; dynamic-depth itext/inum resolve
outer levels including across three-level nests and interpreted
enclosing iters. Full suite 35/0; iter_nest_fn TC004 pins the shapes.
Note for posterity: the first probe battery used word(), which does not
exist in TinyMUX — itext(#-1 ...) atoi'd to 0 and both routes agreed on
the "wrong" answer. extract() is the equivalent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both ECALL paths (builtins and ufuns) now hand guest memory to callees
directly: ecall_arg_or_copy, guest_addr_outlives_ecall's ptr < CARGS_BASE
predicate, and jit_internal_abi_fn are gone, and the fargs arrays are
const UTF8 * (which also removed the mux_exec cast on the ufun path).
The compiled route regains its advantage of not materializing arguments,
per-ECALL LBuf allocation included.
Safe because FUNCTION bodies take const UTF8 * const fargs[] with zero
const_casts, and the mutating command handlers receive FargCopy/FargVec
copies from their wrappers. run_cached_program's fragile-predicate
caveat (#2140/#2139 review) now cites the contract instead of the copy.
Acceptance: smoke TC013 — the #2128 reproduction, asserted 3/3/3 — plus
the full suite (35 passed / 0 failed, jit=yes) and the sidefx_fargs.py
live probes, all green on macOS arm64. Merge remains gated on
make test-poison on a Linux box (see the campaign brief).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes the #2132 conversion: the three remaining inline arms drop the
string tier-2 trio for the integer one. Per element, each arm loses the
two-call SPLIT_TOKEN pair with its cursor round-trips (one SPLIT_STEP,
cursor in a register, next cursor back in a0), the APPEND decimal
plumbing (APPEND_I), and — new in this pass — the CARGS-fit guard's
round-trip: BYTELEN returned its answer as decimal text that the caller
ATOI'd back to compare an integer against 256. rv64_bytelen_i returns
the length in a0; the guard is now a call and a compare.
Conversion shape: each arm's ENTRY GATE now requires the integer trio
(SPLIT_STEP/APPEND_I/BYTELEN_I). With an older blob the arm simply does
not fire and the function takes its generic ECALL fallback — correct,
and cleaner than carrying dual string/int emission paths inside the
arms. FOLD's seed walk (no-base case) also collapses from two calls
plus a conversion to one SPLIT_STEP at cursor zero.
Route verified per the #2132 discipline before any benchmark was read:
all three shapes' compiled programs pulled from the SQLite code cache
and disassembled — JAL targets are rv64_split_step / rv64_append_i /
rv64_bytelen_i, with zero references to the string trio. All three
registrations (softlib export, s_tier2_map, tier2_allowed) were made
together this time.
Measured against the interpreter (macOS arm64, min-of-3, ast=/cached=,
live u-inlined bodies — so the loop win is diluted by real body work):
N=50 N=200 N=800
map 0.62 0.57 0.58 (~1.7x faster)
filter 1.00 0.66 0.54 (up to 1.9x)
fold 0.33 0.28 0.27 (3.5-3.7x)
Functional probes exact: map/filter/fold with default and custom in/out
separators, %b separators, fold with and without a base seed. Full
make test EXPECT_CONFIG="jit=yes": 35 passed, 1 skipped (stubslave, not
configured), 0 failed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The compiled ITER loop lost to the interpreter's C loop per element, and
its per-element cost climbed with N while the interpreter's stayed flat.
Profiling (guest->host block map + ELF symbolization of anonymous JIT
frames) found the mechanism the issue asked for: the tier-2 string ABI.
Every argument crosses as decimal text, so each element paid SIX
integer<->string conversions — ITOA(cursor) caller-side, satoi(cursor)
callee-side twice (elem + next modes), sitoa/ATOI for the next offset,
ITOA/satoi for the append's length and iteration number, sitoa/ATOI for
the new length — digit-proportional loops, executed under DBT expansion,
on numbers (byte cursors, accumulated lengths) whose digit counts grow
with the list. That is both the bulk of per-element cost and the whole
of the residual superlinearity: cost/element ~ a + b*digits(N).
The fix is the calling convention the machine already has. HIR_CALL_T2I
loads TY_INT arguments straight into a0.. from their registers and
TY_STRING arguments as guest addresses, JALs to the blob, and takes the
callee's long return from a0; arguments ride carg[] so liveness, DCE and
copy propagation see them (hir_is_carg_call). val[i]=1 allocates an
output slot passed ahead of the args; HIR_T2I_STR aliases it as the
element string. Two int-native blob entrypoints replace the string
pair: rv64_split_step (ONE call per element instead of two — element
written to out, next cursor returned in a0) and rv64_append_i. The next
cursor stays an SSA value stored in the latch, preserving the
nested-iter safety the string route had.
Measured (macOS arm64, min-of-5, ast=/cached= per CLAUDE.md):
iter(lnum(N),1) us/element ratio vs interpreter
N before after before after
200 0.117 0.030 0.75 0.19
1000 0.125 0.030 0.81 0.20
4000 0.145 0.033 0.89 0.21
4x on the compiled path; the loop that opened the issue LOSING 1.42x on
Linux is 5x FASTER than the interpreter here. Fixed-width N-pair probes
no longer climb (median ratio 0.90, was 1.09 at 10/10 above 1.0), and
the per-char slope halves (one scan per element, not two). MAP/FILTER
still compose the string route; converting them is the follow-up.
Two silent-fallback traps found en route, each now carrying a warning:
- tier2_allowed()'s allowlist quietly vetoed the new names: lookup
returned 0, the lowering kept its graceful string fallback, and every
test passed while the fix did not run. Ground truth came from
disassembling the compiled program out of the SQLite code cache — the
JAL targets do not lie. The allowlist comment now names this failure
shape.
- A same-mtime-second edit after a build produced an engine that had the
new symbols in source and nowhere in the binary (#2118's genre).
Also rides along: the env-gated profiling diagnostics that found this —
TINYMUX_DBT_MAP (guest-pc -> host-address lines at translate time) and
TINYMUX_DBT_CODEDUMP (code_buf + block-cache table at cleanup), which
together let a sampling profiler's anonymous JIT frames be symbolized
against the blob ELF.
Full make test EXPECT_CONFIG="jit=yes": 35 passed, 1 skipped (stubslave,
not configured), 0 failed — including smoke's 1660 goldens, the jit
parity suites, and the format guard. Functional probes cover nested
iter, #@/##, custom in/out separators, runtime leading spaces, and empty
lists, all exact.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A bail_noop memo that outlives a softcode/builtin registration can
refuse forever without re-fetch. Invalidate with the #2068 gate epoch
(#2140 review).
Three self-inflicted costs in how the JIT decides what is resident, all
invisible to single-expression benchmarks and all paid by the workload
shape live servers actually have (~1500 distinct programs at a ~100%
switch rate, per the #2129 profile measurement):
1. A declined shape never touches SQLite again. jit_eval fetched and
fully deserialized a compiled program in order to read four integers
and decline it (bail_noop) — ~19.4us per evaluation of pure waste for
get/v/u-shaped expressions once they fell out of the memory cache
(#2130). The verdict is a pure function of the code, so it is now
memoized by compile-cache key and refused before any cache machinery
runs. Dep-free programs only; cleared with the other caches on
jitstats(flush); noop_memo counts memo refusals (also counted in
bail_noop for dashboard continuity).
2. The in-memory compiled-program cache no longer cliffs at 256. One
expression past the old fixed capacity took the hit rate from 99.9%
to 0.16% and wall +56% (#2130) — round-robin is an LRU's worst case.
Capacity is now jit_compile_cache_max (default 2048, clamped [8,65536]
at use, read at insert time so runtime @admin applies immediately).
3. Slot admission resists cold storms (Kagura's #2139 review finding).
The strict-LRU slot claim helped working sets up to the slot count and
then cliffed: round-robin one past it evicted the entry needed next on
every evaluation — measured ~8% WORSE than the old reset. Slot 0 is
now a probation lane: first-touch and pinned programs run there and
cannot displace a hot resident; protected slots 1+ are evictable only
when cold (hot bit unset across a 64-miss sweep window). slot_evict
now counts protected displacements only — the raise-the-slot-count
signal — with probation churn split out as slot_churn0.
Measured (macOS arm64, 2000 commands round-robin, wall us/command):
distinct slots=1 slots=7 old policy @ K=8: 8% worse
1 19.9 17.4 1.14x new policy @ K=8: 1.60x BETTER
2 53.6 20.7 2.59x
4 49.6 20.7 2.39x skew 4 hot/32 cold (80/20):
6 49.1 20.8 2.36x 49.1 -> 24.6 us/cmd (2.0x)
8 49.2 30.8 1.60x
16 49.4 43.2 1.14x decline memo steady state:
32 49.4 45.5 1.09x memo+1500, sqlite+0
(fetch tier eliminated)
Never worse than the old reset at any size measured; converges to the
old cost from below instead of crossing it.
Also pays two documentation debts from the #2139 review: the read-only
string-pool claim now names its dependency on #2135's ECALL argument
copy (narrowing that predicate reintroduces #2128 silently through the
materialize-skip), and CHANGES states the slot-count regime instead of
an unqualified speedup.
tests/scenario/jit_alternation.py grows from 13 to 19 checks: a
jitstats(flush) clean-slate preamble (deterministic on aged servers), a
cold-storm phase asserting protected residents survive (slot_evict +0,
slot_miss +0 for the hot set afterward) while probation churns, and a
decline-memo phase asserting the second round of declined shapes reaches
neither SQLite nor the compiler. The driver also raises the command
quota itself: a burst phase behind default pacing left the closing
jitstats probe timing out, and empty stats read as +0 deltas — a false
PASS shape for check 18 and a false FAIL for check 17.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
run_cached_program kept translated blocks for exactly one program: a
single uint64_t remembered which program the DBT was last set up for, and
any other program forced dbt_reset + full re-translation. Two alternating
expressions re-translated on every evaluation — a fixed +22..76us per
command on the issue's box — and every JIT benchmark in the tree repeats a
single expression, so none of them could see it. Real command streams are
nothing but alternation; worse, the compiled unit is the WHOLE evaluated
argument (literal prefixes included), so even one command template with
varying literals is a stream of distinct programs.
The collision is at guest PCs: every cached program is compiled at the
same canonical base, so two programs occupy the same addresses with
different bytes. Rather than tagging the block cache (its 16-byte entry
and inline host-code lookup are load-bearing on three backends) or keeping
N whole DBT contexts (N copies of the blob translation), this makes the
guest PC itself the disambiguator — the same model the shared heap and
persistent_vm already run: each program materializes its CODE into its own
16 KB guest slot, and the PC-keyed cache holds every resident program at
once. Only code moves; str/fargs DATA keeps swapping at canonical
addresses, because data content never invalidates a translation.
Generated code turns out to be one relocation away from position-free:
internal control flow is PC-relative, data references are absolute into
regions that do not move, there is no JALR or AUIPC, and the only
position-dependent bytes are the blob-call JALs. Those are re-aimed at
materialize time by a linear decode of the pure 4-byte instruction stream
— no relocation records. A program that ever fails that scan is PINNED
to the canonical slot, which is exactly the old behaviour, scoped to
exactly those programs (today: none; slot_pinned going nonzero means a
new lowering quietly lost slotting).
Slot bases live in the two spans this arena leaves unallocated (0x40000-
0x50000, 0x60000-0x68000), all within JAL range of the blob. Slot reuse
does a range-scoped eviction (dbt_invalidate_guest_range) mirroring
dbt_reset's preserve-blob discipline; code-buffer exhaustion self-heals
through the existing reclaim, since residency means guest bytes, not
translations, and translations rebuild lazily on cache miss. rvbench's
foreign binding releases the slots it invalidates.
jit_code_slots (default 7, runtime @admin) clamps the working set;
1 restores the old single-program behaviour as the A/B lever. Measured
on macOS arm64, 2000 commands round-robin, wall us/command:
distinct slots=1 slots=7
1 25.2 21.5 1.17x
2 60.7 21.4 2.84x
4 57.3 24.7 2.32x
6 51.4 25.7 2.00x
slots=7 is flat across the sweep with slot_miss=0; slots=1 shows the old
per-switch tax with slot_miss == every evaluation.
Tests: tests/dbt/test_reloc checks the JAL codec against golden words
from riscv64-unknown-elf-as and verifies re-aimed targets at every slot
base; tests/scenario/jit_alternation.py drives a live server and asserts
the mechanism (exact results across alternation and eviction, slot_miss
+0 within the slot count, dbt_code_used flat, the knob restoring the old
behaviour). Also fixed in passing: dbt_configure_trace_from_env is no
longer re-read on every program switch, since the switch no longer resets.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A cached compiled program could return a WRONG value from its second
evaluation onward: first run right, every later one wrong, no error and no
counter. Signature was 3/1/1 for
`[words(map(obj/B.CONST,1 2 3))]` called three times.
MUX builtins may destructively tokenize their arguments -- split_token()
writes terminating NULs in place, and the interpreter relies on it, handing
each callee a fresh evaluation buffer it owns. The compiled route passed
pointers straight into guest memory (`fargs[i] = ec->memory + ptr`), so an
argument in a region that OUTLIVES the call was scribbled on permanently:
the program's own constant became `1`, and every later evaluation read the
truncated value.
The region that matters is the shared heap's string pool at 0x40000. A
nested u() runs at depth > 1, which jit_eval routes to the shared heap,
whose pools persist across evaluations by design -- so its constants are
exactly the ones a destructive callee turns into permanent garbage.
The fix copies an argument out of guest memory when BOTH hold:
1. It lives below CARGS_BASE -- compile-time data that outlives the call
across both arenas: one-shot CODE/FARGS/STR (0x0-0x10000), the Tier 2
blob (0x10000-0x40000), the shared heap's string pool
(0x40000-0x60000) and fargs pool (0x60000-0x68000). Everything at or
above CARGS_BASE is rebuilt per evaluation -- run_cached_program
repopulates CARGS/SUBST, clears output slots, the heap cursor resets --
so computed arguments still cost nothing.
2. The callee is not one of the JIT's own ABI helpers. The `_`-prefixed
CA_GOD entries (_WRITE_CARG, _SAVE_CARGS, _RESTORE_CARGS) take a guest
POINTER by address, not a string by value: fun__write_carg converts
fargs[1] back with `fargs[1] - ec->memory` and rejects anything outside
the guest image. Copying breaks them silently -- the carg is never
written and every %0/%1 arrives empty.
Both conditions are required, and each was found by a test rather than by
reading. Bounding at BLOB_LIMIT instead of CARGS_BASE misses the shared
heap by 0x38 bytes; copying unconditionally broke TC010/TC011 with
`plain=0`.
So the boundary copy the issue proposes ("ends the whole class") is not
correct as stated: "ECALL args are the callee's to scribble on" holds for
ordinary builtins and not for the JIT's internal ABI.
Chosen over patching the destructive callees because the contract is real --
in-place tokenization is by design, and fixing fun_map/fun_fold/fun_filter
individually leaves the invariant quietly false for the next one written.
Per-call std::vector<LBuf> scratch, so nested ECALLs (fun_map calls back
into mux_exec, which can ECALL again) each own their copies and cannot
invalidate an outer frame's arguments.
Cost, one binary two arms (temporary MUX_NO_ARGCOPY, removed), best of 5
over 4000 evaluations per shape: deltas run -0.325 to +0.100 us/eval and
straddle zero with inconsistent sign. Below the measurement floor; bounded
under ~0.33 us/eval, not claimed to be free. Expected, since the copy only
fires for compile-time-constant arguments.
TC013 in map_fn.mux guards it, asserting three consecutive evaluations --
two cannot distinguish "stable" from "corrupted identically each run". Its
header records the reproduction trap: the probe MUST use u(name/attr),
because u(#dbref/attr) inlines the body (#2092) and never creates the
cached program that gets corrupted, so a literal-dbref probe passes while
the bug is live. That cost the first reproduction attempt.
make test: 36 targets — 34 passed, 2 skipped, 0 failed, clean build.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Item 5. Every JIT fallback returns the RIGHT answer, so a measurement
that never reached the JIT is indistinguishable from one that did -- and
benchmark() is the instrument used to compare an --enable-jit build
against one without, precisely because it times mux_exec itself and is
not JIT-gated. A "JIT arm" that declined everything would report a
uniform tax and look exactly like a real result.
benchmark(add(1,2),100) 0.000264231
benchmark(add(1,2),100,1) 0.000119 jit_handled=100/100 jit_attempts=100
benchmark(citer(a b c,1),100,1) 0.00135223 jit_handled=0/100 jit_attempts=100
The third line is the point: a plausible time with jit_handled=0 beside
it. #2068's re-measurement was only known not to be that because the
counters were checked BY HAND alongside it. This makes the evidence come
back with the number instead of depending on someone remembering.
#1417 and #1519 are the same shape going unnoticed.
Seconds stay FIRST and unchanged, so the two-argument contract holds and
callers doing arithmetic on the leading token keep working.
Wizard-gated. benchmark() is CA_PUBLIC while jitstats(), astbench() and
rvbench() are all CA_WIZARD; reporting the same internals through the
public function would be a quiet permission downgrade.
Without TINYMUX_JIT it reports jit_handled=n/a rather than 0 -- "no JIT
to decline" and "a live JIT declining" are different findings and must
not render identically.
functions.cpp cannot include dbt_compile.h (it collides with
color_ops.h's C-linkage declarations), so the counters come through a
narrow jit_eval_counters() accessor rather than exporting s_jit_stats.
Tests
------------------------------------------------------------------
benchmark_fn.mux TC005 handled==attempts for a shape the JIT takes,
handled=0 for one it declines, both still
returning a time. tr.done moves to it.
jit_perms.py 6,7 a real mortal may use the two-argument form
and is DENIED the third. Smoke cannot test
this: muxscript runs as #1, and a thing owned
by a wizard counts as wizard.
Negative control -- deliberately a LYING field rather than a missing one:
report handled==iterations unconditionally, and TC005 fails
TC005 ... Failed (declined=0.000263461 jit_handled=10/10 jit_attempts=10)
A test that only checked the field was PRESENT would have passed that.
Given this issue is about instruments that report confidently and
wrongly, an assertion that could not tell a truthful field from a
fabricated one would have been the same bug one level up.
make test: 36 targets -- 36 passed, 0 skipped, 0 failed.
make test-scenario: jit-perms 7 passed, 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
persistent_vm_t bump-allocates its code heap upward from 0x0004 across
many compilations, and placed its string and fargs pools at the
rv_compiler ONE-SHOT constants -- STR_BASE 0x8000, FARGS_BASE 0x4000.
Those sit inside the range the code heap grows through, so compiled code
would eventually be written over string constants.
Nothing bounded it. install_code() memcpy'd at the cursor with no check
at all, and the only bound on the path lived in compile(), which
install_code() does not go through. arena_nearly_full() then called the
code heap's capacity BLOB_BASE - 0x0004 (64 KB), which is around four
times what it actually owned, so the 7/8 watermark could not fire for
code pressure before the overlap.
The fix is not to bound the code heap against the pools but to move the
pools out, because there is no reason for them to be there. Each arena
owns a SEPARATE MEM_SIZE image -- persistent_vm_t, shared_heap_t, and
each jit_run_vm::buffer -- so reusing addresses across arenas is free and
only overlap within one image matters. persistent_vm_t was inheriting
the one-shot layout for nothing.
code 0x00004 .. 0x10000 bump-allocated (unchanged)
blob 0x10000 .. 0x40000 installed once (unchanged)
str 0x40000 .. 0x48000 (32 KB) was 0x8000
fargs 0x48000 .. 0x4C000 (16 KB) was 0x4000
They land in the span left unallocated between the blob and LUA_ARRAY,
so nothing else moves. With the pools elsewhere, arena_nearly_full()'s
code_cap becomes correct rather than an overstatement -- that half of the
defect fixes itself.
Four static_asserts pin the invariants that actually matter here: the
code heap not reaching the blob, the pools sitting above it, str abutting
fargs, and the pools staying clear of LUA_ARRAY_BASE.
Also guards run() against entry_pc == 0. install_code() now returns 0 on
overflow and compile() already did, and PC=0 is the reserved "no entry"
sentinel the code heap starts at 0x0004 to keep free -- so running it
would have executed whatever sits at guest address 0. Found by checking
the caller rather than assuming the sentinel was handled.
Latent rather than live: the only driver is fun_pocvm2 (CA_WIZARD), which
compiles a fixed set once per process, so the cursor never approached the
pools. The struct carries compile_attr() and an attr_cache keyed by
(obj, attr_num) though, so the overlap becomes reachable the moment
anything drives it with real attributes -- and the symptom would be
compiled code silently overwriting string constants, with no counter
moving.
make test: 36 targets — 34 passed, 2 skipped, 0 failed, clean build.
pocvm2_fn TC001-TC003 pass, including the re-entrant case that exercises
install_code() and the code heap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
run_compiled() -- the route fun_rvbench uses, bypassing jit_eval -- took
s_vm[0] unconditionally and never touched s_run_cached_depth. The comment
justified it with "is not reached from a nested run": true, and beside the
point. It MAKES one.
A compiled program whose ECALL re-enters mux_exec per element (fun_map,
fun_filter) lands in run_cached_program, which picks its context with
jit_run_vm *vm = &s_vm[s_run_cached_depth];
reads a depth still at 0 because run_compiled never claimed it, selects
that same s_vm[0], and calls get_dbt() on it -- and get_dbt on a ready
context calls dbt_reset(), rebinding the DBT's guest memory from the outer
program's prog.memory to the context's own buffer while the outer
dbt_run()'s frames are executing out of it.
That is precisely the hazard the comment above run_cached_program's own
slot pick describes, and RunDepthGuard is the lock that prevents it. Only
one of the two entry points into the DBT was taking it.
Instrumented on Linux/aarch64, master before the fix -- every nested run
during an rvbench lands on the outer context:
PROBE run_compiled: depth=0 slot=0 prog_mem=0x7f9b9b7800 buffer=0x7f9c2f9800
PROBE nested rcp: depth=0 slot=0 *** SAME SLOT AS run_compiled *** (x5)
and after:
PROBE run_compiled: depth=0 slot=0 prog_mem=0x7f7adb7800 buffer=0x7f7b6f9800
PROBE nested rcp: depth=1 slot=1 buffer=0x7f7a333800 (x5)
Nested runs now take their own context and buffer; the outer run's DBT is
left bound to prog.memory for the whole of its run.
Fix: claim the context. run_compiled now bounds-checks the depth, selects
s_vm[s_run_cached_depth] (s_vm[0] at top level, as before), and holds
RunDepthGuard for the run. RunDepthGuard moves to file scope because both
DBT entry points need it -- that it was a local struct inside
run_cached_program is arguably how the other caller came to skip it.
On severity: both buffers are live and the same size, so on glibc the
outer program simply continued against the wrong memory and returned a
wrong answer. ASan and UBSan are clean on the repro here -- neither can
see a rebind between two valid allocations. It took a platform whose
allocator left the old region unmapped to turn this into the SIGSEGV that
got it reported, which means Linux has been silently affected, and any
rvbench measurement mixing an inlined program with a fun_map/fun_filter
ECALL program has been suspect.
tests/growth/driver.py: restore the filter(#1/...) jit case held out
pending this fix. It is the regression test -- the harness runs every
case in one muxscript, so a recurrence reports UNMEASURED for everything
after it, which is how #2106 was found.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This reverts 43a4eb094 and 6880a8e42, restoring the guest memory layout to
its state at d16d20761 -- the last commit on which this box is green.
Master has been red since 6880a8e42 merged, in two distinct ways:
d16d20761 ALL 1634 smoke PASSED, test-lua-ecall PASS
5845a81f1 SIGSEGV in nested Lua-in-softcode JIT (#2107)
1c050049a no crash, but 8 Lua seam failures and test-lua-ecall
divergence ~50x wider than before the "fix"
4dc170ccc identical (pristine, with #2108)
43a4eb094 removed the crash without fixing the fault, trading a loud
failure for a quiet one: crashes went to 0 while agree_wrong went 1 -> 46
and exec_wrong 1 -> 57. Silent Lua state corruption is the worse of the
two outcomes, and it is caught only because the Lua oracle compares
against the interpreter.
Reverting rather than iterating forward, because:
- #2066 is a COVERAGE improvement. The JIT declining an expression
with >12KB of pooled constants is not a correctness bug -- it falls
back to the AST evaluator and gets the right answer. That is not
worth trading a working Lua path for, and certainly not worth
leaving master red while it is diagnosed.
- Two red-master events in one day from one file argues for
re-approaching the change behind a green baseline, where a new
failure is attributable to the new work. Both times, an agent
measuring something unrelated had to first prove master's breakage
was not theirs.
Every file this touches is byte-identical to d16d20761 afterwards, so
this restores a known-good layout rather than inventing a third one.
Also reverts, as collateral: the OUT_BASE constant deletion from
43a4eb094 (a genuine cleanup -- OUT_BASE has no live uses and its
comment wrongly claims otherwise), and the two testcases re-pinned above
the old string-pool cliff. Both should return with the re-landed #2066.
#2066 reopened. What it needs on re-approach is a regression test for
nested Lua-in-softcode re-entrancy: the static_asserts added in
#2066/#2113 pin region ABUTMENT, which is not what broke either time.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#2066 expanded STR by moving FARGS from 0x4000-0x8000 up to
0x9000-0xD000. That was enough to SIGSEGV nested Lua-in-softcode JIT
on at least one host while Linux/x86_64 stayed green — the expression
that crashed is tiny, so the enlarged capacity was not the trigger;
the layout move was.
Keep FARGS at its historical window and put the 32 KB (= LBUF) string
pool in the free gap before the blob:
CODE 0x0000-0x1000
(gap) 0x1000-0x4000 (former 12 KB STR; unused)
FARGS 0x4000-0x8000 (unmoved)
STR 0x8000-0x10000 (32 KB)
BLOB 0x10000+
Also fold the layout constants into s_blob_version so any future map
change invalidates the persisted code_cache even when __DATE__/__TIME__
does not move, and fix the legacy single-blob reconstruct high-water
(was FARGS_LIMIT; STR is now above it). Drop the dead OUT_BASE constant.
When a lowering is a single host ECALL with nothing computed around it, the
compiled program makes exactly the host call the AST evaluator would have made
and adds guest entry, marshalling and exit on top. There is no arrangement of
those costs that comes out ahead.
Measured for get(perfobj/perf0) over 10000 evaluations: 108 bytes of RV64, one
ECALL per evaluation, zero tier2_calls, zero folds, zero native_ops -- and not
bailing (eval_handled 213001 vs eval_bailout 24). It compiles successfully and
running it is slower than interpreting it:
no-JIT 0.352us JIT 0.475us penalty 1.35x
The shape covers get/v/u and most attribute and database accessors, which per
#2064 is what live games spend time on rather than arithmetic. The same
regression measures 1.36x on Windows/MSVC/x64 (#2083's attrread phase).
Declines programs with zero native_ops, zero folds, zero tier2_calls and at
most one ecall, after the constant-folded early return so folded programs --
which never execute -- are untouched. Correctness is unaffected either way:
both routes call the same host function.
after: 0.423us, penalty 1.20x -- ~42% of the overhead recovered
Deliberately conservative. Two ECALLs already saved one AST dispatch, and any
native_ops/folds/tier2_calls is work the interpreter would otherwise repeat.
Widening the predicate would decline programs the JIT wins on, and that failure
looks like a speedup on every benchmark not measuring the thing it broke.
Verified with controls that assert eval_handled MOVED for add(1,2),
add(mul(3,4),sub(10,2)), iter(lnum(20),1), strcat(...) and
strlen(get(...)) -- the last being the discriminating case: same ECALL, but
with work around it, so it must keep its compiled path.
Adds bail_noop to jitstats() so a program refused this way is distinguishable
from one never compiled.
jitstats_fn TC007 needed two changes, with different justifications:
* the probe is rewrapped as add(u(...),0). A bare [u(...)] no longer reaches
depth 2 at all, since u() is itself a no-work wrapper and now declines;
giving the outer expression real work restores the depth-2 entry the case
exists to cover, confirmed by its tier 2 delta returning from 1 to 2.
* the ecalls threshold drops >2 to >1. That half is a recalibration: the
chain genuinely contains one fewer counted ECALL, because one of them was
the wrapper this commit stops running. >1 still requires both the u()
ECALL and the depth-2 inner ECALL; >0 would pass on the outer call alone
and be vacuous for a case about nested evals.
Declining the wrapper is also faster, measured on one binary via a temporary
A/B gate: u(me/pr) 0.637 -> 0.607us, u(me/pr2) 0.662 -> 0.619us, values
identical. The inner expression ends up on a depth-1 full JIT path via
fun_u/mux_exec rather than the depth-2 shared heap.
make test: 36 targets, 36 passed, 0 skipped, 0 failed
(jit=yes stubslave=yes nls=yes realitylvls=yes wodrealms=yes).
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* test(map): TC012 pins the >255-byte element boundary (#2080)
The compiled CARGS slots are 256 bytes and _WRITE_CARG REJECTS oversized
values rather than truncate, so a MAP lowering must route long elements
through a fallback arm. This case is what catches a stale or truncated
%0: a 300-char element built at runtime, asserted on both routes against
the interpreter's answer (300x 1x -- the trailing x is body literal,
which the first expectation got wrong; captured, not assumed).
Local groundwork for the lowering; sits atop the u()-inline arm fix
(PR #2092), which the MAP gate pattern depends on.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* perf(jit): MAP lowering — the ITER loop composed with the u()-inline (#2080)
map(#dbref/attr, list, ...) inside a compiled program was an ECALL back
into fun_map with the body interpreted per element -- measured at
~1.1us/element, the worst of the three routes. This composes the two
proven pieces: the #2072 loop shape (SPLIT_TOKEN cursor, pinned-buffer
APPEND accumulator, int q-registers) with the Tier 3 u()-inline
machinery (compile-time attr resolution, CARGS helpers, mod_count
staleness deps, _CHECK_U_PERM).
Measured, map(#1/FN,lnum(1000)) with FN=[add(%0,1)], Darwin/arm64:
interp 717.8us (~718ns/element)
jit route before ~1100us (ECALL fun_map, body interpreted)
jit route now 393.9us (~394ns/element)
1.8x over the interpreter, 2.8x over the previous compiled route, with
the WRITE_CARG ECALL per element now the dominant term (a blob-side
CARGS write is the obvious next constant to attack).
The domain is stated honestly: the inline fires for a LITERAL
#dbref/attr whose runtime executor IS that dbref. fun_map evaluates the
body with THING as executor; an inlined body runs as the program's
executor, and a cached program can be re-run by anyone -- so the gate
compares %! against "#thing" by native STRCMP and everything else takes
the ECALL fallback, exactly as before. (The u()-inline lives with this
executor hole; MAP does not add a second copy of it. Executor-switch
helpers to widen the domain are the documented follow-up.)
Runtime gate, one diamond around the loop: _CHECK_U_PERM == 0, executor
match, and every extra argument fits its 256-byte CARGS slot -- because
_WRITE_CARG rejects oversized values and leaves the slot stale. Per
element, a second diamond for the same limit: elements that fit take the
inlined body; oversized ones ECALL u(#thing/attr, elem), semantically
identical since the gate pinned executor == thing. map_fn TC012 (the
300-byte element) is the case that catches a stale or truncated %0.
New blob primitive rv64_bytelen (bytes, not co_strlen's graphemes),
registered like SPLIT_TOKEN/APPEND -- internal-only.
Growth: two new guards. map(me/...) stays on the fallback (name forms
never inline) and reads b=1.00 both routes; map(#1/...) exercises the
inline and reads b=0.68..1.17 across runs -- SUB-linear, a large fixed
cost amortizing. That exposed a harness assumption: expected classes
were exact matches, so growing SLOWER than expected failed the run.
Guards now treat the expected class as a CEILING -- regressions only
point up, and wrong output at a too-cheap cost is the correctness
batteries' job (README updated).
Validation: 12-case sandbox battery on the inline path (delimiters,
osep, extras, empty list, empty elements, 300-byte element, nesting in
iter, arithmetic wrap, NOEVAL parity -- fun_map ignores object-NOEVAL
and asteval agrees, so the stricter gate is output-identical);
map_fn.mux 12/12 on BOTH smoke routes; full suite 36 targets, 35
passed, 1 skipped (stubslave=no), 0 failed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(growth): drop the ceiling; size the inline case so fixed cost amortizes
Review on #2094 refuted the within_ceiling() contract with the right
counterexample: a guard that accepts anything at-or-below its class cannot
tell a fixed cost amortizing (benign, what the MAP inline case hit) from
the case silently ceasing to do the work (erroring early, constant-folded,
cached -- flat cost, one of the two failures the harness exists to catch).
b = 0 or even negative passed a linear guard. Exact-match semantics are
restored; XPASS behavior was never affected.
The reviewer's fix, adjusted for one local fact: N=8000 is not reachable
-- lnum(8000) is ~39KB against a 32768-byte LBUF, which is why the shape
probes exist -- so the case runs 500..4000 instead. Fixed share at the
top size falls from ~13% to ~7% and the fitted exponent lands at b~1.06,
comfortably inside the linear band, with no contract change at all.
README rewritten to record the refutation and the rule it produced: size
cases until fixed cost amortizes; if a case cannot amortize inside one
LBUF, fit t = a + b*N and classify the slope -- never loosen the bands.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The SQLite code_cache is keyed on blob_hash (s_blob_version), whose tier 1 leg
was a single __DATE__/__TIME__ in jit_compiler.cpp. Its comment stated the
assumption:
"Because the JIT requires a clean rebuild to take effect, this file is
recompiled every such build, so its __DATE__/__TIME__ stamp changes and
folds into the hash, invalidating every previously persisted entry
automatically."
Under incremental make that is false, and false in exactly the case the stamp
was added to defend against. Its own comment names "a codegen change in
another TU (e.g. hir_codegen.cpp) with no version bump" -- and a change in
another TU is precisely when jit_compiler.cpp is NOT recompiled.
Measured before the fix: touching hir_lower.cpp rebuilt hir_lower.eo and left
jit_compiler.eo untouched, so blob_hash stayed byte-identical at ad598f6b...
and every previously persisted entry still matched. The cache then served the
previous build's compiled output -- which contaminated a #2052 retest, where
TINYMUX_DUMP_HIR counted zero compilations while the old behaviour still ran.
On a live game it means attributes keep running the old compiler's output after
an upgrade until something evicts them.
Each tier 1 unit now carries its own stamp and all of them fold into the hash,
so the key moves when any of them is recompiled. Folded into BOTH
s_blob_version computations; the no-blob fallback had the identical hole and is
the leg a blob-less build runs on.
The set is deliberately explicit rather than a glob, because the invalidation
model is three-legged and only two legs belong:
softlib.rv64 MUST invalidate -- cached RV64 calls into it via a PC-relative
JAL to a resolved blob address (hir_codegen.cpp:2262), so
moving a blob function makes cached code jump into the middle
of something else. Already covered.
tier 1 JIT MUST invalidate -- it emits different RV64 for the same
softcode. This commit.
the DBT MUST NOT invalidate -- it only executes the stored RV64 and is
not baked into it (the cache holds guest RV64, never host
code). Invalidating for a dbt_*.cpp change would discard the
cache for no gain.
Verified by rebuilding after touching each unit in turn and reading blob_hash
from a fresh database. All six tier 1 units move the key; dbt.cpp and
dbt_interp.cpp leave it unchanged -- the negative control matters, since a fix
that simply invalidated on everything would pass the positive half and be
wrong.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the second and third quadratic terms in the compiled iter() loop.
With #2076's cursor extraction this completes #2052: the growth harness
reads b=1.12 (linear) on the JIT route, and on the real mux_exec path the
compiled loop now BEATS the no-JIT build through mid-size lists and holds
parity at N=2000.
iter(lnum(N),1), us/call, min-of-3, Darwin/arm64, benchmark():
N JIT before all of this no JIT (#2068 bar) JIT now
50 15.20 8.14 7.06
200 210.45 30.55 26.93
500 1274.47 80.22 62.51
1000 5102.50 153.72 140.10
2000 20226.75 308.65 301.95
The three terms, for the record:
* EXTRACT per element (fixed in #2076): O(list) per element.
* acc = STRCAT(acc, osep, body) rebuilt the accumulator every iteration,
and the string PHI carrying it copied it AGAIN per iteration.
* rv64_split_token strlen'd the WHOLE list on every call -- measured as
the entire residual superlinearity (b=1.40) after the strcat fix.
What replaces the accumulator: the text lives in ONE pinned guest buffer
(rc.alloc_output(), which the liveness allocator never recycles), its
LENGTH rides in QREG_ITER_ACC as an int, and rv64_append writes at
buf+len in place -- the one deliberately mutating callee in the blob,
with the reasons and the checked compile-time hazards documented at its
definition. References to the buffer are emit_sref, the #1309 machinery
that keeps a runtime buffer out of ATOI constant folding --
add(iter(5,##),1) = 6 is the case that rules. The is_first test moved
into the callee, deleting the first/cat block diamond; a reset append
before the loop covers the zero-iteration read. The old STRCAT shape is
kept as the fallback for a blob without APPEND.
rv64_split_token also gains the trim the interpreter applies to the
EVALUATED list: leading spaces produced at runtime (%b, nested eval)
survive hir_lower_trimmed's AST-level trim, and co_extract used to absorb
them by compressing leading delimiters. A cursor walk must skip them
explicitly at offset 0 for the space delimiter or the first element is
"" -- iter(%b%ba b,##) returned " a" instead of "a b".
Correctness: 16-case battery including ##/#@/%i0, custom delimiters,
empty osep, empty list, leading/trailing/run spaces, arithmetic-wrapped
iter, two sequential loops, nested loops (byte-identical to the
interpreter), and parser_fn TC020 on the trigger path. Full suite: 36
targets, 35 passed, 1 skipped (stubslave=no), 0 failed.
The growth harness's iter-jit case flips from xfail(#2052) to a plain
linear expectation -- it XPASS-failed the run the moment the fix worked,
exactly as designed, and is now a permanent regression guard on both
routes of the same expression.
Scalar expressions are untouched (parity with before; the 1.3-1.6x
mux_exec dispatch overhead vs no-JIT remains #2068's open question).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jitstats() reported four distinct guest-memory budgets through two counters,
so production data could not say which one a decline came from -- which is
exactly the question #2070 and #2074 ask.
bail_slots was imprecise in two directions:
* it is incremented for a CODE region overflow (jit_compiler.cpp:1140) as
well as for output slot exhaustion, despite being documented as the
latter;
* the flag it keys off, pool_exhausted, is set by BOTH pool_str and
alloc_fargs, so the string pool and the fargs pool were indistinguishable.
Adds bail_code / bail_strpool / bail_fargs / bail_outslots, counted
independently rather than as a partition -- one compile can exhaust more than
one budget. bail_slots keeps its existing behaviour for dashboard
compatibility and for jitstats_fn.mux TC002.
Also adds want_code_max / want_strpool_max / want_fargs_max /
want_outslots_max, recorded BEFORE the ceiling checks. The existing
code_bytes_max is taken after them and is therefore censored at CODE_LIMIT by
construction: it cannot report a program that did not fit, which is precisely
the program a resize decision needs to see. A server whose attributes need
twice the region reports a comfortable maximum just under it. pool_str,
alloc_fargs and alloc_output now accumulate what was REQUESTED, including
requests that did not fit, so the amount of the overrun survives.
Validated by forcing each ceiling and asserting the specific counter moves and
that the others do not:
strcat(v(A) x64) -> bail_code +1, others 0
repeat(a%b,6128) -> bail_strpool +1, others 0
add(1,2) -> nothing moves
The uncensoring is visible directly: want_code_max reads 4148 against a 4096
ceiling and want_strpool_max 12296 against 12288, while the old code_max reads
108 for the same run.
make test: 36 targets, 36 passed, 0 skipped, 0 failed
(jit=yes stubslave=yes nls=yes realitylvls=yes wodrealms=yes).
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
NOT LANDABLE. parser_fn TC020 fails: two iter() loops in one compiled
program drop all but the last element of the second loop.
expected X Y Z11 21 31
got X Y Z31
Isolated by reverting hir_lower.cpp alone and keeping the blob: TC020 goes
green, so the lowering is the cause, not rv64_split_token.
Every isolated case is correct -- iter(lnum(5),##), #@ indices, custom input
and output delimiters, two iter()s in one `think`, the setr()-wrapped form.
It reproduces only through the smoke harness, i.e. as an attribute triggered
on an object rather than typed. That difference has not been explained and is
the next thing to chase.
Attempted fix that did NOT work: moving the cursor's LOAD_Q from the body
block to the header, alongside inum and acc, on the theory that a
loop-carried q-register must be read where the loop's PHI lives. It still
fails, so either that is not the mechanism or it is not the only one.
What the work does contain, and what it measured:
* rv64_split_token in the blob -- a real cursor walk mirroring split_token's
single-byte-delimiter semantics, including the space-run collapse. Two
modes (element / next offset) rather than an out-parameter, so it stays a
pure function like every other wrapper. Verified present at 0x110b4.
* SPLIT_TOKEN registered in s_tier2_map and the tier2 allowlist. This is what
makes the previously-dead branch reachable at all -- tier2_lookup had
always returned 0.
* The ITER lowering rewritten to carry a byte offset in QREG_ITER_CURSOR
(declared since the beginning, never used).
Measured before the correctness failure was found -- a clean 2x at every
size, with the exponent UNCHANGED:
N before after no JIT
200 210.45us 111.20us 30.55us
500 1274.47us 640.33us 80.22us
1000 5102.50us 2531.14us 153.72us
2000 20226.75us 9939.45us 308.65us
Growth harness b=1.99 -> 1.95. Still quadratic, because the accumulator
`acc = strcat(acc, osep, body)` is a SECOND O(n^2) term of the same size --
now filed as #2072. So even correct, this half alone does not reach the bar
(#2068): the target is the no-JIT column.
NOTE FOR ANYONE RESUMING: do not land the s_tier2_map registration without a
working lowering. Registering SPLIT_TOKEN while hir_lower still holds the
ORIGINAL branch would make that branch live for the first time -- and it
calls emit_call with a hardcoded func_idx of 0 and passes the cursor as an
emit_sconst whose storage the callee is expected to mutate. It has never
executed. The registration and the lowering have to move together.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The cached (JIT) path runs at 13-47 ns/call. Reported as %.2fus that is
"0.01" or "0.02" -- one or two significant digits -- so consecutive
identical runs looked like 100% swings and any regression under about 2x
was invisible. A number that cannot be resolved cannot be gated on, and the
performance arc needs to gate on this one.
Measured across three smoke runs, before and after:
report native spread cached spread
%.2fus 2-10% 0-100% (quantisation, not noise)
%.1fns 2- 9% 3- 13%
So the JIT path was never the unstable one; it was unmeasurable. Both paths
sit at roughly a 10% noise floor on this box, which is the number any future
threshold has to respect.
Nothing parses the BENCH lines yet -- the only reference is a comment in
generate_smoke_suite.py about excluding rvbench_fn from the sanitiser run --
so changing the unit costs no downstream churn. Doing it before a baseline
exists rather than after.
The format string stays one unbroken literal and the explanation sits above
the call: check_formats.py reads the source and requires mux_snprintf()'s
format to be constant, so a comment interleaved with the concatenation trips
the guard. It caught exactly that on the first attempt.
make test 35 passed / 1 skipped / 0 failed
(jit=yes stubslave=no nls=yes realitylvls=yes wodrealms=yes).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Terminating a runaway self-recursive ufun on the compiled route cost time
exponential in function_recursion_limit. Measured here, before:
function_recursion_limit eval_attempts seconds
8 97 0.02
12 1,537 0.01
16 24,577 0.14
20 393,217 0.57
24 6,291,457 7.76
eval_attempts is exactly 3*2^(d-3)+1 at every depth -- a full binary tree
with 2^(d-3) leaves and 2^(d-3)-1 internal nodes. The AST route is 0.00s
at every depth, so the blowup is specific to the compiled route.
Root cause. shared_heap_t holds one dbt_state_t: one guest register
context, one stack pointer, one heap arena. jit_eval's s_jit_depth > 1
arm runs through it, and that program ECALLs into u(), whose body
evaluates another bracket, which re-enters shared_heap_t::eval. The
nested run sets ctx.x[2] = STACK_TOP, resets s_heap_next and calls
tier2_reset_writable -- all underneath the suspended outer run. The outer
run resumes at a program counter that is not its own, the backend refuses
to translate there (XLATE_REFUSE), dbt_resume returns -1, and
shared_heap_t::eval returns false with host ECALLs already run. jit_eval
propagates that, and mux_exec redoes the whole bracket through the AST.
That bracket contains the next recursion level, so each level is evaluated
twice.
The type declares the hazard away in a comment -- "independent of the
outer expression's DBT, so this is safe to call from within an ECALL
handler" -- which is true of the outer expression's DBT and false of its
own. #1309 identified the same hazard for the Lua path and guarded it
with s_run_cached_depth; the softcode path re-enters unguarded.
Instrumented over the runaway, the correlation is total: of 511 runs that
had a nested run inside them, 511 failed and 0 succeeded; of 512 runs with
no nested run, 512 succeeded and 0 failed. Every re-entrant outer run
fails, so the nested run's result is always discarded by the parent's AST
re-run -- declining it up front forfeits no retained work.
After: eval_attempts is exactly limit+1 at every depth, and depth 100 --
which the issue reports as not finishing in any practical time -- answers
instantly.
function_recursion_limit eval_attempts seconds
8 9 0.00
24 25 0.00
100 101 0.00
Not a behaviour change for recursion that should complete: sum(1..n) via a
self-recursive ufun returns n(n+1)/2 on both routes at n = 3..50, checked
against the arithmetic rather than against the other route.
One difference is visible at the boundary. When the limit is low enough
that the recursion cannot complete, the two routes reach it at different
points inside the expression, because the JIT flattens nesting and this
changes how many func_nest_lev levels one softcode level costs. For
[switch(gt(%0,0),1,[u(me/CD,sub(%0,1))],done)] the limit can land in the
condition instead of the arm, and switch() then returns its default --
so the compiled route may answer "done" where the AST answers the limit
error. Enforcement itself is unaffected: func_nest_lev peaked at exactly
limit-2 in every configuration measured, never above. This is the same
class as the INVOCATION-vs-RECURSION difference already noted on #1994.
The counter is exposed as jitstats() bail_shared_busy so the decline is
visible rather than silent.
Adds testcases/tools/jit_recursion/oracle.sh (make test-jit-recursion),
which asserts eval_attempts stays linear. It has to assert a counter
rather than a result: the answer is correct either way -- the AST re-run
recomputes the same string -- which is exactly why this went unnoticed,
and a result-equality check cannot see it. Confirmed by breaking the
guard and re-running: the cost assertion fails at 393,217 while every
result assertion still passes.
A census over the full smoke suite recorded zero re-entrant shared-heap
runs (outer_ok=0 outer_fail=0, flat_ok=113), so smoke never exercised this
path at all -- which is why the regression test is a new oracle rather
than a smoke case.
make test 34/34 with jit=yes stubslave=yes nls=yes realitylvls=yes
wodrealms=yes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A self-recursive user function crashed netmux with a stack overflow at
any function_recursion_limit, including 2:
&RECUR me=[u(me/RECUR)]
think [u(me/RECUR)]
ast.cpp bumps func_nest_lev / func_invk_ctr and tests both limits
immediately before its own fp->fun dispatch. The compiled route reaches
the same builtin through ecall_invoke_fun (and a global through
ecall_invoke_ufun) without passing through that branch, so nothing
counted the call. The recursion cycles jit_eval -> ECALL -> fun_u ->
mux_exec -> '[' -> jit_eval and never touches a counter; it runs until
the stack is gone.
The #1002 static watermark cannot catch this. max_func_depth is a
compile-time property of the program -- for [u(me/RECUR)] it is 1 no
matter how deep the recursion actually goes -- so the watermark compares
against a number that never grows and never declines to the AST. This
is why a bigger stack does not help: the guard is on the route not taken.
Maintain the counters in both ECALL invocation paths, mirroring ast.cpp,
the same way #1124 mirrored its check_access. Underscore-prefixed names
are JIT internal helpers (_SAVE_QREGS, _WRITE_CARG, ...) rather than
softcode calls and are exempt from counting, matching the exemption the
perms gate already uses. RAII returns the nesting level on every exit
path; func_invk_ctr is monotonic, as in ast.cpp.
Measured live on Linux/gcc (the issue reported Windows; it is not
platform-specific -- the JIT route is the default on both):
before jit=1 lim=2/50/500 SIGSEGV, guard never fires
jit=1 nest=500 invk=10 SIGSEGV (invocation limit also unenforced)
jit=0 (control) guard fires, server healthy
after jit=1 lim=2 #-1 FUNCTION RECURSION LIMIT EXCEEDED
jit=1 stock defaults limit error in 0.46s, server responsive
jit=0 (control) unchanged
Sequential (non-recursive) calls were already limited correctly by the
static n_func_calls watermark and are unaffected.
test-smoke 1605/1605 and test-smoke-ast 1605/1605 + 1604/1604, no
crashes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
shared_heap_t::eval() kept a pointer into the cache entry across
dbt_run/dbt_resume. Nested re-entry can lookup() the same key, see a
stale attr_mod_count, and erase that node — so the post-run e->out_addr
read is a use-after-free under recursive self-modifying softcode.
#1938 already snapshotted ecalls/tier2/needs_jit for the same reason.
Copy out_addr with them and use the local for both the folded path and
the post-run result extract.
Phase 4 over-committed pre-entry setup failures (carg/depth/get_dbt) as
#-1 LUA JIT RUN FAIL; fall back to the interpreter when nothing ran.
Protect lua_getglobal and lua_compare via pcall so raising metamethods
cannot abort the server. Apply EQK-class type gates and NIL/BOOL kinds
to register-register CMP_RR equality, and decline mixed-type order.
Value-producing `return x == "0"` is EQK+JMP+LFALSESKIP+LOADTRUE, but
EQK was never in the bool fuse and always declined on TY_LUA_HANDLE.
Add ECALL_LUA_EQ / HIR_LUA_EQ, fuse EQK like EQ/EQI, and route handle
and non-handle EQK through emit_cmp_branch. tostring(0)=="0" and
return nil=="" now execute; AGREE_DECLINE_BUDGET 13→11 on current master.
Softcode CALL_FUNC is already total (guest error strings, never
ECALL_DECLINE). The residual poison was mid-run DBT failure (code buffer
full, harvest miss) returning false → full AST re-run after effects.
Track eval_ctx::host_ecalls on softcode host ECALLs; after entry, commit
#-1 JIT POST-ENTRY FAIL instead of re-running. Document the audit in
plan-softcode-post-entry-contract.md and cross-link the Lua plan.
Grow the seam corpus (TC066–070): softcode t() on lua "0"/hello (boundary
truth), add(lua number), mux.owner across the seam, ifelse(gt(lua)), and
error text under softcode.
fix CALL_VOID: after the callee may have run, commit raised errors like
CALL_STR instead of residual decline — bare error("…") was a loud
POST-ENTRY DECLINE vs the interpreter's LUA ERROR.
luajit: EXEC pins for mux.owner/pennies/location/isplayer returns; AGREE
pins for in-Lua use of type-erased mux.* results and error() shapes.
AGREE_DECLINE 16→18.
#1764 shape 2: default STRING-claim calls no longer marshal to TY_STRING
text immediately. ECALL_LUA_CALL_VAL leaves the first pcall result on the
Lua stack and returns a TY_LUA_HANDLE. Softcode boundary marshals via
ECALL_LUA_MARSHAL (fun_lua rules). TEST/NOT use ECALL_LUA_TOBOOL so
tostring(0)/"" / string.find stay truthy under Lua semantics.
CALL_INT / CALL_VOID unchanged. Equality of a value handle to a string
constant still declines (needs VM compare).
AGREE_DECLINE 16→12: the four #1764 if/not pins now execute.
CALL_STR no longer soft-declines after the callee has run. Marshal the
first result like fun_lua (nil/bool/number/string), so string.find's
integer first value becomes "2" on both routes and executes compiled.
Ratchet POST_ENTRY_LOUD 4→2 (absent-key pin + STATE e2 EFFECT_REFUSED).
Smoke TC061–TC065 grow the softcode↔Lua seam corpus: mux.eval round-trip,
attr get/set across the boundary, nested [lua()] in softcode, and
string.find through the new marshal path.
After RunCompiled, TryJIT always commits the result (success, LUA ERROR,
POST-ENTRY residual, RUN FAIL, or CPU LIMITED) and never returns false to
re-run the chunk in the Lua VM. Pre-entry fallback remains: compile refuse,
empty dump, prior ineligible, cache miss/recompile.
run_cached_program with a Lua state never returns false after dbt_run for
unhandled status — commits #-1 LUA JIT RUN FAIL instead.
UNHANDLED_LUA_BRIDGE becomes a committed LUA ERROR. lua_ecall_decline
rewrites any residual soft decline into ECALL_LUA_ERROR so the poison
retry path cannot return by accident.
Harness still green (post_entry_loud=1: EFFECT_REFUSED only).
ECALL_LUA_LIMITED no longer declines or re-runs. Commit the same softcode
framing as InsnCountHook: #-1 LUA ERROR: instruction limit exceeded.
Budget for-loop and while-true now match on both routes; POST_ENTRY_LOUD
ratchets 3→1 (STATE e2 EFFECT_REFUSED only).
Review of the Phase 1 totalization found a silent wrong answer it
introduced, and the fix exposed a second one that predates it.
INTRODUCED: totalized lua_gettable runs metamethods, but the typed
claim-miss path continued with 0. Measured: an interpreter-installed
__index on a global made compiled `return T[1]` answer 0 where the
interpreter answered "s" -- run_ok=1, decline counter 0, nothing loud.
The campaign's own worst category, from its own Phase 1.
PRE-EXISTING: the same continue-with-0 shape made an ABSENT key read
silently wrong since the typed GETI first landed: `local t={} t[1]=5
return t[2]` answered "0" compiled where the interpreter answers "".
GETFIELD_INT's twin declined; GETI's continued. Nothing pinned it.
The revision makes the typed claims PROVABLE or ineligible:
* lua_referent gains plain_proven -- set only by this chunk's NEWTABLE,
CLEARED when the handle escapes as a call argument (the callee can
setmetatable it). GETI / GETFIELD_INT / GETFIELD_FLT emissions
require the proof; anything else is ineligible at lowering and the
interpreter answers. Exhibit: the metatabled-global read now agrees
("s" on both legs), pinned as STATE case e3.
* Under the proof no metamethod can fire and every compiled store is an
integer, so the one honest miss left is an absent key -- Lua nil,
which a typed integer slot cannot carry. That is now a loud
post-entry fail (named bins GETI_INT_NONINT / _BADIDX) instead of a
silent 0, pinned as an AGREE loud case.
* Total LEN / SET / GETGLOBAL from the original Phase 1 stand: LEN and
SET delegate to the real VM ops and raise what the interpreter
raises; a nil global stays on the stack for its consumer to raise on.
Harness re-baselines: AGREE_DECLINE 2->3 (escaped-read pin, silent by
design), POST_ENTRY_LOUD 6->7 (absent-key pin, loud by design); the
escaped-read EXEC case moves to AGREE with a pre-escape EXEC
replacement, so CALL_VOID+proven-read coverage survives.
make test exit 0; luajit 146 chunks 0 wrong; STATE oracle green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace plain-table / type / missing-global declines with real VM ops under
pcall (lua_gettable, lua_settable, luaL_len). On raise, commit an
interpreter-class #-1 LUA ERROR (ECALL_LUA_ERROR) without re-run. Typed
result claims that miss after a successful get return ok=0 and continue
(same as GETI already did). Nil globals and nil fields stay as handles.
GET/SET/LEN no longer return ECALL_DECLINE. Remaining declines are CALL
and LIMITED (Phases 2–3).
Harness: normalize LUA ERROR agreement (location / parenthetical detail);
ratchet POST_ENTRY_LOUD_BUDGET 7→6 (os.time now raises on both routes).
A discarded compiled Lua run is not free once guest code may have run.
Wire every ECALL_DECLINE after entry to a committed error
(#-1 LUA JIT POST-ENTRY DECLINE (site)), a JIT/RETRY log line, and
jitstats lua_post_entry_decline. effect_refused follows the same rule.
Pre-entry fallback (compile refuse / cache miss) is unchanged.
Also fix lua_ecall_decline so it returns ECALL_DECLINE instead of
recursing (that hang turned budget/LIMITED cases into harness timeouts).
Harness: AGREE/NESTED_AGREE accept loud fails; ratchets POST_ENTRY_LOUD=5
and AGREE_DECLINE=2 from measured counts.
Default-on (#1745) exposed that the compiled path baked the back-edge
budget into each program as a constant:
int budget_init = h.emit(HIR_ICONST, TY_INT, -1, -1,
static_cast<int64_t>(mudconf.lua_instruction_limit));
A compiled program is cached in memory and persisted in code_cache, so the
limit in force was whatever was configured when the chunk happened to
compile -- @admin lua_instruction_limit reported Set. and changed nothing,
which is #1613's bug arriving on the compiled path. test-config's
runtime-bounds case caught it the moment the flip put the compiled path in
its way; on default-configure trees --enable-jit is off and everything
stayed green, which is why the flip validated cleanly elsewhere.
## Fix
A dedicated no-arg ECALL, following the LUA_LEN pattern end to end:
HIR_LUA_INSN_BUDGET -> ECALL_LUA_INSN_BUDGET (0x314)
-> a0 = mudconf.lua_instruction_limit, read per run
The entry seed becomes ECALL + STORE_Q, so the program carries no config
value at all. That fixes the in-memory cache and makes the persisted
code_cache safe by construction rather than by flush discipline; blobs from
before this change carry the old entry-store, and JIT_BUILD_STAMP
(__DATE__ __TIME__) already invalidates them on rebuild.
The handler clamps the limit to >= 1: a zero or negative limit must abort
loops, not arm an effectively unbounded unsigned countdown.
The op is listed in needs_int_reg() -- whose own comment documents that
omitting an int-producing ECALL fails silently (codegen emits nothing and
the consumer reads garbage), which is the trap this listing avoids.
## Verified
FAIL: lowering lua_instruction_limit at runtime had no effect before
ok: lua limits apply at runtime, both directions, and read back after
## Second layer found under this one: #1748
Full smoke under default-on now completes and fails exactly two cases
(TC013 mux.name, TC014 mux.eval -- compiled path answers wrongly instead of
declining). On the UNFIXED merge commit those cases are unreachable: the
smoke chain stalls at 164/320 dispatched with 800+ lost verdicts. So this
fix converts a catastrophic stall into two known failures, filed as #1748
with the pre/post evidence. make test on JIT trees stays red at those two
until #1748 resolves; lua_jit 0 remains 1561/1561.
Also amends plan-lua-jit-product.md, replacing "residual optional polish
only" with the post-flip regression record -- as promised in the #1747
review.
Refs #1325, #1613, #1732, #1747, #1748.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Convert remaining engine `int x = mux_atoi64(...)` (and explicit
static_cast<int> forms) to int64_t so parse width is not discarded on
any platform. Includes functions/funceval/funmath, mail folder numbers,
comsys charge parse, JIT register indices, and HIR lower mid/left/right.
dbref parse keeps full-width parse then rejects values outside int
range. Channel charge still stores int after an in-range check.
Numeric `for` compiles and runs; `while`/`repeat` (backward JMP) and
generic for (TFOR) still decline. Three things had to be true at once:
RIGHT SEMANTICS. The FORPREP/FORLOOP lowering behind #1326's reject
implemented Lua 5.3 -- signed sBx offsets, init-step pre-subtraction --
against a 5.4 VM, and had never executed. 5.4's FORPREP falls INTO the
body (jumping forward past FORLOOP only on a zero trip count) and
FORLOOP jumps BACK by an unsigned Bx. First cut takes STATIC BOUNDS
only: init/limit/step must be integer constants, so trip direction,
zero-trip, and freedom from wraparound are compile-time facts -- 5.4's
counter model exists precisely because a naive idx<=limit test misses
at the integer edge, and declining the edge is cheaper than reproducing
the counter.
LOOP-CARRIED VALUES. A plain HIR value crosses blocks only under
dominance, and the #1422 transition drops the rest -- fatal for the
accumulator in `for i=1,4 do s=s+i end`. Loop protos now route Lua
registers through q-registers (reg r -> qreg r), the one traffic
hir_ssa_construct PHI-converts: store-at-write after every
non-terminator instruction, reload at every block entry. Backing is
claimed only where every path stores first -- the entry block, or
FORLOOP for its visible index, whose readers the latch dominates. The
first draft skipped reloads for registers still holding entry
CONSTANTS; the harness answered `return s` with the loop INDEX --
dominance is availability, not currency, and inside a loop the entry
value is one iteration stale. Reloads are unconditional now, and
FORLOOP reads its ICONST bounds from entry_final[], the register state
frozen at the entry block's exit.
EXHAUSTION THAT ABORTS. The old budget folded exhaustion into the
loop condition -- an early exit with a WRONG PARTIAL SUM, which is what
#1326 refused to ship. Each back edge now branches to a shared block
whose ECALL_LUA_LIMITED declines the entire run; the caller fails over
to the interpreter, which re-runs the chunk into its own hook and
raises "#-1 LUA ERROR: instruction limit exceeded" -- the player sees
the interpreter's error verbatim, from one budget. The re-run is why
loop protos must be RERUN-SAFE: eligibility rejects calls, SELF and
SETTABUP inside them, and the referent (#1725) declines stores into
global-shaped tables while chunk-local NEWTABLE stores stay compiled.
EXEC pins the accumulator, an order-sensitive a*10+i, and the
zero-iteration path (where a reload of a never-stored qreg would read
the surrounding command's %q). AGREE pins budget exhaustion against
the interpreter's error text. AGREE declines: 5 of 35, every one
deliberate.
luajit: 133 chunks, 0 wrong, 0 crashes; smoke 1561/0; make test clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The third argument is what finally graduates CALL_INT/CALL_STR off the
packed val[] encoding: fn stays src1, the arguments ride the carg[] list
exactly as HIR_CALL's do (emit_lua_call), and val[] keeps only the
two-bit argument kinds. hir_val_operand's CALL branch and
hir_operand_set's re-packing hack are DELETED rather than grown a third
shape -- every operand-walking pass now sees call arguments through the
ARG slots, closing the seam that made CALL_INT's second argument
invisible to liveness (20d39472f) and that the codegen comment had been
naming since the packing landed.
On that footing, three call-surface features:
* Three arguments (string.sub). The third rides x14, so CALL_STR's
out addr/size shift to x15/x16 -- an internal encoding, changed
everywhere in this commit.
* Handle arguments, kind 3: the register carries the stack index and
the handler does lua_pushvalue -- the one use of a handle that is
ABOUT the thing it points at (#1579), which table.concat({...},",")
needs. Codegen's register move for kind 0 was already exactly right.
* HIR_LUA_CALL_VOID for nresults == 0 (table.insert): pcall asked for
zero results, nothing to type-check. It exists only for its side
effect and produces no value, so nothing downstream can keep it
alive -- it joins has_side_effects(), without which DCE NOPs it
(#1145's SETI lesson).
EXEC pins each: three args, a handle arg with a string result, and the
void call read back through t[2] -- #t alone stays plausible when an
insert silently never ran, but the inserted element cannot. AGREE
declines fall 8 -> 5; the survivors are all deliberate (loops x2,
os.time, string.find's result-count pin, select's four arguments).
luajit: 129 chunks, 0 wrong, 0 crashes; smoke 1561/0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
math.pi, math.huge and math.maxinteger declined: a field read on a
library table takes a reference, and a handle to a number is something
nothing downstream can consume. The referent now carries a table of
known VALUE members, and the field read takes the value itself --
GETFIELD (INT) for maxinteger/mininteger, the new GETFIELD_FLT for
pi/huge, whose double rides the same raw-bits lane the call arguments
use and lands directly in its FP slot.
Function members deliberately do NOT join the table: their return
claims stay in lua_call_claim, so each fact lives exactly once. And as
with every referent claim, this is eligibility, not soundness -- a game
that rebinds math.pi to a string declines at the handler's type check
(a genuine float only; lua_isnumber alone would coerce "3.7" and
integers, which have their own routes).
HIR_LUA_GETFIELD_FLT is the first Lua opcode producing TY_FLOAT, which
found the FP twin of the needs_int_reg() silent-registration trap:
a float producer missing from needs_fp_reg() gets no slot, loc[].addr
stays 0, and the post-ECALL store writes guest address 0 (#1159's
shape). Noted in both places.
EXEC pins each value USED in arithmetic, not just returned -- an
unwritten slot cannot survive x*2 or x-1. AGREE declines fall 11 -> 8;
ratchet tightened as the harness requires.
luajit: 126 chunks, 0 wrong, 0 crashes; smoke 1561/0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
math.floor(3.7), math.type(3.0) and math.type(2^3) all declined because
the call argument walk accepted only integers and constant strings.
Floats could not be smuggled as rendered text -- coercion lies to a
type-sensitive callee: math.type("3.0") is nil, not "float" -- so they
travel honestly, as raw double bits through the integer argument
register, the same FMV.X.D lane ECALL_LUA_FTOA already proved on both
execution routes. Runtime floats work identically to constants: the
value FLDs from its FP slot at call time.
The argument encoding widens from one kind bit to two per argument
(0 integer, 1 string address, 2 double bits), and the widening forced
the factoring the duplication deserved anyway: one emitter in codegen
(emit_lua_call_args) and one decoder in the handler
(ecall_lua_push_call_args) replace two near-identical copies of each --
the same #1457 drift shape the lowering's twin call branches were merged
out of. The packed kind bits are the single source of truth end to end;
codegen no longer re-derives argument shapes from h.kind.
Four EXEC cases pin the lane: a float constant to each result variant,
two floor calls with different fractions (catches a reused argument
slot), and a RUNTIME float a constant-folding accident cannot fake.
AGREE declines fall 15 -> 11; ratchet tightened in this commit as the
harness requires.
luajit: 124 chunks, 0 wrong, 0 crashes; smoke 1561/0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tostring(42) -> 42 type(42) -> number tonumber("17") -> 17
Decline count 23 -> 15. Eight chunks, the largest step so far, and most of
it came from REMOVING things rather than adding.
Two changes.
A callable handle may now come from GETGLOBAL as well as GETFIELD_REF.
tostring is the global itself, a function rather than a library table. The
lowering does not try to know which globals are functions -- the ECALL
already checks lua_isfunction, so math called rather than indexed declines
there.
And CALL_INT and CALL_STR now share ONE argument encoding. CALL_INT took
integers only; CALL_STR took integers or constant strings. That difference
was never justified: the two differ in RESULT type, so they have no business
differing in how arguments arrive. tonumber("17") is the case that proves
it -- string argument, integer result, needs both halves. The
tonumber(mux.args[1]) family fell out for free and was not on the list.
WHAT THE LOWERING NOW GUESSES, AND WHY IT IS ONE QUESTION
math.max(3,9) and tostring(42) have identical argument shapes and opposite
result types. HIR result types are STATIC and Lua's are not, so nothing at
the call site distinguishes them and the callee's NAME is the only carrier.
lua_callee_returns_int is a short whitelist; a name not on it declines, and
declining is always correct.
That is the second place the lowering guesses what a handle points at. The
first is GETFIELD_REF vs GETFIELD_INT by provenance. Both are the same
question -- "handle to WHAT" -- and both want the same fix: a handle type
that carries its referent, including a function's return type. Two
independent pressure points now argue for it rather than one.
Tests pair tostring with tonumber deliberately: same argument shapes,
opposite result types, so an implementation inferring the result type from
the arguments cannot pass both. All three report lua_run_ok=0 on master.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
string.upper("ab") -> AB string.lower("AB") -> ab
string.rep("ab",2) -> abab
Decline count 26 -> 23. This answers the convention #1713 deliberately
deferred: where does a string result go, and who bounds it.
It goes in the output slot the register allocator already gives any
TY_STRING value, and the ECALL is told that slot's SIZE rather than assuming
it. ECALL_ORD is why: its bound read like one and was not -- 64 bytes of
headroom against a loop writing per codepoint -- and it wrote 15k (#1679).
Two choices made against the lazier option:
Decline on overflow, do not truncate. A silently shortened string is a
wrong answer, and the interpreter can produce the whole thing.
Accept LUA_TSTRING only, rather than lua_tolstring on anything. That
function coerces a number AND mutates the stack slot in place, which would
disturb a live handle and make number->string conversion the JIT's rules
instead of Lua's. ITOA/FTOA already carry the interpreter's rules.
Arguments may be integers or constant strings; a kind bit per argument tells
the handler which register holds which. A runtime string argument needs its
own guest buffer and waits for something that needs it.
#1715 EARNED ITSELF HERE. CALL_STR packs three fields into val[] --
nargs | kinds<<8 | arg1<<16 -- and teaching the operand accessor about it was
ONE edit. Before that refactor it was four separate walks, three of which
fail silently when missed, and all four would have needed it while the
string convention was also being designed.
A test that could not fail, caught in the minute it was written: I added
string.sub("hello",2,3) as an EXEC case with a comment claiming it covered
mixed argument kinds. It takes THREE arguments against a ceiling of two, so
it declines and can never execute -- the comment asserted coverage that did
not exist. string.rep("ab",2) is genuinely string-plus-integer and does
cover it. The EXEC contract caught this, because a declining chunk there is
a hard error rather than a pass.
All three cases report lua_run_ok=0 on master.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
math.max(3,9) -> 9 math.min(3,9) -> 3 math.abs(-7) -> 7
Decline count 29 -> 26, the largest single step so far, and the first that
crosses THREE ECALLs in one compiled run:
GETGLOBAL name -> stack index of the library table
GETFIELD_REF table + name -> stack index of the function
CALL_INT function + up to two integer args -> integer result
#1519 flagged stack discipline across ECALLs as needing proof rather than
assumption. It holds: a handle from one ECALL stays valid across the next
two inside a run, bounded by TryJIT's settop around the whole thing.
Narrow on purpose. Integer args, integer result, nothing marshalled --
math.floor(3.7) still declines because its argument is a float. That keeps
this increment about STRUCTURE and leaves the string-result convention to be
decided on its own, where a bad choice would be expensive (see ECALL_ORD's
unbounded write, #1679).
WHAT THIS BUILD PROVED THE IR CANNOT DO
Two seams, neither of which I would have written down from taste:
1. An instruction needs N OPERANDS. A call has three or four; the IR has
src1, src2 and val[]. So nargs is bit-packed with an instruction index
into val[]:
int64_t packed = nargs | (int64_t)(a1 + 1) << 8;
That is val[]'s FOURTH meaning after immediate, guest address, and
SETI's third operand -- and hir_val_operand() cannot see the arg1 index
at all. Nothing breaks today only because two-argument calls are simple
enough that liveness incidentally holds. That is luck, and it is the
same shape that produced #1711's wrong answer.
What the upper layer needs, plainly: an operand list every pass can walk
without knowing the opcode.
2. The type system knows "handle" but not "handle to WHAT". Choosing
GETFIELD_REF over GETFIELD_INT is a guess from provenance -- did this
handle come from GETGLOBAL or from NEWTABLE -- because a library member
is a function and a data-table member is a value, and TY_LUA_HANDLE
cannot tell them apart. Second place this pass has had to guess.
Both are recorded rather than worked around silently, because the refactor
they argue for should be driven by what the upper layer demonstrably needs.
Tests use asymmetric two-argument calls, max(3,9) and min(3,9), for the same
reason #1711's use two distinct keys: one argument cannot distinguish
correct passing from an argument being ignored or the pair being swapped.
All three report lua_run_ok=0 on master.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
local t={a=3,b=4} return t.a+t.b -> 7 run_ok, no fallback
local t={} t.x=5 t.y=6 return t.x*10+t.y -> 56
Decline count 30 -> 29.
The key travels as an ADDRESS into the program's own string pool, never as
marshalled text, so nothing downstream can mistake it for a value. Only the
integer result comes back, in a register; a non-integer field declines
inside the handler rather than guessing a marshalling.
Two more registration points, both silent if missed, and both found by
READING rather than by debugging -- hir.h's comment about val[] operands is
what prompted the check:
hir_val_operand() was gated strictly on HIR_LUA_SETI. SETFIELD parks
its value there too, so the liveness walker would not
have seen it and the register could be recycled before
the ECALL read it.
has_side_effects() SETFIELD is a store with no result. DCE deletes it.
That makes seven places a new opcode may need to appear, four of which fail
with no diagnostic: enum, lowering, codegen case, needs_int_reg,
hir_kind_name, hir_val_operand, has_side_effects.
Also the k flag again: OP_LUA_SETFIELD's C is a CONSTANT index when k is
set, exactly as OP_LUA_SETTABI's was, so `t.x=5` declined until it was
handled.
A WRONG ANSWER, caught before merge and worth recording
local t={a=3,b=4} return t.a+t.b answered 8, not 7
HIR_SCONST lives as loc[].addr with in_reg=false. Passing the key through
ra_get_reg returned a register that was never loaded, so a1 held the same
stale address on every call and every field read returned the LAST value
written -- 4+4.
The harness was fully green while that was true: agree_wrong 0, exec_wrong
0, and my own first EXEC case `local t={a=7} return t.a` PASSED, because
with one key "always return the last write" is indistinguishable from
correct. A probe with two distinct keys is what exposed it.
So the EXEC cases here read TWO DISTINCT KEYS deliberately. For any keyed
operation a single-key test proves almost nothing: stale key register, key
ignored, and all-keys-alias are each invisible unless two keys are read back
independently.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
