Compiled iter() levels are now published in a guest-side loop-context
table (rv_compiler::LOOPCTX_BASE): depth, plus each live level's element
buffer address and 1-based iteration number. At ECALL time an RAII
GuestLoopContext pushes those levels onto the interpreter's
itext[]/inum[]/in_loop stack around the callee, so anything that
evaluates softcode — fun_u's mux_exec, fun_itext, fun_ilev — sees the
composed stack instead of an empty one. %i0 inside u() called from a
compiled iter now answers, and nested programs compose correctly
because each nesting depth owns its own s_vm buffer while the caller's
RAII push stays active for the callee's whole run.
The marshal also retires #2170's remaining compile bails: ilev() and
dynamic-depth itext()/inum() inside compiled levels now lower to the
plain ECALL and are correct at runtime, and constant depths naming
enclosing interpreted iters no longer need the depth adjusted down by
the compiled levels. Constant depths naming levels in THIS program
still resolve at compile time. A nest deeper than the table
(LOOPCTX_MAX_LEVELS = 10) declines the compile rather than publish a
partial stack.
Two codegen subtleties found by live probes, not review:
- The element's table payload must go through rv_load_guest_addr: its
buffer can be an output-frame slot, and storing the tagged constant
raw handed the host an out-of-range guest address (the push then
aborted, and ilev() read an empty stack).
- HIR_LCTX_KEEP (emits no code) references the element at the END of
the level's body, extending its live interval so the slot allocator
cannot recycle the buffer for an inner loop's element or a body
temporary while a callee could still read it through the table.
run_cached_program zeroes the table's depth each run — the VM buffer is
shared across programs, so a loop-free program must publish no levels.
Stale-cache compatibility is not a concern: the code cache is flushed
after a rebuild because the program hashes regenerate.
Verified live on direct-#1 stdin (the JIT-gated context),
jit_handled=5/5: u() callees see %i0/%i1/itext/inum across one and two
compiled levels; ilev() composes; dynamic-depth itext/inum resolve
outer levels including across three-level nests and interpreted
enclosing iters. Full suite 35/0; iter_nest_fn TC004 pins the shapes.
Note for posterity: the first probe battery used word(), which does not
exist in TinyMUX — itext(#-1 ...) atoi'd to 0 and both routes agreed on
the "wrong" answer. extract() is the equivalent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Block LAYOUT order is not EXECUTION order. The lowering allocates an
outer iter's latch block before the blocks of an inner loop it encloses,
and the program's result-assembly block before later loops. Linear-scan
intervals were computed over layout points, so a value defined in an
outer body and used in its latch was considered dead exactly where the
inner loop's points sit — the allocator handed its register to
inner-loop values, the latch stored inner_final+1 into the shared
counter q-slot, and a triple-nested iter exited after one trip: N^2
results where N^3 is correct, silently, on the default configuration.
Two-deep nests worked only because their latch happened to land after
the inner loop in layout order. The same hole in the string
output-buffer allocator let LATER loops recycle EARLIER fragments'
result slots (a words() result came back as a later lnum's "0 1 2").
The fix is an execution-reachability liveness closure in
compute_live_ranges: for every use of v, extend v's interval to every
block B with def->B and B->use both reachable (back edges included).
Cycles make a loop's blocks mutually reachable, so the loop case falls
out; the out-of-order straight-line case is the acyclic instance. RPO
was tried and rejected — a reverse postorder may also place a latch
before the body it follows at runtime (measured: it moved the bug to
the middle loop). The closure replaces the old rpo-range extension,
which both under-extended (values defined inside the loop got nothing)
and mis-tested membership (rpo_pos ranges are not loop membership).
Verified live at depths 2/3/4 with per-level markers (o#@/m#@/i#@
asserts every level's element and position), the mixed-consumer
single-program shape, and lnum + literal lists: all byte-identical to
the AST route. Warm perf unchanged: iter(lnum(100),[ifelse(...)])
min-of-5 0.00043-0.00055s/30 iters vs the same box's 0.00046-0.00056
baseline before the fix; jit_handled 30/30 throughout.
New smoke file iter_nest_fn.mux: TC001 pins depth-3/4 structure and
counts, TC002 pins the cross-fragment slot-survival shape. The corpus
previously contained zero triple-nested iters — this class was
invisible to every existing net, and only VALUE assertions can see it
(jit_handled reported success while wrong).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The compiled ITER loop lost to the interpreter's C loop per element, and
its per-element cost climbed with N while the interpreter's stayed flat.
Profiling (guest->host block map + ELF symbolization of anonymous JIT
frames) found the mechanism the issue asked for: the tier-2 string ABI.
Every argument crosses as decimal text, so each element paid SIX
integer<->string conversions — ITOA(cursor) caller-side, satoi(cursor)
callee-side twice (elem + next modes), sitoa/ATOI for the next offset,
ITOA/satoi for the append's length and iteration number, sitoa/ATOI for
the new length — digit-proportional loops, executed under DBT expansion,
on numbers (byte cursors, accumulated lengths) whose digit counts grow
with the list. That is both the bulk of per-element cost and the whole
of the residual superlinearity: cost/element ~ a + b*digits(N).
The fix is the calling convention the machine already has. HIR_CALL_T2I
loads TY_INT arguments straight into a0.. from their registers and
TY_STRING arguments as guest addresses, JALs to the blob, and takes the
callee's long return from a0; arguments ride carg[] so liveness, DCE and
copy propagation see them (hir_is_carg_call). val[i]=1 allocates an
output slot passed ahead of the args; HIR_T2I_STR aliases it as the
element string. Two int-native blob entrypoints replace the string
pair: rv64_split_step (ONE call per element instead of two — element
written to out, next cursor returned in a0) and rv64_append_i. The next
cursor stays an SSA value stored in the latch, preserving the
nested-iter safety the string route had.
Measured (macOS arm64, min-of-5, ast=/cached= per CLAUDE.md):
iter(lnum(N),1) us/element ratio vs interpreter
N before after before after
200 0.117 0.030 0.75 0.19
1000 0.125 0.030 0.81 0.20
4000 0.145 0.033 0.89 0.21
4x on the compiled path; the loop that opened the issue LOSING 1.42x on
Linux is 5x FASTER than the interpreter here. Fixed-width N-pair probes
no longer climb (median ratio 0.90, was 1.09 at 10/10 above 1.0), and
the per-char slope halves (one scan per element, not two). MAP/FILTER
still compose the string route; converting them is the follow-up.
Two silent-fallback traps found en route, each now carrying a warning:
- tier2_allowed()'s allowlist quietly vetoed the new names: lookup
returned 0, the lowering kept its graceful string fallback, and every
test passed while the fix did not run. Ground truth came from
disassembling the compiled program out of the SQLite code cache — the
JAL targets do not lie. The allowlist comment now names this failure
shape.
- A same-mtime-second edit after a build produced an engine that had the
new symbols in source and nowhere in the binary (#2118's genre).
Also rides along: the env-gated profiling diagnostics that found this —
TINYMUX_DBT_MAP (guest-pc -> host-address lines at translate time) and
TINYMUX_DBT_CODEDUMP (code_buf + block-cache table at cleanup), which
together let a sampling profiler's anonymous JIT frames be symbolized
against the blob ELF.
Full make test EXPECT_CONFIG="jit=yes": 35 passed, 1 skipped (stubslave,
not configured), 0 failed — including smoke's 1660 goldens, the jit
parity suites, and the format guard. Functional probes cover nested
iter, #@/##, custom in/out separators, runtime leading spaces, and empty
lists, all exact.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The SQLite code_cache is keyed on blob_hash (s_blob_version), whose tier 1 leg
was a single __DATE__/__TIME__ in jit_compiler.cpp. Its comment stated the
assumption:
"Because the JIT requires a clean rebuild to take effect, this file is
recompiled every such build, so its __DATE__/__TIME__ stamp changes and
folds into the hash, invalidating every previously persisted entry
automatically."
Under incremental make that is false, and false in exactly the case the stamp
was added to defend against. Its own comment names "a codegen change in
another TU (e.g. hir_codegen.cpp) with no version bump" -- and a change in
another TU is precisely when jit_compiler.cpp is NOT recompiled.
Measured before the fix: touching hir_lower.cpp rebuilt hir_lower.eo and left
jit_compiler.eo untouched, so blob_hash stayed byte-identical at ad598f6b...
and every previously persisted entry still matched. The cache then served the
previous build's compiled output -- which contaminated a #2052 retest, where
TINYMUX_DUMP_HIR counted zero compilations while the old behaviour still ran.
On a live game it means attributes keep running the old compiler's output after
an upgrade until something evicts them.
Each tier 1 unit now carries its own stamp and all of them fold into the hash,
so the key moves when any of them is recompiled. Folded into BOTH
s_blob_version computations; the no-blob fallback had the identical hole and is
the leg a blob-less build runs on.
The set is deliberately explicit rather than a glob, because the invalidation
model is three-legged and only two legs belong:
softlib.rv64 MUST invalidate -- cached RV64 calls into it via a PC-relative
JAL to a resolved blob address (hir_codegen.cpp:2262), so
moving a blob function makes cached code jump into the middle
of something else. Already covered.
tier 1 JIT MUST invalidate -- it emits different RV64 for the same
softcode. This commit.
the DBT MUST NOT invalidate -- it only executes the stored RV64 and is
not baked into it (the cache holds guest RV64, never host
code). Invalidating for a dbt_*.cpp change would discard the
cache for no gain.
Verified by rebuilding after touching each unit in turn and reading blob_hash
from a fresh database. All six tier 1 units move the key; dbt.cpp and
dbt_interp.cpp leave it unchanged -- the negative control matters, since a fix
that simply invalidated on everything would pass the positive half and be
wrong.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Spill slots lived at SP-8, SP-16, ... -- BELOW the stack pointer. That kept
them clear of the output buffers, but RV64 has no red zone: the first JAL
into a blob function let the gcc-compiled callee build its frame right on
top of them. A spilled value reloaded after any tier2 call read the
callee's dead locals.
Nothing ever noticed because nothing ever both spilled AND called. The
allocator has 10 registers, and no compiled program had held more than 10
integers live across a call until the ITER cursor rework added one int PHI
and one ATOI per loop. Then:
[iter(A,%i0)][iter(B,%i0)] gave "A B" (expected "AB")
parser_fn TC020 (four loops) gave "X Y Z31" (lost elements)
The pinning observation: in the same loop iteration, #@ (inum+1, register
operand) printed 1 2 3 correctly while is_first = EQ(inum, 0) (spilled
operand, reloaded after two split_token calls) read garbage. Same SSA
value, right at one use, wrong at the other -- with tier2 calls in between.
More loops meant more spills meant grosser corruption, which is why the
4-loop smoke expression failed harder than anything typed at `think`, and
why every isolated test in the WIP handoff was correct: none of them
spilled.
Slots now sit at +8*slot from the post-prologue SP, inside the frame the
prologue reserves; the backpatch adds a 16-byte-aligned spill area below
the output slots and emits the SUB even when there are no output slots.
Callee frames start below SP and cannot reach either region.
Validated by perturbation both ways: with only this hunk reverted, the
two-loop expression reads "A B"; restored, "AB". Full suite: 36 targets,
35 passed, 1 skipped (stubslave=no), 0 failed.
Latent on master for any spilled program that makes calls; the ITER work
is merely the first to compile one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Value-producing `return x == "0"` is EQK+JMP+LFALSESKIP+LOADTRUE, but
EQK was never in the bool fuse and always declined on TY_LUA_HANDLE.
Add ECALL_LUA_EQ / HIR_LUA_EQ, fuse EQK like EQ/EQI, and route handle
and non-handle EQK through emit_cmp_branch. tostring(0)=="0" and
return nil=="" now execute; AGREE_DECLINE_BUDGET 13→11 on current master.
#1764 shape 2: default STRING-claim calls no longer marshal to TY_STRING
text immediately. ECALL_LUA_CALL_VAL leaves the first pcall result on the
Lua stack and returns a TY_LUA_HANDLE. Softcode boundary marshals via
ECALL_LUA_MARSHAL (fun_lua rules). TEST/NOT use ECALL_LUA_TOBOOL so
tostring(0)/"" / string.find stay truthy under Lua semantics.
CALL_INT / CALL_VOID unchanged. Equality of a value handle to a string
constant still declines (needs VM compare).
AGREE_DECLINE 16→12: the four #1764 if/not pins now execute.
Default-on (#1745) exposed that the compiled path baked the back-edge
budget into each program as a constant:
int budget_init = h.emit(HIR_ICONST, TY_INT, -1, -1,
static_cast<int64_t>(mudconf.lua_instruction_limit));
A compiled program is cached in memory and persisted in code_cache, so the
limit in force was whatever was configured when the chunk happened to
compile -- @admin lua_instruction_limit reported Set. and changed nothing,
which is #1613's bug arriving on the compiled path. test-config's
runtime-bounds case caught it the moment the flip put the compiled path in
its way; on default-configure trees --enable-jit is off and everything
stayed green, which is why the flip validated cleanly elsewhere.
## Fix
A dedicated no-arg ECALL, following the LUA_LEN pattern end to end:
HIR_LUA_INSN_BUDGET -> ECALL_LUA_INSN_BUDGET (0x314)
-> a0 = mudconf.lua_instruction_limit, read per run
The entry seed becomes ECALL + STORE_Q, so the program carries no config
value at all. That fixes the in-memory cache and makes the persisted
code_cache safe by construction rather than by flush discipline; blobs from
before this change carry the old entry-store, and JIT_BUILD_STAMP
(__DATE__ __TIME__) already invalidates them on rebuild.
The handler clamps the limit to >= 1: a zero or negative limit must abort
loops, not arm an effectively unbounded unsigned countdown.
The op is listed in needs_int_reg() -- whose own comment documents that
omitting an int-producing ECALL fails silently (codegen emits nothing and
the consumer reads garbage), which is the trap this listing avoids.
## Verified
FAIL: lowering lua_instruction_limit at runtime had no effect before
ok: lua limits apply at runtime, both directions, and read back after
## Second layer found under this one: #1748
Full smoke under default-on now completes and fails exactly two cases
(TC013 mux.name, TC014 mux.eval -- compiled path answers wrongly instead of
declining). On the UNFIXED merge commit those cases are unreachable: the
smoke chain stalls at 164/320 dispatched with 800+ lost verdicts. So this
fix converts a catastrophic stall into two known failures, filed as #1748
with the pre/post evidence. make test on JIT trees stays red at those two
until #1748 resolves; lua_jit 0 remains 1561/1561.
Also amends plan-lua-jit-product.md, replacing "residual optional polish
only" with the post-flip regression record -- as promised in the #1747
review.
Refs #1325, #1613, #1732, #1747, #1748.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Numeric `for` compiles and runs; `while`/`repeat` (backward JMP) and
generic for (TFOR) still decline. Three things had to be true at once:
RIGHT SEMANTICS. The FORPREP/FORLOOP lowering behind #1326's reject
implemented Lua 5.3 -- signed sBx offsets, init-step pre-subtraction --
against a 5.4 VM, and had never executed. 5.4's FORPREP falls INTO the
body (jumping forward past FORLOOP only on a zero trip count) and
FORLOOP jumps BACK by an unsigned Bx. First cut takes STATIC BOUNDS
only: init/limit/step must be integer constants, so trip direction,
zero-trip, and freedom from wraparound are compile-time facts -- 5.4's
counter model exists precisely because a naive idx<=limit test misses
at the integer edge, and declining the edge is cheaper than reproducing
the counter.
LOOP-CARRIED VALUES. A plain HIR value crosses blocks only under
dominance, and the #1422 transition drops the rest -- fatal for the
accumulator in `for i=1,4 do s=s+i end`. Loop protos now route Lua
registers through q-registers (reg r -> qreg r), the one traffic
hir_ssa_construct PHI-converts: store-at-write after every
non-terminator instruction, reload at every block entry. Backing is
claimed only where every path stores first -- the entry block, or
FORLOOP for its visible index, whose readers the latch dominates. The
first draft skipped reloads for registers still holding entry
CONSTANTS; the harness answered `return s` with the loop INDEX --
dominance is availability, not currency, and inside a loop the entry
value is one iteration stale. Reloads are unconditional now, and
FORLOOP reads its ICONST bounds from entry_final[], the register state
frozen at the entry block's exit.
EXHAUSTION THAT ABORTS. The old budget folded exhaustion into the
loop condition -- an early exit with a WRONG PARTIAL SUM, which is what
#1326 refused to ship. Each back edge now branches to a shared block
whose ECALL_LUA_LIMITED declines the entire run; the caller fails over
to the interpreter, which re-runs the chunk into its own hook and
raises "#-1 LUA ERROR: instruction limit exceeded" -- the player sees
the interpreter's error verbatim, from one budget. The re-run is why
loop protos must be RERUN-SAFE: eligibility rejects calls, SELF and
SETTABUP inside them, and the referent (#1725) declines stores into
global-shaped tables while chunk-local NEWTABLE stores stay compiled.
EXEC pins the accumulator, an order-sensitive a*10+i, and the
zero-iteration path (where a reload of a never-stored qreg would read
the surrounding command's %q). AGREE pins budget exhaustion against
the interpreter's error text. AGREE declines: 5 of 35, every one
deliberate.
luajit: 133 chunks, 0 wrong, 0 crashes; smoke 1561/0; make test clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The third argument is what finally graduates CALL_INT/CALL_STR off the
packed val[] encoding: fn stays src1, the arguments ride the carg[] list
exactly as HIR_CALL's do (emit_lua_call), and val[] keeps only the
two-bit argument kinds. hir_val_operand's CALL branch and
hir_operand_set's re-packing hack are DELETED rather than grown a third
shape -- every operand-walking pass now sees call arguments through the
ARG slots, closing the seam that made CALL_INT's second argument
invisible to liveness (20d39472f) and that the codegen comment had been
naming since the packing landed.
On that footing, three call-surface features:
* Three arguments (string.sub). The third rides x14, so CALL_STR's
out addr/size shift to x15/x16 -- an internal encoding, changed
everywhere in this commit.
* Handle arguments, kind 3: the register carries the stack index and
the handler does lua_pushvalue -- the one use of a handle that is
ABOUT the thing it points at (#1579), which table.concat({...},",")
needs. Codegen's register move for kind 0 was already exactly right.
* HIR_LUA_CALL_VOID for nresults == 0 (table.insert): pcall asked for
zero results, nothing to type-check. It exists only for its side
effect and produces no value, so nothing downstream can keep it
alive -- it joins has_side_effects(), without which DCE NOPs it
(#1145's SETI lesson).
EXEC pins each: three args, a handle arg with a string result, and the
void call read back through t[2] -- #t alone stays plausible when an
insert silently never ran, but the inserted element cannot. AGREE
declines fall 8 -> 5; the survivors are all deliberate (loops x2,
os.time, string.find's result-count pin, select's four arguments).
luajit: 129 chunks, 0 wrong, 0 crashes; smoke 1561/0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
math.pi, math.huge and math.maxinteger declined: a field read on a
library table takes a reference, and a handle to a number is something
nothing downstream can consume. The referent now carries a table of
known VALUE members, and the field read takes the value itself --
GETFIELD (INT) for maxinteger/mininteger, the new GETFIELD_FLT for
pi/huge, whose double rides the same raw-bits lane the call arguments
use and lands directly in its FP slot.
Function members deliberately do NOT join the table: their return
claims stay in lua_call_claim, so each fact lives exactly once. And as
with every referent claim, this is eligibility, not soundness -- a game
that rebinds math.pi to a string declines at the handler's type check
(a genuine float only; lua_isnumber alone would coerce "3.7" and
integers, which have their own routes).
HIR_LUA_GETFIELD_FLT is the first Lua opcode producing TY_FLOAT, which
found the FP twin of the needs_int_reg() silent-registration trap:
a float producer missing from needs_fp_reg() gets no slot, loc[].addr
stays 0, and the post-ECALL store writes guest address 0 (#1159's
shape). Noted in both places.
EXEC pins each value USED in arithmetic, not just returned -- an
unwritten slot cannot survive x*2 or x-1. AGREE declines fall 11 -> 8;
ratchet tightened as the harness requires.
luajit: 126 chunks, 0 wrong, 0 crashes; smoke 1561/0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
math.floor(3.7), math.type(3.0) and math.type(2^3) all declined because
the call argument walk accepted only integers and constant strings.
Floats could not be smuggled as rendered text -- coercion lies to a
type-sensitive callee: math.type("3.0") is nil, not "float" -- so they
travel honestly, as raw double bits through the integer argument
register, the same FMV.X.D lane ECALL_LUA_FTOA already proved on both
execution routes. Runtime floats work identically to constants: the
value FLDs from its FP slot at call time.
The argument encoding widens from one kind bit to two per argument
(0 integer, 1 string address, 2 double bits), and the widening forced
the factoring the duplication deserved anyway: one emitter in codegen
(emit_lua_call_args) and one decoder in the handler
(ecall_lua_push_call_args) replace two near-identical copies of each --
the same #1457 drift shape the lowering's twin call branches were merged
out of. The packed kind bits are the single source of truth end to end;
codegen no longer re-derives argument shapes from h.kind.
Four EXEC cases pin the lane: a float constant to each result variant,
two floor calls with different fractions (catches a reused argument
slot), and a RUNTIME float a constant-folding accident cannot fake.
AGREE declines fall 15 -> 11; ratchet tightened in this commit as the
harness requires.
luajit: 124 chunks, 0 wrong, 0 crashes; smoke 1561/0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tostring(42) -> 42 type(42) -> number tonumber("17") -> 17
Decline count 23 -> 15. Eight chunks, the largest step so far, and most of
it came from REMOVING things rather than adding.
Two changes.
A callable handle may now come from GETGLOBAL as well as GETFIELD_REF.
tostring is the global itself, a function rather than a library table. The
lowering does not try to know which globals are functions -- the ECALL
already checks lua_isfunction, so math called rather than indexed declines
there.
And CALL_INT and CALL_STR now share ONE argument encoding. CALL_INT took
integers only; CALL_STR took integers or constant strings. That difference
was never justified: the two differ in RESULT type, so they have no business
differing in how arguments arrive. tonumber("17") is the case that proves
it -- string argument, integer result, needs both halves. The
tonumber(mux.args[1]) family fell out for free and was not on the list.
WHAT THE LOWERING NOW GUESSES, AND WHY IT IS ONE QUESTION
math.max(3,9) and tostring(42) have identical argument shapes and opposite
result types. HIR result types are STATIC and Lua's are not, so nothing at
the call site distinguishes them and the callee's NAME is the only carrier.
lua_callee_returns_int is a short whitelist; a name not on it declines, and
declining is always correct.
That is the second place the lowering guesses what a handle points at. The
first is GETFIELD_REF vs GETFIELD_INT by provenance. Both are the same
question -- "handle to WHAT" -- and both want the same fix: a handle type
that carries its referent, including a function's return type. Two
independent pressure points now argue for it rather than one.
Tests pair tostring with tonumber deliberately: same argument shapes,
opposite result types, so an implementation inferring the result type from
the arguments cannot pass both. All three report lua_run_ok=0 on master.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
needs_output_buffer omitted CALL_STR, so loc[i].addr stayed 0 and every
string-returning library call aliased guest address 0. Single-result
EXEC cases still pass by luck; concurrent/live values would clobber.
string.upper("ab") -> AB string.lower("AB") -> ab
string.rep("ab",2) -> abab
Decline count 26 -> 23. This answers the convention #1713 deliberately
deferred: where does a string result go, and who bounds it.
It goes in the output slot the register allocator already gives any
TY_STRING value, and the ECALL is told that slot's SIZE rather than assuming
it. ECALL_ORD is why: its bound read like one and was not -- 64 bytes of
headroom against a loop writing per codepoint -- and it wrote 15k (#1679).
Two choices made against the lazier option:
Decline on overflow, do not truncate. A silently shortened string is a
wrong answer, and the interpreter can produce the whole thing.
Accept LUA_TSTRING only, rather than lua_tolstring on anything. That
function coerces a number AND mutates the stack slot in place, which would
disturb a live handle and make number->string conversion the JIT's rules
instead of Lua's. ITOA/FTOA already carry the interpreter's rules.
Arguments may be integers or constant strings; a kind bit per argument tells
the handler which register holds which. A runtime string argument needs its
own guest buffer and waits for something that needs it.
#1715 EARNED ITSELF HERE. CALL_STR packs three fields into val[] --
nargs | kinds<<8 | arg1<<16 -- and teaching the operand accessor about it was
ONE edit. Before that refactor it was four separate walks, three of which
fail silently when missed, and all four would have needed it while the
string convention was also being designed.
A test that could not fail, caught in the minute it was written: I added
string.sub("hello",2,3) as an EXEC case with a comment claiming it covered
mixed argument kinds. It takes THREE arguments against a ceiling of two, so
it declines and can never execute -- the comment asserted coverage that did
not exist. string.rep("ab",2) is genuinely string-plus-integer and does
cover it. The EXEC contract caught this, because a declining chunk there is
a hard error rather than a pass.
All three cases report lua_run_ok=0 on master.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The call path in #1713 proved the IR could not express what the upper layer
needed, and named it: an operand list every pass can walk without knowing
the opcode. This is that, at the access layer rather than the storage
layer.
FOUR copies of one walk existed -- src1, src2 with a BRC exception, val[],
then the carg[] loop:
hir_opt.cpp copy propagation
hir_opt.cpp dead code elimination
hir_codegen.cpp liveness / last-use
hir_codegen.cpp loop live-range extension
Each new operand shape had to be added to all four, and missing one does not
fail loudly: the live range ends at the definition and the register
allocator hands the register to something else while the ECALL still expects
it. CALL_INT's second argument was invisible to some of them until
20d39472f.
Now: hir_operand_count / hir_operand_get / hir_operand_set, and the four
walks become
for (int sl = 0; sl < hir_operand_count(h, i); sl++)
... hir_operand_get(h, i, sl) ...
Three pieces of scattered knowledge move into those functions:
BRC keeps a BLOCK NUMBER in src2, not an operand. DCE knew and skipped
it; the liveness walks each re-derived it; nothing enforced it.
val[] is a plain index for SETI/SETFIELD and PACKED with nargs for
CALL_INT. copy propagation wrote `h.val[i] = r` unconditionally, which
preserves neither. Latent rather than observed -- I could not construct a
chunk where resolve_copy fires on a call argument -- and exactly the class
a hand-rolled walk keeps re-introducing. hir_operand_set now re-packs.
Argument lists live in carg[] for CALL/STRCAT and pval[] for PHI.
What deliberately did NOT change: hir_codegen's per-opcode case blocks still
read src1/src2/val[] directly. Codegen for HIR_LUA_SETI should know where
SETI keeps things. The rule is that GENERIC passes use the accessor and
OPCODE-SPECIFIC code knows its own opcode.
Access layer, not storage. The failure mode -- a pass silently missing an
operand -- is gone without moving any data, so the existing suite validates
it cheaply. If storage does need to change later, every generic pass now
goes through one interface and that change becomes mechanical.
Behaviour identical: 8 executing / 26 declining before and after, no wrong
answers, JIT q-register and ifelse oracles agree. make test green;
test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
math.max(3,9) -> 9 math.min(3,9) -> 3 math.abs(-7) -> 7
Decline count 29 -> 26, the largest single step so far, and the first that
crosses THREE ECALLs in one compiled run:
GETGLOBAL name -> stack index of the library table
GETFIELD_REF table + name -> stack index of the function
CALL_INT function + up to two integer args -> integer result
#1519 flagged stack discipline across ECALLs as needing proof rather than
assumption. It holds: a handle from one ECALL stays valid across the next
two inside a run, bounded by TryJIT's settop around the whole thing.
Narrow on purpose. Integer args, integer result, nothing marshalled --
math.floor(3.7) still declines because its argument is a float. That keeps
this increment about STRUCTURE and leaves the string-result convention to be
decided on its own, where a bad choice would be expensive (see ECALL_ORD's
unbounded write, #1679).
WHAT THIS BUILD PROVED THE IR CANNOT DO
Two seams, neither of which I would have written down from taste:
1. An instruction needs N OPERANDS. A call has three or four; the IR has
src1, src2 and val[]. So nargs is bit-packed with an instruction index
into val[]:
int64_t packed = nargs | (int64_t)(a1 + 1) << 8;
That is val[]'s FOURTH meaning after immediate, guest address, and
SETI's third operand -- and hir_val_operand() cannot see the arg1 index
at all. Nothing breaks today only because two-argument calls are simple
enough that liveness incidentally holds. That is luck, and it is the
same shape that produced #1711's wrong answer.
What the upper layer needs, plainly: an operand list every pass can walk
without knowing the opcode.
2. The type system knows "handle" but not "handle to WHAT". Choosing
GETFIELD_REF over GETFIELD_INT is a guess from provenance -- did this
handle come from GETGLOBAL or from NEWTABLE -- because a library member
is a function and a data-table member is a value, and TY_LUA_HANDLE
cannot tell them apart. Second place this pass has had to guess.
Both are recorded rather than worked around silently, because the refactor
they argue for should be driven by what the upper layer demonstrably needs.
Tests use asymmetric two-argument calls, max(3,9) and min(3,9), for the same
reason #1711's use two distinct keys: one argument cannot distinguish
correct passing from an argument being ignored or the pair being swapped.
All three report lua_run_ok=0 on master.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
local t={a=3,b=4} return t.a+t.b -> 7 run_ok, no fallback
local t={} t.x=5 t.y=6 return t.x*10+t.y -> 56
Decline count 30 -> 29.
The key travels as an ADDRESS into the program's own string pool, never as
marshalled text, so nothing downstream can mistake it for a value. Only the
integer result comes back, in a register; a non-integer field declines
inside the handler rather than guessing a marshalling.
Two more registration points, both silent if missed, and both found by
READING rather than by debugging -- hir.h's comment about val[] operands is
what prompted the check:
hir_val_operand() was gated strictly on HIR_LUA_SETI. SETFIELD parks
its value there too, so the liveness walker would not
have seen it and the register could be recycled before
the ECALL read it.
has_side_effects() SETFIELD is a store with no result. DCE deletes it.
That makes seven places a new opcode may need to appear, four of which fail
with no diagnostic: enum, lowering, codegen case, needs_int_reg,
hir_kind_name, hir_val_operand, has_side_effects.
Also the k flag again: OP_LUA_SETFIELD's C is a CONSTANT index when k is
set, exactly as OP_LUA_SETTABI's was, so `t.x=5` declined until it was
handled.
A WRONG ANSWER, caught before merge and worth recording
local t={a=3,b=4} return t.a+t.b answered 8, not 7
HIR_SCONST lives as loc[].addr with in_reg=false. Passing the key through
ra_get_reg returned a register that was never loaded, so a1 held the same
stale address on every call and every field read returned the LAST value
written -- 4+4.
The harness was fully green while that was true: agree_wrong 0, exec_wrong
0, and my own first EXEC case `local t={a=7} return t.a` PASSED, because
with one key "always return the last write" is indistinguishable from
correct. A probe with two distinct keys is what exposed it.
So the EXEC cases here read TWO DISTINCT KEYS deliberately. For any keyed
operation a single-key test proves almost nothing: stale key register, key
ignored, and all-keys-alias are each invisible unless two keys are read back
independently.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
local t={1,2,3} return #t -> 3 run_ok, no fallback
3 is the number that used to come back as 22.
The lowering measured a stack INDEX that the named bridge had marshalled out
as a decimal string -- strlen of the text, not the length of the table.
#1579 contained it by typing handles and declining `#` on one, which kept
the answer correct at the cost of never compiling it. ECALL_LUA_LEN_INT
asks the VM instead, and the index never leaves a register where anything
could measure it as text. Correct AND compiled.
lua_rawlen is only equivalent to `#` for a table with no __len metamethod,
and that is exactly what ecall_lua_plain_table already refuses -- range,
istable, and no metatable. The guard is load-bearing here, not incidental.
Decline count 31 -> 30, the ratchet's second fire in the improving
direction. It failed the build until AGREE_DECLINE_BUDGET moved with the
change, which is the point of it.
All five registration points, hit deliberately rather than discovered:
1. hir_kind enum hir.h
2. lowering hir_lower_lua.cpp
3. codegen case hir_codegen.cpp
4. needs_int_reg() hir_codegen.cpp
5. hir_kind_name() hir_codegen.cpp
plus the ECALL constant and its handler. 4 and 5 are the ones that bite:
omitting 4 fails SILENTLY, because codegen's `if (!dest) break;` emits
nothing and the consumer reads a stale register (measured on NEWTABLE as
`SETI idx=0 type=function`); omitting 5 only shows up in a dump and was
caught after merge by someone else. This list is here so the next increment
does not rediscover either.
EXEC cases for both shapes; on master both report "lua_run_ok=0 (compiled
path did not execute)".
table.insert(t,4) still declines -- a library CALL needs the
global-lookup-and-call path, not a table primitive. Different shape.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First table chunks to EXECUTE rather than decline:
local t={} t[1]=5 return t[1] -> 5 run_ok, no fallback
local t={} t[1]=7 t[2]=9 return t[1]+t[2] -> 16 run_ok, no fallback
Computed by the JIT, not by the interpreter covering for a failed run.
Three pieces. HIR_LUA_NEWTABLE is new: table creation had only the named
HIR_CALL form, which marshalled the resulting stack index through guest
memory as a decimal string and never completed. HIR_LUA_SETI already had
codegen and nothing lowered to it, so OP_LUA_SETTABI now emits it instead of
the named call. And SETTABI's value can be a CONSTANT rather than a
register when the k flag is set -- reading lua_reg[C] there yields -1, which
is why `t[1]=5` declined even once the rest was wired.
Integer values only. The dedicated ECALL carries the value in a register,
so there is nowhere for a string to ride; anything else declines and the
interpreter answers, correctly. That is the direction #1309 settled: an
index typed TY_LUA_HANDLE and passed in a register, which lua_is_handle can
refuse to let escape into arithmetic, rather than an untyped index passed as
text where "22" is indistinguishable from 22 (#1424).
A fourth registration point, which is the part worth knowing: a
value-producing opcode must also appear in needs_int_reg() in
hir_codegen.cpp. Enum, lowering and codegen are the obvious three; omitting
the fourth fails SILENTLY, because codegen's `if (!dest) break;` emits
nothing at all, the ECALL never runs, and the consumer reads whatever was in
the register. Measured as `SETI idx=0 type=function` -- found by tracing the
index across the ECALL boundary, which reading the code would not have shown.
Tests go in EXEC rather than AGREE deliberately. Agreement is also what a
decline produces, so only lua_run_ok > 0 separates "the JIT computed this"
from "the interpreter did" (#1426). Verified both directions: on master both
cases report "lua_run_ok=0 (compiled path did not execute)".
agree_declined stays 32. The harness's other table chunks use CONSTRUCTORS,
which lower through SETLIST -- a different path, and the next increment.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lua_mod 6, hir_lower 3, hir_lower_lua 2, hir_codegen 1 -> 0. With
jit_compiler in the previous commit that is every JIT-owned site that CAN be
converted; legacy total 79 -> 41.
Two checks rather than assumptions, either of which would have been a silent
behaviour change:
%.14g in hir_lower_lua is a float rendered onto the compiled path, and
mux_vsnprintf implements %g by hand -- if it disagreed with libc, JIT and
interpreter would print different numbers for the same value. Compared
across 19 values including 1e-300, 1e+300, the 1e15/1e16 exponent-form
boundary, and 9007199254740993 (past exact integer representation): zero
differences.
Every converted site also loses its `if (n < 0)` / `if (n >= size)` clamp.
Those are not redundant, they are dead: snprintf returns what it WOULD have
written, mux_snprintf what it DID. lua_mod's five result writes lose the
explicit pResult[n] = '\0' for the same reason. That difference is why each
site was re-read rather than renamed.
dbt_test.cpp (22) and dbt_x64_div_harness.c (1) cannot be converted: tests/dbt
builds them with "$(COMPILE) -o $@ $(SRCS)" and links no libmux, so
mux_snprintf is not reachable without changing that build. Frozen with the
reason recorded rather than exempted -- an exemption would let new sites in,
a frozen count still may not grow, and neither file writes player-facing
text.
The remaining 18 are outside the JIT split: attrcache 4, ast 3, mail 2,
match 2, predicates 2, and six singles.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review of #1664 found that lifting the reentrancy guard is not safe on its
own. `make test` excludes `test-lua-jit` -- it is opt-in -- so the target
named for the configuration the PR changes was never run:
master Succeeded: 1561 Failed: 0
nesting, no fixes Succeeded: 1558 Failed: 3
All three are wrong answers, not crashes, and all three were previously
unreachable only because the refusal sent every nested lua() to the
interpreter.
TC020, `#mux.args` answered 8. A mux.* table is carried through lowering as
an SCONST holding its own NAME, so OP_LUA_LEN's TY_STRING branch measured the
sentinel: strlen("mux.args"). It is not a handle, so the #1424 guard above
does not catch it. Decline; resolving to the call's ncargs is #1519's work.
TC009, the instruction limit stopped applying. Limits live in
CLuaMod::InsnCountHook, a Lua VM hook the compiled path does not have --
RunCompiled has only max_dispatch and the wall alarm, and a loop spinning
inside one translated block issues no dispatches. `while true do end`
answered an empty string where the documented result is
"#-1 LUA ERROR: instruction limit exceeded". New LUA_BC_HAS_LOOP rejects any
backward branch. Bounded loops go too: trip counts are not known here, and
guessing on the unsafe side is how this was reachable at all.
TC059, INT64_MIN rendered as -'..--).0-*(+,))+(0(). rv_emit_itoa negated the
value to get a magnitude, but -INT64_MIN wraps and stays negative, so REM
produced negative digits and '0' + (-d) wrote below '0' -- every byte off by
twice its digit. Negating a POSITIVE value cannot overflow, so accumulate in
negative space and negate each digit instead, where the magnitude is at most
9. INT64_MIN then needs no special case. Swept 0, +/-1, +/-10, +/-100,
INT64_MAX, -INT64_MAX and INT64_MIN as both identity and negation.
TC020 and TC059 also fix pre-existing wrong answers under
`lua_jit 1, jit_eval_brackets 0`, which is how they were reachable before
nesting.
The tier that should have caught all three was mine, and did not: it ran
three pure-arithmetic chunks, so it stayed green while smoke regressed. It
now carries all three shapes -- but the first attempt at that was no better,
because two of them passed against the broken build:
- a loop WITH a body declines for unrelated reasons; only TC009's bare
`while true do end` compiles
- run_one passes (2,3), so `-mux.args[1]` renders -2 and never reaches the
one input in 2^64 that fails -- reach INT64_MIN by overflow instead
Verified in both directions rather than assumed: pre-fix nested_wrong=3,
post-fix nested_wrong=0.
Two tiers, because two of the shapes now DECLINE and declining is the fix:
NESTED requires lua_run_ok > 0, NESTED_AGREE requires only agreement. A
decline that returned the wrong answer would still be a bug and only a
comparison sees it.
Also moves AGREE_DECLINE_BUDGET out from under the EXEC header and gives
NESTED its own banner, per review.
test-lua-jit 1561/0 matching master; make test green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The HIR type lattice names where a value lives -- integer register, FP
register, guest memory -- which is the right shape for MUSHCode, where
everything is semantically a string. Lua is typed, and forcing it through a
representation lattice is what let a global holding the integer 22 and a
table at stack index 22 come back byte-identical: both were TY_STRING. `#t`
answering 22 was that, not a length bug (#1424).
TY_LUA_HANDLE carries both facts. Representationally it is still a string
buffer, so codegen needs no new machinery -- one line in
needs_output_buffer() and a name in the dumper. Semantically it is opaque:
the eight bridge calls that return a VM reference (__lua_getglobal,
__lua_getfield, __lua_geti, __lua_newtable, __lua_call, __lua_get_result)
now produce it, and arithmetic, bit ops, comparison, length, concatenation
and returning all reject it.
hir_lower.cpp is untouched, as predicted: MUSHCode never produces a handle.
Measured, interpreter as oracle, per-chunk disposition from jitstats:
chunk before after
local t={1,2,3} return #t bailed declined
local t={1,2,3} table.insert(t,4) ... bailed declined
local t={1,2,3} return #t + 1 bailed declined
local t={1,2,3} local n=#t return n*2 bailed declined
local x=math.floor(3.7) return x bailed declined
return math.huge bailed declined
local t={a=7} return t.a declined declined
local t={10,20,30} return t[2] bailed bailed
Six of eight move from a run-time bail to a lowering-time decline, answers
unchanged and still correct. `t[2]` is untouched because it routes through
ECALL_LUA_GETI_INT, one of the two bridge ECALLs that is actually live, and
returns a value rather than a reference -- so it is correctly not a handle.
Worth being plain about what this does and does not buy today. It is not a
correctness fix: #1518 already prevents the same wrong answers by failing
closed on unimplemented bridge names, and `#t` returning 22 is not currently
reproducible. What it buys is that the rejection stops being incidental.
#1518's protection holds only while the bridge names stay unmatched, and
evaporates the moment #1519 implements them -- at which point the ambiguity
is unanswerable, because a stack index and an integer are the same bytes.
This makes the rejection a property of the type instead of an accident of
what is unimplemented, which is what stops #1519 from re-introducing the
class as it lands.
make test green: smoke 1559/1559 both routes, tests/luajit PASSED with
agree_wrong 0, exec_wrong 0, exec_no_run 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lua 5.4 distinguishes integers from floats, and the distinction is
observable: tostring(3.0) is "3.0", math.type(3.0) is "float", and one
float operand makes the whole expression float. The compiled path threw
that away in three separate places.
- OP_LOADF loads a *float* whose value is the signed immediate. The
lowering read the immediate and emitted an integer constant, so
`return 3.0` produced the integer 3. Lua constant-folds arithmetic on
literals, so this also caught `4/2`, `2^3`, `7.0//2.0`, `1e3` and
`-3.0`, all of which reach the JIT already folded into a LOADF.
`return 4 / 2` lowered to a bare ICONST 2 with no FDIV anywhere.
- emit_lua_constant() deliberately demoted an integral float constant to
ICONST "for compatibility with integer arithmetic". That made
`a * 1.0` an integer multiply and `a + 0.0` print "3".
- return_as_string() formatted floats with "%.17g" and never appended
".0". Lua uses LUA_NUMBER_FMT ("%.14g") and appends ".0" when the
result looks like an integer (lobject.c tostringbuff), so the compiled
path disagreed with the interpreter on every float: integral ones lost
the subtype, and the rest printed more digits.
The fold now renders floats Lua's way, and the runtime path gets
HIR_LUA_FTOA / ECALL_LUA_FTOA, which is ECALL_FTOA's sequence against a
host formatter that follows Lua's rules rather than MUX's. Both share
lua_format_double(), so the compile-time fold and the run-time ECALL
cannot drift.
Verified against the interpreter as oracle, reading jitstats()
lua_run_ok/lua_run_fail per chunk rather than compile counts -- a chunk
that declines agrees trivially, and one that compiles can still bail at
run time and let the interpreter answer. 28 chunks, 11 divergences
before, 0 after, with 20 confirmed executing compiled code.
Smoke 1509/1509 on both routes, 316/316 dispatched, make test green.
Also re-measured the string->number half of #1425 (`return "3" + 4` → 6):
already correct on master, and covered by four cases here.
The `^` half of this work landed independently as #1548 while it was in
flight; this branch keeps master's version of that hunk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects, and the second hid the first.
1. The returned value was pinned to the first RET.
HIR_RET only emits an exit; the value reaches the caller through
rc.final_out, a single address derived at compile time from h.result.
h.result is set from the *first* return site -- deliberately, because Lua
appends a trailing dead RETURN0 whose empty SCONST would otherwise clobber
the real value (#1309).
That heuristic holds only while one return is reachable. With a branch, both
are, and the answer came from the first no matter which executed:
local x=1 if x>3 then return 777 else return 888 end jit 777 interp 888
local x=9 if x>3 then return 777 else return 888 end jit 777 interp 777
local x=1 if x<3 then return 777 else return 888 end jit 777 interp 777
local x=9 if x<3 then return 777 else return 888 end jit 777 interp 888
Always 777, correct only when 777 happened to be right. Any chunk with more
than one reachable return is affected; this is not specific to if/else.
Now every HIR_RET materializes its value into one shared output slot before
exiting, and final_out names that slot, so the value follows the path taken.
The copy mirrors the string-PHI materialization already in hir_codegen.
Only for multi-block programs. A single block runs top to bottom and exits at
its first HIR_RET, so first-return is the executed return there, and the
compile-time result is kept -- straight-line chunks like `return 42` keep
their constant folding rather than being forced to run.
2. emit_cmp_branch had the branch polarity inverted.
Lua's conditional ops are "if (cond ~= k) then pc++", and the pc++ skips the
JMP that follows, so the JMP is taken exactly when cond == k. true_target is
that JMP's destination, so the branch condition must be (cond == k): negate
when k is 0. It negated when k is 1. OP_EQK carries the opposite convention
-- no JMP to fuse, so its true_target is the skip -- and negating on k is
right there. The two look alike and mean opposite things.
Fixing either alone looks like nothing happened, which is why this sat:
neither always 777 (value pinned)
RET only 777/888 inverted (polarity now visible)
both 888/777/777/888 (matches the interpreter)
Verified against the interpreter with lua_jit 0 vs 1, classifying each case by
whether a code_cache row appeared -- a declined chunk agrees trivially and
reads as a pass. All the above compile. Also correct for three-way elseif
chains, string returns, and genuine runtime conditions (`mux.args[1]+0`) in
both directions. Straight-line chunks still compile and fold. Smoke
1505/1505 on both routes.
Smoke cannot cover this: lua_jit is default-off, so the suite exercises the
interpreter (#1426). The differential table above is the acceptance test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Completes the sweep the issue called for. mux_atol returns long, which
is 32-bit on LLP64, so every caller silently truncated on Windows. Two
of those were real defects (the truthiness family and cf_size, fixed in
the preceding commits); the rest were latent, waiting for a value large
enough to matter.
Rather than audit 290 sites for whether each can reach 2^31 today, use
the 64-bit parser everywhere and remove the class. A dbref cannot
overflow now, but nothing stops a later caller passing that same site a
timestamp or a byte count.
Pure 1:1 substitution: 285 lines changed, and every removed line
contained mux_atol while every added line contains mux_atoi64. No
control flow, no types, no behaviour beyond the wider parse.
This is a NO-OP on LP64 -- long is already 64-bit on Linux and macOS, so
the generated code there is unchanged. It only widens the parse on
Windows. Narrowing destinations are unaffected either way: `int x =
mux_atoi64(s)` truncates exactly as `int x = mux_atol(s)` did, on both
models.
Left alone: mux_atol itself in mathutil, its declaration, and three
comments that name it. Callers that genuinely want 32-bit semantics can
still ask for them; none appear to.
Verified on Windows: full solution builds clean with no new warnings,
smoke is 1418 passed / 16 failed / 0 crashes / 306 of 306 dispatched --
identical to before the sweep, with the same 16 build-configuration
failures (exp3 module not loaded, hmac/digest behind UNIX_DIGEST).
Spot checks after the change: the boolean family returns 1 for multiples
of 2^32, cf_size round-trips 3000000000 and still reads -1 as unlimited,
and arithmetic, string and list functions are unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Lua compile path never memcpy'd rc.code into guest memory (softcode
did). Compact/materialize then installed zeros at entry_pc; the DBT
translated illegal opcode 0 as exit_with_pc(same) and spun at guest PC 0
— the mux.args arithmetic "hang".
Also:
- ATOI/STRCMP of runtime_ref SCONSTs use guest addresses at runtime
(empty sval is not a compile-time constant).
- Keep needs_jit when sref_addrs is non-empty so return mux.args[1]
does not take the empty folded path.
Verified: hello/42/mux.args[1]//args alone, A2→A1, no dbt dispatch limit.
Conflict in hir_codegen.cpp: this branch adds the HIR_NEG case next to
HIR_ABS, and master removed HIR_ABS entirely (#1256/PR #1257). Kept
HIR_NEG, dropped HIR_ABS -- the opcode no longer exists in the enum, so
retaining it would not compile.
HIR_NEG was classified for int regs and named in dumps, but the main
codegen switch fell through default:break with no RV64 emission. Emit
integer negate as SUB dest,x0,rs. Guard const-fold of -INT64_MIN (C++ UB)
by keeping INT64_MIN, matching two's-complement wrap and i64Division style.
Add Lua smoke for runtime NEG of INT64_MIN and fold of -math.mininteger.
#1256: Remove integer HIR_ABS (enum, codegen, fold, purity). Softcode
abs() is float after #1150; the branchless ABS sequence corrupted
INT64_MIN into non-numeric text. Leaving the dead op would reintroduce
that when a future emitter returned.
#1255: fun_abs rejects the exact integer INT64_MIN like iabs (#1114);
fval could not format |INT64_MIN| as a non-negative value and returned
a negative string. Const-fold of abs() matches. Smoke covers both.
Follow-up to the #1156 review, which flagged #1145 as an incomplete fix.
HIR_LUA_SETI is seti(tbl,key,value) and keeps its VALUE operand as an
instruction index in val[], not in src1/src2 -- hir_codegen reads it
back as `s3`. Every operand walker missed it:
hir_dce did not mark it used
hir_copy_prop did not resolve COPYs through it
compute_live_ranges did not extend its live range (both the main
walk and the loop-aware extension)
#1145 made SETI side-effecting so DCE keeps the store. That is correct
on its own, but it turns the omission above into a live hazard: before,
an unused SETI was deleted whole; now it survives while DCE can NOP the
value it stores and linear-scan can hand that register to something
else in the interval the SETI still needs it.
Adds hir_val_operand() in hir.h as the single documented accessor and
routes the four walks through it. Strictly gated on the opcode --
val[] is a plain integer elsewhere, and for HIR_LUA_ALOAD it is a guest
ADDRESS, so a blanket rule would be badly wrong.
Also records, next to is_pure_op, why HIR_LUA_ALOAD must NOT be
restored to pure for performance. #1144 looked like it gave up real CSE
(emit_qreg_read loads QREG_LONGBITS once per %q read, so
strcat(%q0,%q1,%q2,%q3) emits four identical ALOADs of one address),
but GVN never collapsed them even when ALOAD was pure: each carries its
own ICONST 1 and HIR_ICONST is not value-numbered, so the ValueKeys
never match. Measured -- still 4 ALOADs post-opt either way. The
flagged regression does not exist.
NOTE ON TESTING: HIR_LUA_SETI is currently emitted by nothing, so this
is latent-correctness only and cannot be exercised by a test until the
opcode is wired up. Verified no regression: smoke 1389/1389 on x86-64
with --enable-jit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hir_codegen's FP slot bump allocator tested `fp_pool + 8 >
rv_compiler::STR_LIMIT` -- the one-shot compiler's static 0x4000 --
instead of `rc.str_pool_limit`, the limit this particular compile was
constructed with.
The shared heap (used for every nested evaluation, s_jit_depth > 1)
lays its string pool at 0x40000..0x60000. Its str_pool therefore starts
above the static STR_LIMIT, the bound test failed on the very first FP
value, the loop broke, and every FP slot address stayed 0 -- `loc` is
memset to zero. All float temporaries then aliased guest address 0.
Nothing reported an error: FCONST wrote its double to address 0, ATOF
overwrote it, and FMUL loaded both operands from 0, so mul(12.75,N)
returned N*N and fdiv(x,y) returned x/x == 1. Strings were unaffected
because pool_str() already used the per-instance str_pool_limit; only
floats went through the static constant.
Any expression containing runtime (non-folded) float arithmetic that is
evaluated inside another compiled expression was affected -- u() and
ulocal() bodies, and anything wrapping them. A bare call at the prompt
compiles through the one-shot compiler and was always correct, which is
why this survived manual spot-checks.
Genuine pool exhaustion now sets out_exhausted to fail the compile
rather than silently handing back address 0.
testcases/nested_float_fn.mux is new. Reverting the one-line bound and
rebuilding fails all four of its cases (cat=<9> add=10 len=1 first=9,
div35=<1> div2=<1>, iter=9 switch=9 ulocal=<9>) while the other 1330
smoke cases still pass, so it isolates this defect.
Verified on x86-64 with --enable-jit: smoke 1334/1334, make test green
(q-register and ifelse oracles OK), jit_diff 300 expressions 0 LOGIC.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- #1143: ifelse/switch/u() PHI merge keeps TY_FLOAT (or coerces via
FTOA/ITOA); codegen float PHI uses FLD/FSD, not strcpy of bits
- #1144/#1145: ALOAD/GETI/SETI are not pure; SETI is side-effecting
so DCE/GVN/LICM cannot drop or CSE guest memory ops
- #1146: native idiv only for known non-zero const divisor; runtime
zero falls through to fun_idiv for the divide-by-zero error string
- #1147: rollback patch sites on translate overflow; bounds-check
backpatch/resolve against CODE_BUF_SIZE and code_used
- #1148: SysV scopy/memcpy/memset save guest dst in R14 so CALL
keeps 16-byte stack alignment (no push after and rsp,-16)
Restart (#1039–#1043): zero process-local DESC fields after load,
restore peer via getpeername, strip non-survivable TLS/WS flags,
drop TLS and WebSocket before exec, tear down adopt failures with
shutdownsock, abort partial restart loads cleanly.
DB (#1044–#1047): cache_flush_writes returns success only after
Commit; leave dirty queue on failure; atr_head union SQLite with
pending cache/queue; set mudstate.bStandAlone in DbConvert; flush
attr queue before import Commit.
Queue (#1048–#1052): setup_que refunds on OOM; sql_que frees BQUE
on Query fail and inits waittime; do_wait rolls back semaphore if
wait_que fails; include nest depth limit 50; CPU halt_que uses
Owner+object for machines.
JIT (#1053–#1057): full 64-bit rv_load_i64; setq/r single-char
RegisterSet; decline long CARGS; accumulate inline u() watermarks;
bounded guest_strnlen on ECALL paths.
Conf/convert (#1058–#1061): @enable/@disable sync g_dc; live
driver knobs for output/retry/max_players/timeouts/quota; safe
heap encode for t5x/t6h/r7h attributes.
The cand/cor fast-path lowering emits BRC/BR instructions with -1
placeholder targets that are patched once the result blocks exist. A
mid-chain arg that is not integer-lowerable bailed to general_lowering,
abandoning the already-emitted placeholders unpatched. Codegen then
called emit_phi_copies with to_blk=-1, and indexing block_first[-1]
read wild memory -- crashing muxscript (stack-guard SIGBUS) or emitting
garbage, depending on binary layout.
Snapshot n_insns/n_blocks/cur_block/n_pargs (and the per-insn vectors)
before the chain loop and restore them on the bail, so the ECALL path
lowers the whole node from a clean slate. Also add a defensive range
check in emit_phi_copies so any future unpatched placeholder is flagged
and skipped instead of dereferenced.
Corpus triggers (found via temporary instrumentation): the cand()
chains in extract_fn TC008, right_fn TC003, cachestats_fn, and
parser_fn -- any cand/cor with a runtime-integer arg followed by a
runtime-string arg.
Fixes#858
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The RV64 codegen handed raw byte offsets to the B-type/J-type encoders,
which silently drop the bits that do not fit. Three places could emit
corrupt instructions (or corrupt allocator state) if the JIT arena ever
grew past the encodable reach:
- #719: the four Tier-2 JAL call sites (blob dispatch, rv64_strtod fast
path, HIR_FCALL1/HIR_FCALL2 intrinsic stubs) computed
offset = func - pc and passed it straight to rv_JAL. JAL's immediate is
21-bit signed (+/-1 MiB); a larger offset jumped to a wrong address.
- #722: the branch backpatch loop re-encoded B-type (13-bit, +/-4 KiB)
and JAL (21-bit) offsets with no range check.
- #720: the linear-scan and output-buffer allocators wrote
result.reg/addr/spill_slot[iv.value] without verifying
iv.value < HIR_MAX_INSNS. The HIR builder caps n_insns, so this is
defensive against a future pass that synthesizes virtuals out of band.
Add rv_jal_offset_ok()/rv_branch_offset_ok() and route every computed
offset through them; on overflow set rc.out_exhausted so the compiler
driver discards the blob and the AST evaluator handles the expression --
the same bail path already used elsewhere. The allocators gain explicit
HIR_MAX_INSNS bounds guards that bail the same way. All checks pass
trivially for in-range offsets, so normal compilation is unchanged.
Smoke: JIT 1061/0, non-JIT 1050/0.
Closes#719Closes#720Closes#722
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
hir_dump was called in Phase 1 before hir_build_cfg computed
block_first/block_last ranges. Uninitialized values caused
out-of-bounds access into the sval vector, crashing in strlen.
Initialize block_first[0]=0, block_last[0]=-1 in hir_program::init()
so the Phase 1 dump sees an empty range. Add bounds checks in
hir_dump for block_last < n_insns and sval/call_name vector size.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Track code_bytes (total/max), hir_insns (total/max), and spills
(register allocator spill count) across all compilations. Smoke
suite baseline: 49KB RV64 across 1244 programs, largest 988 bytes,
average 10 instructions, zero register spills.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three fixes for SQLite persistent code cache correctness:
1. Constant-folded ICONST results (e.g., add(1,2)→3) were stored with
a frame-relative output address near STACK_TOP, outside the persisted
memory range (first 32KB). On cache restore, the no-JIT fast path
read garbage. Fix: convert ICONST to string at compile time via
pool_str, keeping the result in low memory.
2. out_pool_end was never persisted, defaulting to STACK_TOP-8 on
restore, making the output-clearing loop a no-op. Fix: use
OUT_STACK_LIMIT as conservative lower bound for JIT programs.
3. Codegen could set rc.needs_jit when emitting runtime ITOA for the
final result, but compile_expression only checked h.needs_jit.
Fix: take the union of both flags.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add rv_patch_fargs() which emits code to resolve frame-relative output
references stored in fargs arrays before tier2 JAL calls. Called before
both generic tier2 dispatch and the STRCAT tier2 fast path. Currently
a no-op since alloc_output() still returns absolute addresses.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add rv_load_guest_addr() which resolves frame-relative tagged addresses
via s0 before loading, falling through to rv_load_val for absolute
addresses. Replace all ~30 rv_load_val calls that load output buffer,
string, or fargs addresses with rv_load_guest_addr. Currently a no-op
since alloc_output() still returns absolute addresses.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The prologue now snapshots the incoming SP into s0 (RA_FRAME_TOP)
before decrementing SP for the output frame. This allows future
frame-relative output addressing to resolve buffer locations even
across nested calls.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reassign register roles in preparation for frame-relative output
addressing: s0 is reserved as RA_FRAME_TOP (will capture entry SP
in the prologue), RA_SCRATCH moves to s11, and the allocatable set
shrinks from 11 to 10 registers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Output buffers are now allocated on the guest stack (counting down
from STACK_TOP) instead of in fixed slots scattered around the blob.
This eliminates the gap-skipping allocator and prepares for
re-entrant function calls where each frame needs its own outputs.
Key changes:
- alloc_output() counts down from STACK_TOP-8 instead of up from OUT_BASE
- hir_codegen emits a prologue (LUI+ADDI+SUB) to set up the stack frame
- Frame size = 8 + n_output_slots * OUT_SLOT, ensuring SP lands below
all output buffers so spill slots don't alias output memory
- Removed OUT_GAP_LO/HI/GAP2 gap-skipping logic
- compile_expression default out_start=0 maps to STACK_TOP-8 via
the rv_compiler constructor, fixing the legacy JIT path
- Persistent VM code_heap starts at 0x4 to avoid guest_pc=0 which
collides with the block cache empty-slot sentinel
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lays the groundwork for a persistent RV64 runtime where compiled
expressions coexist in shared guest memory without DBT resets:
- dbt_resume(): enters dispatch loop without zeroing CPU context,
preserving block cache and code buffer across invocations
- rv_compiler::code_base: allows compiling code at non-zero guest
addresses; current_pc() helper for correct JAL offset calculation
- compiled_program::entry_pc: tracks each program's entry point
- hir_codegen: all 4 JAL offset sites (tier2_call, ATOF, FCALL1,
FCALL2) now use rc.current_pc() instead of assuming code starts at 0
- compile_expression: accepts code_base/pool start parameters
- POCVM(): hand-assembled PoC proving cross-function JAL and
dbt_resume work in a persistent guest memory (100+42=142)
- POCVM2(): real compiler PoC — two expressions (first/rest) compiled
at code_base 0x0000 and 0x1000 in shared memory, both correct
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
HIR_ATOF now JALs to the blob's rv64_strtod symbol when available,
which the DBT intercepts and executes as a native x86-64 call to
host strtod(). Falls back to ECALL_ATOF when the blob is not loaded.
Eliminates the ECALL exit/re-enter overhead on the string→double path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When math functions are chained (e.g., sin(add(1,2))), the compiler now
keeps values as native doubles between operations instead of marshalling
through strings at every step. String literals that feed into float
context are parsed at compile time (SCONST → FCONST).
New HIR infrastructure: FCALL1/FCALL2 for direct FP blob intrinsic calls,
known_float[] type tag, is_float()/is_numeric() queries, ECALL_ATOF for
runtime string→double, tier2_sym_addr() for raw blob symbol lookup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase B of array optimization. When FORPREP detects t[i] access in the
loop body (where i is the loop variable), it pre-scans the bytecodes,
emits an __lua_pin_array ECALL to copy the table's integer array into
guest memory at LUA_ARRAY_BASE (0x30000), then replaces GETTABLE with
HIR_LUA_ALOAD — a native 5-instruction RV64 sequence (ADDI, SLLI, load
base, ADD, LD) with zero ECALL overhead per iteration.
Also adds rv_SLLI and rv_SLT instruction helpers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OP_GETTABI and OP_GETTABLE with integer keys now emit HIR_LUA_GETI,
which uses dedicated ECALL_LUA_GETI_INT (0x308). This returns the
table element as a raw int64 in a register instead of converting
through snprintf→guest string→ATOI. Eliminates the ITOA→ECALL→snprintf
→ATOI round-trip that made t[i] access ~10x slower than necessary.
New ECALLs: GETI_INT, SETI_INT (integer fast-path), PIN_ARRAY and
UNPIN (infrastructure for future Phase B native loop access).
3 new smoke tests (TC056-TC058) covering integer table sum, for-loop
table access, and loop with index arithmetic.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
