Three consecutive defects escaped this fuzzer for the same reason: the
corpus was string and list functions over literal operands.
#1143 float PHI arms collapsed to a string PHI, copying raw double
bits as a C string
#1157 a bare %N used as an ifelse() condition folded to 0
#1159 FP slot addresses pinned at 0 in nested compiles, so every
float temporary aliased guest address 0
None of the three is reachable with literal operands: a folded float
allocates no FP slot and a folded condition emits no branch. So runtime
values come from v(fz.*) attributes, defined in a per-batch preamble.
v() is an ECALL the compiler cannot see through, whereas compile-time %q
tracking can fold %q back to a constant -- the trap that made
ifelse(%q0,...) look healthy while ifelse(%0,...) was broken.
New default-on shapes, roughly 15% float / 15% branch / 14% nested:
- float arithmetic (fdiv, mul, sqrt, power, fmod, round, ...) with at
least one runtime operand guaranteed
- ifelse()/switch() over a runtime condition with type-MIXED arms,
which is the PHI-typing surface
- u()/ulocal() of a generated attribute body, always consumed by an
outer call -- the wrapper is the point, since a bare u() compiles
through the one-shot compiler and was always correct while a u()
inside a compiled expression routes through the shared heap
Also fixes two minimizer defects that made the new shapes unusable:
- run_chunk and show_value never replayed the corpus setup, so
v(fz.*) was empty and u(me/fz.uN) undefined in every minimized
run. Every nested finding reduced to "no divergence" and was
misfiled as STATE-DEPENDENT. gen.py now writes setup.txt and the
minimizer replays it.
- show_value ran BOTH sides in the toggle-on process, so its INTERP
column was really the JIT value and it printed two identical
numbers under a heading claiming they differ. It now splits across
exp.conf/int.conf exactly as run_chunk does.
Before: 5 bogus reproducers with equal values, 23 STATE-DEPENDENT.
After: 28 distinct minimal reproducers, 0 STATE-DEPENDENT.
Verified against 681b1bc66 (all three defects present): 28 LOGIC, with
minimal reproducers landing on each class -- u(me/fz.uN) for #1159,
ifelse(ab cd,...) for #1157, switch() with float arms returning empty
for #1143. Against current master: 0 LOGIC at SEED=1 and SEED=7, and
--brackets/--utf8/--longreg all still clean.
SEED=42 surfaces #1171, a real pre-existing mul()/power() precision
divergence this corpus found on a clean master. sqrt/power are left in
deliberately rather than trimmed to keep runs green; it is recorded
under Known divergences in the README.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard-lift campaign's final change (docs/plan-jit-evalbracket-lift
.md): eval brackets are JIT-compiled by default. Per the #1001 review
checklist, the same commit removes the netmux.conf soak opt-in (that
file ships) and retires jiteval() — with the default on, the
production route reaches everything the gate-bypass existed for.
Harnesses moved off default-off assumptions:
- jit_diff's I-side conf sets jit_eval_brackets 0 explicitly (relying
on the default would compare JIT against JIT).
- The q-register oracle now compares PRODUCTION evaluation across two
workspaces (default conf = JIT vs explicit 0 = AST) with u()
carriers and a jitstats canary; all nine shapes green.
The flip surfaced three latent items — Makesmoke bakes the smoke
database by EXECUTING setup commands under the conf default, a third
evaluation context never before run toggle-on:
1. #$ (switch token) was unimplemented in the lowerer and fell
through as literal text: @switch actions like [idiv(#$,2)]
computed idiv("#$",2) = 0. The lowerer now bails compilation on
#$, preserving AST semantics.
2. fdepth()/fcount() read func_nest_lev/func_invk_ctr, which
compiled code does not maintain (native lowering flattens the
nest): they read 0. The lowerer bails on both. The wider
function_recursion_limit design question (cost guard vs semantic
contract) is filed as #1002 — not a blocker, as the divergence
direction is fail-open into more capability.
3. The Smoke classifier greped case-sensitively for 'Failed';
nested_depth.mux's lowercase "failed" message slipped through and
the suite reported ALL PASSED around a real failure. Now -ci.
Final matrix: smoke 1318/1318 with the new default AND with the
toggle explicitly off; the smoke.flat re-bake is byte-identical to
the pre-flip bake; oracle 9/9 on the production route; sweeps
standard and brackets+utf8+longreg(SEED=7) both 400/0 LOGIC.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- soak.sh halts on any COLOR-encoding divergence too (post-#995 the
baseline is 0 everywhere, so a new one is a regression signal that
previously scrolled by as a summary count), and appends a
timestamped header instead of truncating prior soak history.
- Plan doc records the completed two-box soak evidence and the flip
checklist — including that the default-flip commit removes the
netmux.conf soak opt-in (that file ships; the opt-in must not
outlive the soak).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- testcases/tools/jit_diff/soak.sh: soak driver — rounds of 400-expr
differential sweeps with rotating seeds across every corpus mode
combination (brackets always on; utf8/longreg rotate), plus a full
toggle-on smoke run every 10th round. Stops with the work dir
preserved on any LOGIC divergence. Runs identically on the Linux
box.
- mux/game/netmux.conf: dev instances now run with jit_eval_brackets
on, so all interactive/manual testing exercises the bracket surface.
The SHIPPED default (conf.cpp init) remains off until the soak
completes and the default flips.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the reviewed #996 step-2 design: registers whose
authoritative value exceeds the 256-byte SUBST slot are tracked in a
guest-memory bitmap, and %q reads branch to the authoritative fun_r
ECALL instead of reading a truncated slot. Long-register programs now
stay on the JIT (step 1's entry declines removed) and the previously
unfixable mid-program case — a long value created inside the program
by SETQ_SYNC — reads correctly.
Per the review requirements:
- QREG_LONGBITS: dedicated u64 at the SUBST-end/DMA gap; the entry
marshal whole-word writes it every run (computed bits for masked
registers, zero elsewhere) so the reused runtime buffer can never
leak a previous program's stale bits.
- Writers: entry marshal, ECALL_SETQ/ECALL_SETQ_PACK (bit update from
the vlen they already measure), and the post-ECALL resync. Scope
restores fall out of the resync — proven by the new four-transition
oracle shapes (long->short->restore and the SETQ_SYNC-created-long
inverse: '1300' / '3001').
- Readers: single choke point emit_qreg_read() at the only
SUBST_QREG0 sref site (grep-audited); the diamond composes from
existing HIR ops (HIR_LUA_ALOAD as plain guest u64 load, SHR, BAND,
BRC/PHI) — no new opcodes or codegen.
- fun_r exempted from the conservative post-ECALL resync (pointer
compare), so the long path doesn't re-marshal per read.
- Honest cost: load+test+branch per %q read; rvbench six-read expr
0.19us/call vs 0.16 pre-diamond, still ~3x faster than native AST.
bail_longreg remains in jitstats and now reads 0 permanently (the
acceptance signal that declines were superseded). Corpus --longreg
gains mid-program setq-then-read shapes; TC018 locks mid-program and
both scope transitions bracket-free.
Tests: smoke 1318/1318 both toggles; oracle 9/9; sweeps standard,
longreg x2 seeds, longreg+utf8+brackets all 400/0 LOGIC; entry (303),
mid-program (303), and short (203) shapes verified with
eval_handled=1 bail_longreg=0.
Closes#996.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
%q SUBST slots are 256 bytes; register values longer than 255 bytes
silently truncated on every slot write, so a JIT %q read returned 255
bytes where the interpreter returns the full value (found by review,
#996; production-reachable: setq(0,repeat(x,300)) then a %q0 read).
Step 1, the entry-side half: marshal_qregs_to_slots now reports when a
masked register exceeds the slot, and both entry-marshal sites decline
the run to the AST evaluator — nothing has executed yet, so declining
is side-effect-free (the #875 "decline cleanly" pattern). New
bail_longreg jitstats counter. The mid-program resync cannot decline
and still truncates; that half is #996 step 2, designed for review in
docs/plan-jit-evalbracket-lift.md (guest-memory long-register bitmap
maintained by marshal/SETQ/resync, %q reads branching to a fun_r ECALL
when the bit is set; OUT_SLOT=LBUF so long values flow intact
everywhere else).
Corpus: JITDIFF_LONGREG=1 mode — an interpreter preamble sets %q9 to
lengths straddling 256 and the measured expression reads it (mid-
program setq-then-read shapes deliberately excluded until step 2).
TC017 locks the decline via a fresh u() inner evaluation: 300 bytes
declines (result 303, was 258), 200 bytes stays on the JIT.
Tests: repro returns 303 with eval_bailout=1 bail_longreg=1; smoke
1317/1317 both toggles; oracle 7/7; sweeps standard, longreg x2 seeds,
and longreg+utf8+brackets all 400/0 LOGIC.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Systematic audit of all 79 rv64/co_*_wrap tier2 wrappers against their
interpreter authorities (three parallel code readers; every claimed
divergence verified or refuted empirically on both routes before
touching anything). Five live divergences, all default-route:
- co_lpos_wrap: mirror fun_lpos — empty pattern defaults to a space
(was: empty result) and the FULL pattern matches via color-aware
co_search (was: first byte only, so lpos(abxab,bx) reported every
'b'). The constant-fold path used single-byte co_lpos too; it now
declines patterns longer than one byte.
- co_splice_wrap: search-word validation counts words like countwords
(space delim trims/collapses), so a blank or padded search word is
no longer "#-1 TOO MANY WORDS".
- setunion/setdiff/setinter: the blob comparator implements only
a/i/n/d — AutoDetect ('?' or present-but-empty) and f/u/c sort
types now gate to the interpreter at compile time (was: '?' sorted
ASCII where the interpreter autodetects numeric).
- rv64_isnum: full ParseFloat mirror — exponents ([eE][+-]1..4
digits), surrounding whitespace, and Inf/Ind/Nan are numbers.
- rv64_isint: is_integer mirror — surrounding whitespace accepted.
Refuted (unreachable, verified empirically incl. runtime-computed
separators): the multi-char sep and osep-edge divergences the wrappers
would have are all blocked by the hir_lower delim gating. Dead rows
noted in the plan: co_vislen_wrap, rv64_dec2hex/hex2dec (unrouted),
rv64_null, rv64_round (deliberately ECALL). ~65 wrappers MIRROR OK.
Adds JITDIFF_UTF8=1 corpus mode (multi-byte words incl. emoji+modifier
cluster in every generator shape) and bracket-free TC016 locking the
five fixes on the default JIT route. softlib.rv64 rebuilt.
Tests: smoke 1316/1316 both toggles; oracle 7/7; sweeps standard,
utf8, and utf8+brackets(SEED=7) all 400/0 LOGIC.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hunting the #993 "state-dependent" divergence found it was
deterministic all along: minimize.py's isolation replay was never
updated for the Phase 4 J/I process split — it replayed both sides in
one toggle-on workspace, comparing JIT against JIT, and misclassified
the finding as STATE-DEPENDENT. The minimizer now replays the I side
in a separate process under int.conf (toggle off), like run.sh.
The underlying bug: rv64_after/rv64_before did a raw byte search with
no trim_space_sep, so the default-pattern haystack kept leading and
trailing spaces the interpreter trims — after(cat(a,),) returned "b "
vs the AST's "b" (the #993 LOGIC, surfacing through
stripansi(after(cat(...),extract(...))) in the bracket sweep).
Both wrappers are now exact fun_after/fun_before mirrors:
- default single-space pattern trims the haystack via a pointer+length
trim_space_sep equivalent (guest fargs are const);
- the search is color-aware co_search, and after() advances past the
needle with co_visible_advance over visible code points.
The color-aware search also fixes#980 (a colored needle never matched
the raw-byte comparison: after(ansi(r,ij x cd ab x),ansi(r,cd))
returned empty) and #993's COLOR-encoding case (raw shas now equal).
Locked by bracket-free jit_parity_fn.mux TC015 (trim + color + explicit
-pattern regression) on the default JIT route. softlib.rv64 rebuilt.
Tests: smoke 1315/1315 both toggles; oracle 7/7; jit_diff standard +
brackets x 3 seeds (default/7/13) all 0 LOGIC, 0 COLOR — the two
formerly-failing seeds fully clean.
Fixes#993. Fixes#980.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The guard-lift itself (docs/plan-jit-evalbracket-lift.md, Phase 4):
jit_can_handle() now admits terminated [...] eval brackets when the new
jit_eval_brackets config directive is on (cf_bool, CA_GOD, default
OFF — production behavior unchanged). Unterminated brackets always
bail; EV_NOFCHECK text is gated at the call site (literal passthrough
that the lowerer doesn't model).
Harness upgrades required to make the toggle-on green check
meaningful:
- jit_diff J/I sides split into separate processes: the I side always
runs with the toggle off, keeping the eval-bracket bail (the true
production interpreter route, production flags) as the oracle. An
asteval({...})-forced in-process I side was tried first and
manufactured ~113 false LOGIC divergences — fun_asteval trims a
trailing space after an empty-yielding bracket that production
preserves (filed as #987).
- JITDIFF_BRACKETS=1 mode: bracket-wrapped corpus (embedded/adjacent/
pure shapes), toggle in the J-side conf only, and a canary that
fails fast if the toggle didn't take.
- SMOKE_EXTRA_CONF passthrough in the smoke runner for whole-suite
toggle-on runs.
Results: with the toggle ON, letq_fn TC002 (the historical INNERINNER
failure) and the localize scoping cases route through the JIT and
pass; the bracket sweep is 400/0 LOGIC. The toggle-on smoke run also
witnessed two PRE-EXISTING production JIT bugs (bracket-independent,
reproduced bracket-free on today's default route): #988 maxArgsParsed
comma-catenation dropped (sha1(abc,def) -> sha1("abc")) and #989
tier2 wordpos() UTF-8 position miscount. Both block the Phase 5
default-on flip, not this landing.
Toggle OFF (default): oracle 7/7, smoke 1310/1310, standard sweep
400/0 LOGIC — byte-identical, cleanly gated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the isolation gap from the #784 review:
- The original expression is confirmed to diverge IN ISOLATION (fresh
single-expression muxscript process) before shrinking. If it only
diverges alongside its fuzz batch -- the order/state-dependent #778
class -- it is reported as STATE-DEPENDENT with its original form,
instead of shrink() returning the input and show_value() printing a
"minimal reproducer" whose JIT and interp values are equal.
- Candidate reductions are still screened in shared batches for speed,
but a reduction is only ACCEPTED after it diverges standalone; batch
artifacts fall through to the next-shortest candidate. diverges()
is no longer dead code.
- run.sh preserves the work dir on a failing run (trap - EXIT) so a
STATE-DEPENDENT finding's batch (b*.txt) can actually be replayed --
previously the trap deleted the only reproducer.
Robustness items from the same review:
- Batches are chunked at 50 candidates per fresh muxscript process
(variants() can emit hundreds, overflowing the command queue), and
"Run away" in any run now warns to stderr instead of silently
reading dropped results as "does not diverge".
- Process timeout moved from a parsed "timeout 90 <path>" prefix
string (which broke on paths with spaces) to subprocess timeout=
with a JITDIFF_MUX_BIN / JITDIFF_TIMEOUT interface; the legacy
JITDIFF_MUX prefix is still accepted. TimeoutExpired warns and
degrades instead of crashing.
- Result-line indices are digit-validated, so stray J~/I~ text in
evaluated output can no longer crash the minimizer (ValueError).
Verified end-to-end against a real divergence by installing the
pre-#789 blob (known remove() trailing-empty-word bug):
squish(remove(strcat(cd|x|gh,|),x,|)) shrank to the remove() call
with correct differing values; a non-diverging input was classified
STATE-DEPENDENT; a full run.sh fuzz round found, minimized, and
preserved the work dir with exit 1. Restored blob: 2x200
expressions, 0 LOGIC, 0 COLOR.
Closes#784.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- delim_parity_fn.mux TC007: encode64()-based raw-byte assertions --
no-op ldelete/replace/insert return their colored input
byte-identically, setunion tie-breaking (1x1 keeps LIST1's copy,
general merge keeps LIST2's), and mid() excludes color outside the
copied cluster range.
- jit_diff gen.py: delimiter lists can now be single-element, the
shape that reaches handle_sets' identical-1x1 special case.
- README: COLOR reports are no longer expected noise -- a clean run is
0 LOGIC and 0 COLOR; any COLOR report is a real finding.
smoke.flat regenerated (1109 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- delim_parity_fn.mux TC006: eleven fail-closed assertions covering
the empty-element model -- words(a||b,|)=3, member of middle and
trailing empties, match(,)=1, graball/revwords/elements/remove with
trailing empty words.
- jit_diff gen.py: delimiter-list elements are now empty ~30% of the
time, the coverage that surfaced the rv64_remove trailing-empty
drop. smoke.flat regenerated (1108 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two false-pass paths in the differential fuzzer:
- --enable-jit is off by default and the whole JIT path is compiled
out without it, so on a default build both sides ran the AST
interpreter and run.sh exited 0 claiming "no logic divergence".
Probe via jitstats() (only registered under TINYMUX_JIT) before
generating anything, and exit 2 with reconfigure instructions on a
non-JIT build. README now states the --enable-jit requirement.
- If an entire batch crashed or timed out, its ids never reached
results.txt and vanished from the awk seen[] array -- no MISSING
lines, exit 0, with only a lower "compared" count on stderr as a
hint. Iterate ids 0..COUNT-1 instead of the ids that produced
output, so dropped batches surface as MISSING.
Also mark #772 fixed in the README's known-divergences section (the
bTrunc flip landed in 4fc70fa54) and document the minimizer's
shared-process limitation (#784).
Verified: run.sh 30 -> probe passes on this JIT build, 30 compared,
0 LOGIC, 0 missing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- minimize.py: shrink divergent expressions to minimal reproducers via
delta-debugging (typed leaf replacement + subtree hoisting), tested by
running candidates through muxscript; prints deduped minimal forms with
their actual JIT and interpreter values. run.sh auto-minimizes the LOGIC
divergences instead of dumping verbose raw expressions.
- gen.py: ~25% of roots now exercise custom single-char delimiters and
output separators (incl. multi-char osep), built on flat leaf lists so the
list/delimiter stay consistent — the class where ldelete/extract bugs lived.
Immediately paid off: the delim coverage found that setunion/setdiff/setinter
drop a multi-char osep to a single char under the JIT (e.g. setdiff(...,|,::)
-> "cd:kl" vs interp "cd::kl") — same class as #768, but the set functions
were missing from the hir_lower multi-char-separator guard. Fix follows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A developer tool that finds correctness divergences between the softcode JIT
(tier2/rv64 wrappers + HIR lowering) and the interpreter (ast_eval), by
generating random typed, nested softcode and evaluating each expression both
ways (JIT via an @if condition, interpreter via an eval-bracket), comparing
by SHA1.
It catches compositional bugs the per-function audit misses — it found #772
(ljust/rjust/center don't truncate when width < content), which only surfaces
through nested width args.
Soundness: the generator never embeds a function inside literal text, because a
bracket-less mid-string call (foo add(2,3) bar) is correctly literal when bare
but evaluated inside [...], which would be a false positive (the cause of the
closed#773). Colored leaves are a single ansi() call wrapping the list.
Results are split into LOGIC (stripansi'd hashes differ — real) and COLOR
(raw differ, stripped match — internal color-encoding only, non-fatal).
Validated: a 120-expr run reports only ljust/rjust/center (the open #772) as
LOGIC, with color-encoding cases correctly tagged COLOR and no mid-string
false positives.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
