Commit graph

3 commits

Author SHA1 Message Date
Stephen Dennis
57d520f927 engine/lua: cap read_size() varint length to prevent size_t overflow
read_size() decoded a Lua base-128 size varint (lundump.c LoadUnsigned format:
continuation bytes have the high bit clear, the terminator sets it) with an
unbounded loop. A malformed bytecode stream with a long run of continuation
bytes would keep shifting the accumulator left by 7 per byte, overflowing size_t
and wrapping to a small bogus length that then slips past the caller's bounds
check.

Cap the byte count at sizeof(size_t)*8/7 + 1 (10 on a 64-bit build) — the most
groups a size_t can legitimately need — and fail the read (ok = false) past
that, matching how the upstream loader guards the same loop against overflow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 18:41:38 -06:00
Stephen Dennis
5e05f5472b Fix SIGABRT crash in Lua bytecode deserializer
read_size() returns size_t but was cast to int, causing negative
values when the bytecode reader gets out of sync.  The negative int
was then promoted back to a huge size_t by vector::resize(), throwing
std::length_error which escaped to std::terminate and killed the
server via SIGABRT on every smoke test run during lua() evaluation.

Use size_t throughout and add a 1M sanity cap on all deserialized
array sizes so malformed bytecode fails gracefully instead of crashing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 13:07:19 -06:00
Stephen Dennis
21243e8312 Add Lua bytecode → HIR → RV64 → x86-64 JIT pipeline (Phase 2)
Lua scripts compiled by lua_mod.so can now be JIT-compiled through
the existing HIR/RV64/x86-64 pipeline in engine.so. The bytecode
deserializer reads lua_dump() output without requiring Lua headers.

New COM interface mux_IJITCompile on engine.so with CompileLuaBytecode,
RunCompiled, IsCompiled, and Invalidate methods. lua_mod.so acquires
this interface and transparently attempts JIT before falling back to
the Lua VM.

Opcode coverage: data movement (MOVE/LOADI/LOADK/LOADNIL/etc.),
integer arithmetic (ADD/SUB/MUL/IDIV/MOD/UNM + immediate/constant
variants), comparisons (EQ/LT/LE/EQI/LTI/LEI/GTI/GEI), control flow
(JMP/TEST/TESTSET/FORPREP/FORLOOP), returns, and mux.* bridge calls
pattern-matched from GETTABUP+GETFIELD+CALL to engine API ECALLs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 09:59:45 -06:00