* Char-mode input: deliver real keystrokes (BS/DEL, whole UTF-8, raw ESC); fix NAWS lost at logon
Five input-path fixes that make raw-keystroke (get_char) applications
viable, found by building the LPC TUI library on top of them:
- comm.cc: char mode delivered "" for Backspace/Delete (the byte was
zeroed before delivery, making BS/DEL/NUL indistinguishable). The
literal byte is now delivered; line-mode in-buffer editing unchanged.
- comm.cc: char mode delivered one *byte* per callback, splitting a
multi-byte UTF-8 character into 2-4 invalid one-byte strings.
Extraction is now UTF-8 aware: a complete sequence arrives as one
callback carrying one valid character; malformed bytes still go
byte-at-a-time (no stalls).
- comm.cc: the "no ansi" + "strip before process input" ESC->space
rewrite (both default on) also applied to char mode, so arrow keys
arrived as literal "[A". The rewrite is an anti-ANSI-injection
protection for line-mode commands; char mode now always passes ESC
through.
- net/telnet.cc: each received chunk was u8_sanitize()d independently,
so a UTF-8 character split across TCP segments became U+FFFD in any
input mode. An incomplete trailing sequence (new u8_incomplete_tail(),
GTest-covered) is now carried over in interactive_t and prepended to
the next chunk.
- net/telnet.cc + comm.cc: fast clients answer the initial DO NAWS
while ip->ob is still the master object, so the window_size apply
fired on the wrong object and the size was lost until the next
resize. The last report is cached and replayed on the user object at
logon.
docs/efun/interactive/get_char.md documents the delivery contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RpXv4yGCWpkzbkifyZy9EE
* Add /std/tui: an LPC TUI library (readline + ncurses for the prompt line and full-screen apps)
A terminal-UI toolkit in pure LPC, strictly layered (design + rationale
in testsuite/std/tui/DESIGN.md; user docs in docs/concepts/general/tui.md):
- ansi.lpc: escape builders plus the width toolkit the driver lacks
(visible_width/wslice/wpad are ANSI-blind and wide-char aware).
- keys.lpc: keystroke decoder state machine: the get_char byte stream ->
key events (CSI/SS3 with xterm modifiers, Alt prefixes, bracketed
paste as one event, SGR mouse, UTF-8, lone-ESC via caller-driven
flush()).
- readline.lpc: the line editor: emacs keymap (motion/kill/yank/
transpose), Up/Down history, C-r/C-s incremental search, Tab
completion, masked mode, horizontal scrolling with wide-char aware
viewport; repaints only its own line so it works at any scroll
position.
- screen.lpc: virtual cell grid + minimal-diff frame renderer with
copy-on-write rows (frame cost tracks touched rows, not W x H);
wide chars own two cells, boxes/fills/attrs as SGR param strings.
- widget.lpc + w/ (label, list, textfield) + app.lpc: widget protocol,
focus cycling, event routing; textfield embeds a readline engine.
- terminal.lpc: the one impure module, inherited by the user object:
get_char re-arm loop (I_NOECHO across re-arms), NAWS/TTYPE caching
via the window_size/terminal_type applies, lone-ESC walltime timeout,
guaranteed teardown. tui_readline() is a drop-in input_to()
replacement; tui_open() runs full-screen apps on the alt screen.
Everything below terminal.lpc is a pure state machine: 194 checks in
single/tests/std/tui/ cover the decoder, editor sessions, history
search, screen diffing and widgets headlessly. `tuidemo` /
`tuidemo app` (command/tuidemo.lpc) demo both modes over any telnet
client; the whole stack was verified end-to-end over a live connection
including split-packet UTF-8, modifier keys, paste, resize and
alternate-screen teardown.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RpXv4yGCWpkzbkifyZy9EE
* /std/tui v2: pterm/blessed-inspired widgets, inline printers and prompts, showcases
Reviewed the pterm (Go) and blessed widget catalogs and ported what earns
its keep in a MUD; DESIGN.md is renamed to README.md and documents the
full set (including what was deliberately not ported).
New layers:
- print.lpc — pterm-style printers that compose with plain write():
p_table (boxed, width-aware, header rule), p_tree, p_bars, p_spark,
p_panel, p_bullets, p_header, p_progress, p_info/success/warn/error,
and p_bigtext via the existing /std/bitmap_font.
- menu.lpc + terminal glue — inline interactive prompts in the normal
output flow: tui_select(), tui_multiselect() (Space toggles, list
windows by height, collapses to a "? prompt: answer" line) and
tui_confirm() (y/n with a default).
New widgets (/std/tui/w/): table (columns + header + selection), tree
(collapsible, arrows fold), checklist, radiolist, button, progress,
spinner (app-driven tick()), and log (bottom-anchored scrollback pane).
Showcases (command/tuidemo.lpc): `tuidemo select` (prompt chain),
`tuidemo print` (all printers), `tuidemo dashboard` (animated spinner,
progress bars, sparkline, live table and log on a call_out tick), and
`tuidemo form` (textfield, radio group, checkboxes, buttons).
Coverage: three new test files (print exact-output, menu sessions,
widgets2) bring /std/tui to 8 files in the suite; the live-connection
e2e run now drives all six showcases end-to-end (52 checks), including
the select->multiselect->confirm chain, unattended dashboard animation,
and full form entry.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RpXv4yGCWpkzbkifyZy9EE
* /std/tui: charts — braille canvas, line charts, vertical bars, heatmap
Fills the chart gap left by v2 (only the horizontal p_bars existed):
- canvas.lpc: a braille dot canvas (each cell is a 2x4 dot grid from the
U+2800 block — the blessed-contrib/drawille technique), giving sub-cell
resolution: c_set/c_unset/c_get, Bresenham c_line, and c_plot (scale a
value series across the canvas), with per-cell colour. Renders as a
string, per-row strings, or per-cell ({ ch, attr }) pairs for blitting
into a screen.
- print.lpc: p_chart (multi-series braille line chart with y-axis gutter
and coloured legend), p_vbars (vertical bar chart with eighth-block
partial tops and optional value row), p_heatmap (2D matrix as
256-colour cells, cool-to-hot ramp, optional axis labels).
- w/chart.lpc: the live line-chart widget — add_series()/add_point()
rolling history sized to the widget, auto or fixed y-range; replaces
the dashboard's sparkline label with a real animating graph.
- `tuidemo charts` showcase; chart docs in README.md and the docs page
(heatmap removed from the not-ported list).
Tests: single/tests/std/tui/charts.lpc pins the braille bit math with
exact glyphs, line drawing, plot endpoints, exact p_vbars output,
chart/heatmap structure, and the widget's rolling window. The live e2e
run verifies `tuidemo charts` output and braille frames streaming from
the dashboard chart.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RpXv4yGCWpkzbkifyZy9EE
* docs: move the TUI page from Concepts to STDLIB
/std/tui is a mudlib library like base64/break_string/json, so its page
belongs under docs/stdlib/ with the rest of the /std modules, not under
Concepts. Registered in stdlib/index.md; README file-layout pointer
updated. (Concepts' sidebar is autogenerated, so the old entry disappears
with the file.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RpXv4yGCWpkzbkifyZy9EE
---------
Co-authored-by: Claude <noreply@anthropic.com>
Fully implements the docs/driver/ffi-plan.md design. LPC can now load
native shared libraries, call C functions whose signatures are described
at runtime, manage native memory, pass in/out parameters, and expose LPC
function pointers to C as callbacks.
Package (src/packages/ffi, option PACKAGE_FFI ON, libffi via pkg-config):
- ffi_load/unload/symbol; ffi_prepare/ffi_call (ffi_prep_cif + ffi_call);
ffi_alloc/free/sizeof/peek/address; ffi_read/write; ffi_struct_layout;
ffi_callback/ffi_callback_addr/ffi_callback_free (libffi closures that
re-enter the VM via safe_call_function_pointer); ffi_error/ffi_status.
- Buffers are the currency for all pointer/byte data; raw pointer VALUES
(returned pointers, buffer/callback addresses) are ints. LPC strings
are UTF-8-native and never implicitly marshalled -- a char* is a
buffer the caller encoded (pinned by ffi_string.lpc).
- Native allocations are LPC buffers (GC-tracked); handle tables freed at
shutdown (ffi_cleanup) and marked for DEBUGMALLOC (mark_ffi).
Security: master apply valid_ffi(op, arg, caller) gates every
load/symbol/prepare/callback (VALID_FFI added to the applies table); a
missing apply denies by default. Optional "ffi allowed libraries" config
allow-list (rc.cc + runtime_config.h + regenerated config.md, new
Security category). __PACKAGE_FFI__ predefine added.
tools/ffi/generate.py: turns a C header into LPC bindings (buffer params
for char*, optional --string-convenience UTF-8 overloads) plus a struct
layout include; reports+skips unsupported forms; --emit-json contract.
Dependency-free test.py.
Tests: 20 testsuite/single/tests/efuns/ffi_*.lpc (every efun, the qsort
callback round trip, the generated-bindings end-to-end path), guarded by
__PACKAGE_FFI__ with a libc-reachability probe. The efuns are VM-stack-
based, so the LPC testsuite is the surface -- libffi's call/closure paths
run there under ASan/UBSan and the per-file check_memory leak gate.
The clang RelWithDebInfo sanitizer caught an error()-unwind leak: both
ffi_prepare and ffi_callback allocated before a code_to_type() that can
error() -- now unique_ptr/custom-deleter owned (AGENTS.md section 4).
Verified: testsuite x3 (ASan Debug) + ctest 297, RelWithDebInfo suite x3
+ ctest 298, clang RelWithDebInfo sanitizer (leak-clean), tools/ffi
test.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Systematic review of all 326 spec-declared efuns against
testsuite/single/tests/efuns/; every efun now has a test file named
after it. Pure functions get exact behavioral pins (trig/log/vector
math with domain-error cases, trim family, pcre group semantics,
min/max index form, compress/encoding round trips, bit strings,
Levenshtein string_difference, class assembly/introspection, deep
copy() independence, dump_trace shape, save-string round trips);
environment-dependent efuns get honest contracts (sockets: real
create/bind/listen/connect lifecycle on ephemeral ports; interactive/
protocol/ed efuns: graceful no-interactive behavior; package-gated
efuns guarded by __PACKAGE_*__ / efun_defined()). testsuite/include
gains the driver-shipped socket.h/socket_err.h. Fixtures:
catch_tell_probe (catch_tell recorder + self-mover + make_living),
event_probe, shadow_probe, syntax_parent.
The new coverage flushed out SEVEN real driver bugs, all fixed:
- memory_summary: four division-by-zero sites in memory_share() when a
value's refcount is 0 (UBSan)
- send_zmp/start_request_term_type: command_giver dereferenced before
the null check -- crash with no interactive (UBSan)
- socket_create: LPC int loaded into enum socket_mode before
validation -- UB for out-of-range modes; validated as int first
- async_db_exec: manual callback ref taken before handle validation --
error() unwind leaked the function pointer (AGENTS.md section 4)
- link(): epilogue abandoned both string arguments -- two shared-string
refs leaked per call
- assemble_class(): built through copy_array then retagged T_CLASS,
skewing num_arrays/num_classes and the DEBUGMALLOC tag; now built
with allocate_class_by_size
- parser_mark_verbs(): marked only the HEAD of each verb's rule list
(later rules unaccounted), double-marked base verbs through synonym
entries (verb_syn_t::real overlays the node slot), and computed
header offsets off NULL for rule-less verbs (UBSan)
- pending resolve() queries had no DEBUGMALLOC accounting at all: new
pending-query registry + mark_dns_requests(), mirroring
mark_call_outs
Runner: uncaught errors are now PRINTED as well as recorded (a failing
file was otherwise silent about why).
Verified: testsuite x3 randomized (ASan Debug) + ctest 297, clang
RelWithDebInfo sanitizer full suite, RelWithDebInfo full ctest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The runner (command/tests.lpc) now speaks the gtest protocol: a
[ RUN ]/[ OK ]/[ FAILED ] block per test file with per-file timing
(perf_counter_ns), then a recap with total checks/files/elapsed.
Failed checks are RECORDED (tests.h OUTPUT -> master::record_failure;
clear_last_error doubles as the check counter) and the run CONTINUES --
one run reports every failure, then the recap lists each failed file
and the driver exits nonzero; "Checks succeeded." + exit 0 is the pass
signal. Interactive runs report without shutting the driver down.
-ftest also accepts globs now (-ftest:efuns/dual*), and the fail-dir
compile-log copy is skipped when a filtered run compiled nothing.
The suite is a first-class ctest test: add_test(lpc-testsuite) with
PASS/FAIL_REGULAR_EXPRESSION and LABELS lpc; CI's two test steps are
now ctest -LE lpc (GTest) and ctest -L lpc (this suite).
New coverage:
- compiler/preprocessor.lpc: end-to-end pins for ## paste, # stringize,
nested-comma arguments, backslash continuation, self-reference
termination, redefinition-takes-effect, #warn survival, #undef,
defined()/token-precedence #if, __FILE__/__DIR__/__LINE__
- compiler/fail/inherit_exact_ext.lpc (+ dedicated never-loaded
fixture): an explicit ".c" inherit of an .lpc-only file must fail --
fixture is private to the test because the extension-blind registry
would otherwise satisfy the inherit under randomized order
- dual_extension.lpc: children() is extension-blind across spellings;
a ".c"-spelled inherit of an already-loaded .lpc program compiles
(registry identity, the positive twin of the fail pin)
- save_object.lpc: ".lpc"/".c" are stripped before ".o" is appended
Verified: full suite x3 (ASan Debug, randomized order) + ctest 297/297,
and RelWithDebInfo ctest -L lpc / -LE lpc all green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
coverage along the failure axes
RENAME + NORMALIZE:
- lex.l -> lexer.l, lex.h -> lexer.h, lex.autogen.cc -> lexer.autogen.cc
(matching lexer_rules/lexer_utils); the flex POST_BUILD copy runs the
same NORMALIZE path-sanitizing step as bison's outputs, with a
symmetric DENORMALIZE no-flex fallback.
LEXTOKENSTREAM REMOVED (the scanner IS the interface):
- It had become a forwarding wrapper; compile_file and stage_output
drive the reentrant scanner directly, with the scanner guard ordered
BEFORE the cleanup guard.
FIVE REAL BUGS -- two caught in production on a live mud:
1. Unwind-order UAF: the wrapper was declared after the cleanup DEFER,
so exception unwinds destroyed the scanner before teardown deleted
the flex buffers. Fixed by guard ordering.
2. Stale active-scanner UAF (pinned by FatalAbortThenRecompile): an
aborted compile skips end_new_file; the NEXT compile's current_line
save read destroyed scanner guts. Owners now call
lpc_lex_scanner_destroyed() before yylex_destroy.
3. Release-only spin: lpc_dump_stage_tokens skipped the shared-string
table reset; a second dump walked stale hash chains into a freed
A_STRINGS block. The stage env now mirrors the harness.
4. FIELD BUG -- #include with trailing comment: the include parser
assumed the closing delimiter was the line's LAST character, so
`#include <mudlib.h> /* ... */` swallowed the comment into the
filename (classic-mudlib ftpdconf.h failed). Scan to the closing
delimiter; trailing text is ignored (historical behavior); missing
close is a clean error.
5. FIELD BUG -- teardown-vs-mem_block ordering UAF: an abort with a
LIVE include buffer (inherit or fatal INSIDE an included file) had
clean_parser free the mem_block areas BEFORE buffer teardown, whose
include-accounting pops write into them (ASan WRITE on a live mud).
Teardown now runs first in clean_parser; the success epilog is
ordered symmetrically.
Also: compile_file_fd's explicit fd flag (a -1 sentinel silently
compiled the empty view), parse loop skipped on load failure, unified
stage_output cleanup.
COVERAGE (all portable: tmpfile(), no /tmp/mkstemp/open_memstream):
16 new unit tests -- fd entry success/bad-fd/pipe-growth, fatal aborts
from the main buffer, inside an include, and inside a #if expression
(suppress_expansion unstranded), unterminated text block/template,
include depth limit, empty and no-final-newline includes, trailing
block/line-comment includes with clean-name diagnostics, stage-output
pp/tokens/load-failure. Two LPC testsuite pins run under ASan on every
platform: include_trailing_comment.c (ftpdconf.h shape) and
inherit_via_include.c (the abort-and-reload with the inherit inside an
included file).
Post-CI portability/leak follow-ups folded in: the pipe-growth test is
fork-free (fits the pipe buffer; _pipe on Windows -- sys/wait.h broke
MinGW), and the two test environments share one init guard (mixing the
full boot with the tokenizer harness in one process double-called
init_strings, orphaning a 64KB string table -- LeakSanitizer-caught in
the CI sanitizer job).
297 tests under ASan/UBSan (leak detection on), driver-autotest x3.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
