Commit graph

59 commits

Author SHA1 Message Date
Yucong Sun
814c4fa5b9
vm: string foreach/ref fixes, buffers as byte arrays (foreach, strict bytes, to_buffer), thorough ref tests; #1196 docs follow-up (#1250)
* docs/tests: fix constructs index regression from #1196; add ref page to sidebar; extend & ref tests

- Restore extension-less links and the text_blocks entry in
  docs/lpc/constructs/index.md (the PR was recreated from a pre-Docusaurus
  branch and reintroduced .html links, which fail the docs build under
  onBrokenLinks: 'throw', and dropped text_blocks)
- Add lpc/constructs/ref to the hand-authored sidebar in docs/sidebars.ts
- Drop the stale VitePress 'layout: doc' frontmatter from ref.md
- Extend testsuite/single/tests/operators/ref.lpc: & in parameter
  declarations, ref keyword in foreach, and bitwise &/&= non-regression

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

* vm: fix string foreach crash and off-by-one ref reads; support += / -= on string chars

Three related defects around the shared string-codepoint lvalue
(global_lvalue_codepoint), all reproduced on the unfixed binary:

- Nested foreach over strings SEGFAULTED: the iteration cursor lived in
  the shared global, so the inner loop's F_EXIT_FOREACH reset the
  iterator out from under the outer loop (null deref in
  post_index_to_offset). The EGC cursor now lives in each loop's own
  stack slot (the T_NUMBER slot under the loop variable); the ref case
  re-arms the shared codepoint lvalue every iteration, and exit only
  clears the global when it still points at this loop's string slot.

- foreach (int ref c in str) read the WRONG characters: the shared
  index was advanced before the body ran, so every read through the ref
  was off by one ('abc' summed to 197 instead of 294, and the final
  iteration read one past the end). The global index now stays on the
  current character for the whole body. Writes through the ref still go
  to the loop's by-value stack copy and never reach the iterated
  variable -- semantics pinned by tests/operators/foreach.lpc.

- s[i] += n / s[i] -= n threw "Bad Argument 1 to +=()": F_ADD_EQ and
  f_sub_eq handled buffer byte lvalues (T_LVALUE_BYTE) but not string
  codepoint lvalues (T_LVALUE_CODEPOINT), even though ++/--/= worked.
  Both now route through a new codepoint_lvalue_add() helper and produce
  the resulting character as the rvalue.

Regression tests: nested (plain / ref / multi-byte UTF-8) string foreach
and ref-read correctness in tests/operators/foreach.lpc; compound
assignment on string chars (incl. reverse index, rvalue result, and
non-number rhs error) in tests/operators/string_index.lpc. Verified on
the unfixed binary: the nested-foreach test segfaults, the others fail.
Full LPC suite passes 2x on clang ASan/UBSan Debug and 2x on
RelWithDebInfo.

Docs: correct the ref.md note on foreach-over-strings; document the
string-char lvalue rules in AGENTS.md (section 8 + audit checklist 9)
and the ref/& feature in README.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

* vm: foreach ref over strings shares the s[i] char-lvalue logic; thorough ref tests

Arming a string-char lvalue now goes through one shared helper,
aim_lvalue_codepoint(): it validates the target EGC (out-of-bounds and
multi-codepoint error cleanly, same messages as s[i]) and points the
shared codepoint state at it. Both s[i] lvalues (push_indexed_lvalue)
and foreach ref loop variables use it, and every write consumer already
funnels into assign_lvalue_codepoint() -- so ref loop chars follow
exactly the s[i] rules:

- single-codepoint characters (and EGCs up to 4 bytes, which index as
  their first codepoint) keep working: reads deliver the character,
  assignments through the loop variable succeed
- wider EGCs (flag emoji, ZWJ sequences) now raise the catchable
  "Indexed character is multi-codepoint" error when the ref loop
  reaches them, instead of silently reading as -1; the non-ref form
  still iterates and delivers -1 for such clusters

tests/operators/ref.lpc is now a thorough pass-by-reference suite:
ref/& parameter declarations and call arguments, by-value contrast,
ref forwarding through call chains, call-site refs to array elements /
mapping values / string chars (forward and reverse index, write-back),
foreach ref over arrays / mapping values / strings (read correctness,
multi-byte codepoints, by-value write semantics, assignment error
parity, the multi-codepoint error, non-ref contrast), compile-time
rejections via generated sources (ref outside an argument list, ref to
a range), and bitwise &/&= non-regression -- 39 checks.

Full LPC suite passes 2x on RelWithDebInfo and 2x on a clean clang
ASan/UBSan Debug build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

* tests: cover ref across all LPC types

Extend tests/operators/ref.lpc to 58 checks: ref parameters of every
value type (int, float, string, array, mapping, object, function,
buffer, class, mixed) verifying reassignment propagates; the by-value
contrast for reference-typed containers (member writes propagate,
reassignment doesn't); call-site refs to buffer bytes (forward and
reverse index) and class members (both . and -> spellings); foreach
ref over mapping values of mixed types; the compile-time rejection of
a ref mapping KEY in foreach; and the clean runtime error for foreach
over a non-iterable type (buffer).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

* buffers behave like byte arrays: foreach, strict 0..255 bytes, string/array promotion, to_buffer()

Buffers are now first-class byte containers:

- foreach iterates a buffer like an array, delivering each byte as an
  unsigned int 0..255. A ref loop variable mutates the buffer in place;
  each ref carries its OWN T_LVALUE_BYTE (in ref->sv), so nested buffer
  ref loops and b[i] lvalues in the body can't alias each other.
  T_LVALUE_BYTE consumers now read the lvalue's own pointer/subtype
  instead of reaching for the shared global_lvalue_byte (which remains
  only as the scratch instance b[i] arms).

- every LPC byte write path (=, ++, --, +=, -=) range-checks the result:
  a value outside 0..255 raises "Buffer byte value out of range" and
  leaves the byte unchanged, instead of silently truncating/wrapping.
  += / -= on bytes also yield the resulting value as their rvalue.

- strings and arrays of ints 0..255 PROMOTE to buffers: a new
  to_buffer() efun (registered like to_int/to_float) is wrapped around
  the rhs by do_promotions() / rule_expr_assign for 'buffer b = str',
  'b += str', initializers, and 'b + str'; range assignment and the
  runtime + / += paths convert unpromoted (mixed) values through the
  same svalue_to_buffer_bytes() helper. A string contributes its raw
  UTF-8 bytes; an array must hold only ints 0..255 (validated before
  allocation) or the conversion errors with the target unchanged.

- fixed a pre-existing ref_t leak: a foreach ref loop variable reused by
  a re-entered inner loop leaked one ref per outer iteration (flagged by
  the debug memory checker as 'Found temporary block: make_ref').

tests/operators/buffer_bytes.lpc pins the byte-range semantics, + / +=
concatenation, all promotion forms, and the error paths (106 checks);
foreach.lpc and ref.lpc pin buffer iteration and ref-loop independence;
buffer_range_assign.lpc gains range-read pins. New docs for to_buffer
(sidebar regenerated) and a rewritten lpc/types/buffer page.

Full LPC suite passes 2x on RelWithDebInfo and 2x on a clean clang
ASan/UBSan Debug build (no ref-checker warnings); GTest suite 312/312.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-12 01:39:30 -04:00
gesslar
1ba6584da5
feat: allow & as syntactic sugar for ref (#1196)
Adds '&' as an alternative to the 'ref' keyword for pass-by-reference in
parameter declarations, call arguments, and foreach loops. One-line grammar
change (the 'ref' rule now accepts both L_REF and '&'); no ambiguity with
binary bitwise AND since ref is always in prefix position. Includes
regenerated parser, testsuite coverage, and new docs for the ref construct.
2026-07-11 22:45:47 -04:00
Yucong Sun
d38dc2c833
lexer: strip comments from directive payloads before parsing (#1240) (#1241)
A '//' comment after a #define body was captured INTO the stored macro
body. Expansion buffers carry no newline to end it, so when the macro
expanded inside a spliced line (a function-like macro's substituted
body, as in the report's MIN(credits, m[e][CREDITS])), the '//' ate the
rest of the splice and the parse failed with a baffling 'unexpected ;'
attributed to the outer macro. The same missed strip made
'#undef X // why' erase nothing and '#ifdef X // why' look up the
wrong name and silently take the false branch.

Comments are whitespace (C translation phase 3). dispatch_directive now
strips them from the payload before parsing for #define (name, params,
body -- body also right-trimmed so a stripped comment can't turn '1'
vs '1 ' into a spurious redefinition warning), #undef, #ifdef, #ifndef,
and #pragma (word-list payload); #if/#elif already stripped.
strip_directive_comments() now folds a block comment to ONE space
instead of nothing, so '1 -/*c*/-1' keeps its token boundaries instead
of pasting '--'. #error/#warn/#echo payloads stay raw.

Covered by five new Preprocessor unit tests (the report's repro shape,
paste-prevention via #if branch selection, param-list comment,
testsuite/single/tests/compiler/preprocessor.lpc; documented in
docs/lpc/preprocessor/define.md.

Fixes #1240


Claude-Session: https://claude.ai/code/session_016FMBJLkpkpVpZdz6PWzeWe

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-11 12:15:19 -04:00
Yucong Sun
c364856f0d
lexer: block comments on directive lines may span physical lines (#1236) (#1239)
* tests: clear the active scanner before yylex_destroy in the harnesses

Both tokenizer harnesses destroyed their scanner without
lpc_lex_scanner_destroyed(), leaving the global active_scanner dangling;
the next compile's first current_line read then dereferenced the
destroyed scanner's guts (lpc_lex_current_line_ref ->
innermost_real_buffer_index). Order-dependent and layout-dependent: any
Preprocessor test followed by CompileEntry.FatalInsideIfExpressionRecovers
segfaulted deterministically in a minimal pair and intermittently in full
runs. Pre-existing (reproduces on master); surfaced while adding the
#1236 tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016FMBJLkpkpVpZdz6PWzeWe

* lexer: a directive-line block comment may span physical lines (#1236)

The single anchored directive rule captures one physical line (plus
backslash continuations), so a /* comment opened after a #define body
and closed on a later line ended the capture at the newline:
strip_directive_comments() silently swallowed the open comment, the
define itself parsed, and the comment's remaining lines were tokenized
as code -- a regression against the old lexer for a pattern real
mudlibs use.

New lpc_lex_complete_directive() runs before the terminating-newline
consumption: it scans the captured text (quote-aware, same rules as
strip_directive_comments) and, when the line ends inside an open block
comment, pulls raw bytes through lpc_lex_getc() until the comment
closes and the logical line really ends. Comments fold to a single
space, so text after the close still belongs to the directive (C
semantics), and the tail may open further comments, strings, '//', or
backslash continuations. Newlines pulled this way get the same
bookkeeping as the rule's own terminator, and the count is backed out
of lpc_lex_on_directive()'s first-line attribution so diagnostics still
point at the directive. EOF inside the comment reports the same error
as SC_BLOCK_COMMENT's <<EOF>> rule instead of spinning.

Covered by new Preprocessor unit tests (repro shape, comment tail,
__LINE__ bookkeeping, live/dead #if, string-literal '/*', EOF) and
testsuite/single/tests/compiler/preprocessor.lpc pins; documented in
docs/lpc/preprocessor/define.md.

Fixes #1236

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016FMBJLkpkpVpZdz6PWzeWe

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-11 02:07:07 -04:00
Yucong Sun
55956a24b7
Add inherit_program / include_file master applies; auto hot-reload demo (#1230)
* Add inherit_program / include_file master applies; auto hot-reload demo

New compile-time master applies, consulted for every inherit statement
and #include directive:

* mixed inherit_program(string from, string path, int priv)
  Called while compiling `from` for `inherit "path";` (priv nonzero for
  private inherits). A string return is an alternate path for the
  inherited file; an array-of-strings return is the inherited program's
  source itself (compiled via load_object_from_source under the inherit
  statement's name, through the existing load_object retry loop); any
  other return prevents the inheritance.

* mixed include_file(string compiled, string from, string path)
  Called when `from` is about to include `path` while compiling
  `compiled`. A string return is the translated path (resolved absolute
  from the mudlib root or relative to the includer; returning `path`
  unchanged keeps the "..."-vs-<...> search semantics); an
  array-of-strings return is the included text itself (pushed as an
  in-memory include buffer with the usual file-identity bookkeeping);
  any other return prevents the inclusion.

Both follow the valid_override/get_include_path precedent for calling
master LPC mid-compile (skipped without a VM context or master object),
and a missing apply keeps stock behavior.

The applies expose the full compile-time dependency graph, which the
testsuite uses to demonstrate mudlib auto hot-reload on file changes:
/single/hot_reload.lpc registers as the master's compile hooks, records
which source files each program's bytecode was built from (own source,
includes, inherited programs, transitively), and its call_out poller
destructs+reloads watched blueprints whose dependency closure changed
on disk - including reloading a stale parent when only the parent's
include changed.

Testsuite: single/tests/compiler/{inherit_program,include_file}.lpc pin
the apply semantics (redirect, inline source, deny, priv flag, argument
shapes on nested includes) via the scriptable /clone/compile_hook;
single/tests/applies/hot_reload.lpc demonstrates end-to-end hot reload
over a runtime-written inherit+include fixture chain. Docs added for
both applies.

Validated: full LPC suite (532 files) x2 on RelWithDebInfo, x2 on
Debug+ASan/UBSan, plus the 297 GTest unit tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* docs: add hot reload guide for the new compile-time master applies

New concepts page (concepts/general/hot_reload.md) explaining why
"file changed -> reload" needs the compile-time dependency graph, how
the inherit_program / include_file master applies expose it, and a
step-by-step mudlib implementation with examples: master delegation,
dependency recording, closure computation, change detection with
size+mtime snapshots, parent-first reload ordering, and the call_out
poller. Documents blueprint-reload semantics and caveats (clones keep
the old program, no compiles inside the applies, records only complete
for compiles observed by the daemon), pointing at the testsuite
reference implementation. Cross-linked from both apply reference pages
and the concepts indices.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* docs/testsuite: say "master copy", not "blueprint"

Align the hot-reload guide, apply reference, daemon, and test comments
with the project's terminology for the object loaded from a file (see
clonep(3): "the master copy").

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* testsuite: edge cases for the compile-time applies; docs review fixes

compile_hooks_edge.lpc pins that unusual inherit_program/include_file
return values produce clean LPC errors or the documented behavior,
never a driver crash: empty array, non-string elements, redirect to
self, empty-string redirect, inline source vs already-loaded object,
multiline inline content, the apply itself throwing mid-compile
(safe_apply falls back to default resolution), extension-spelled
redirects, and denying the auto-included global include file.
Fixtures in /clone/adv_*; /clone/adv_hook wraps the scriptable hook
with a throwing include_file.

Docs fixes from review: the apply-page examples now look the daemon up
with find_object() instead of a path call_other (which would load the
target and trigger a compile mid-compile -- the exact pattern the same
pages forbid), and the hot-reload guide's ancestors() snippet carries
the seen-mapping argument to match the reference daemon.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* hot_reload: fix multi-watch and failed-reload handling; harden tests

Three defects from the LPC review round, each now pinned by the
demonstration test:

* check_now() collected and reloaded in one pass, so the first reload's
  snapshot refresh erased the change evidence for every other watched
  program sharing the dependency (a common header, or a watched parent
  iterating before its watched child - which then stayed bound to the
  destructed old parent forever). The stale set is now collected before
  any reload runs.

* A reload whose recompile throws (a syntax error mid-edit - the most
  common event in a hot-reload workflow) unwound poll() before the
  re-arm, killing the poller forever and leaving the watched master
  copy destructed. poll() re-arms first, check_now() catches per
  program, reload_count only counts successes, and closure_changed()
  treats a watched-but-not-loaded program as stale so the retry
  self-heals once the file compiles again.

* The three apply tests registered master compile hooks (and the
  hot-reload test armed the poller and wrote /data/hot) with cleanup on
  the success path only; a thrown check would leave the hook routing
  every remaining compile of the randomized run. Each test now runs its
  checks under catch and unhooks/tears down unconditionally, re-raising
  the error afterward.

The demo test now also covers the shared-dependency pass (watch parent
and child, change the common include, both reload in one pass) and the
broken-edit recovery cycle. The hot-reload guide's snippets are updated
to match the daemon.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

* simulate: let inline inherit source resolve its own unloaded inherits

From the C++ review round: master::inherit_program returning inline
source whose text itself inherits a not-yet-loaded program failed the
whole load with "#inherit is not supported when compiling from
in-memory source" -- reachable in the feature's primary use case, since
synthesized programs routinely inherit ordinary on-disk library files.

load_object_from_source() now runs the same iterative dance as
load_object(): when the compile aborts on an unloaded parent, load that
parent (from disk, or from further master-supplied inline source, so
synthesized-inheriting-synthesized chains work), then recompile the
same source string -- which is in hand, unlike the historical
no-filename rationale for rejecting #inherit here. Mirrors
load_object()'s guards: illegal-to-inherit-self, the duplicate-name
check after the parent's arbitrary LPC ran, and an inherit-chain bound
on the retry loop as a backstop against a master that redirects to a
fresh unloaded name on every recompile.

The inherit_program test now covers both new shapes (inline inheriting
unloaded on-disk, and two levels of inline source); the apply doc drops
the already-loaded-only caveat.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DGhzJfPhDGJA94EPh1gEK

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-10 10:27:27 -04:00
Yucong Sun
887f9ebbd1
docs: document UTF-8 native strings in the LPC language reference (#1226)
Add a 'UTF-8 Native Strings' section to lpc/types/strings.md covering
what the driver actually implements: lengths and positions are measured
in extended grapheme clusters (UAX #29), indexing yields code points and
errors on multi-code-point clusters, ranges/explode/strsrch operate on
character boundaries, display width (strwidth, UAX #11) vs length,
\uXXXX and surrogate-pair escapes, UTF-8 validity requirements, and the
encoding boundary (set_encoding for connections, string_encode /
string_decode / buffer_transcode elsewhere). Note in the old
sub-ranging section that positions are characters, not bytes.

Clarify sizeof() (string = grapheme clusters, buffer = bytes) and
strsrch() (character offsets, character-boundary matches).

Every documented example is pinned by a new testsuite file,
testsuite/single/tests/compiler/utf8_doc_examples.lpc, verified against
a freshly built driver (19 checks). Notably replace_string() is
byte-oriented, so it is deliberately NOT listed among the
grapheme-aware operations.


Claude-Session: https://claude.ai/code/session_01TSzcESzU9947zkGzQ6SMmE

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-10 07:42:20 -04:00
Yucong Sun
c914f03d66
Add local search and documentation guide for docs site (#1221)
* docs: add local full-text search and a contributor README

Add @easyops-cn/docusaurus-search-local to the Docusaurus site so the
docs get an offline search bar (index built at build time, no external
service). English and zh-CN pages are both indexed, and matched terms
are highlighted on the target page.

Add docs/README.md describing the Docusaurus setup, local dev/build
commands, search behavior, directory layout, and gotchas; exclude it
from the published site alongside CLAUDE.md. Point the root README's
docs/ entry at it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TSzcESzU9947zkGzQ6SMmE

* docs: remove dead framework leftovers, fix index generation, complete the nav

Delete the VitePress (.vitepress/) and Jekyll (_layouts/, css/) leftovers,
the one-shot migration scripts (fix_md_header.py, fix_seealso.py), and the
stale keywords.json snapshot; prune the matching .gitignore entries and
docusaurus exclude patterns.

Rewrite gen_index.py for Docusaurus: it emitted dead .html links and
legacy 'layout: doc' frontmatter, choked on non-markdown entries, and
dropped nested categories — regenerating an index would have broken it.
It now emits the extension-less links the site actually uses, links
nested category indexes (restoring apply/* on the zh-CN index), and
refuses to run on the docs root. Fix update_index.sh's copy-paste titles
(zh-CN efun/build were titled 'APPLY'), stop it clobbering the
hand-written lpc/index.md, and cover cli/. Regenerated indexes pick up
the missing driver/ffi-plan entry. add_missing_efuns.py now takes the
keywords.json path as an argument instead of requiring a stale copy.

Move CNAME and the Google site-verification file into static/ so they
actually reach the published build output.

Complete the sidebar: link the CLI category to cli/index and add the
missing portbind/symbol/generate_keywords pages, and expose the
previously orphaned stdlib section under Reference.

Promote onBrokenLinks to 'throw' now the build is warning-free, and drop
the empty Demo section from the landing page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TSzcESzU9947zkGzQ6SMmE

* docs: strip legacy 'layout: doc' frontmatter from all pages

Mechanical sweep removing the Jekyll-era 'layout: doc' line from every
doc page's frontmatter (Docusaurus ignores it), and the matching line
from the templates in docs/CLAUDE.md so new pages don't reintroduce it.
No content changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TSzcESzU9947zkGzQ6SMmE

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-09 22:09:55 -04:00
Yucong Sun
fd705f0634 docs: reconcile FFI plan + preprocessor/diagnostics docs with implementation
Doc-only review pass checking every branch-authored doc against the
shipped code. Fixes where docs described intent rather than the result:

- docs/driver/ffi-plan.md (began life as a plan, drifted from the
  shipped package): ffi_status() returns a mapping, not mixed*;
  callbacks are implemented and in-scope (ffi_callback/_addr/_free added
  to the efun surface and moved out of "v2 deferred"); the DEBUGMALLOC
  section now describes the actual std::unordered_map + TAG_BUFFER scheme
  (no TAG_FFI / mark hook exists); valid_ffi gates load/symbol/prepare/
  callback; testing is the 20-file LPC suite + tools/ffi/test.py (there
  is no GTest fixture).
- docs/lpc/preprocessor/index.md: document the #warn directive.
- docs/lpc/diagnostics.md + preprocessor/pragma.md: show_error_context is
  a legacy flag that no longer changes clang-style compiler diagnostics
  (render_diagnostic ignores PRAGMA_ERROR_CONTEXT; only the runtime
  smart_log path still reads it).
- tools/lpc-syntax/README.md: test.mjs is 49 assertions, not 47.

Verified accurate, no change needed: AGENTS.md, testsuite/README.md,
compiler/internal/README.md, and the source-files / float / strings /
text_blocks / define / conditionals / inherit / include docs, sprintf %g,
and the tools/ffi + vscode READMEs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 20:48:48 -04:00
Yucong Sun
55e47edc07 Float exponent notation: 2.5e2, 1e3, 2E-5 are float literals now
Neither master's hand-written scanner nor the flex lexer ever accepted
exponents ("2.5e2" lexed as REAL(2.5) IDENT(e2)) -- but the hand-written
EBNF already documented the form and it should exist. Two lexer rules
add it: an optional exponent on the fraction form, and a bare-exponent
form ("1e3"); "1e" without exponent digits stays NUMBER(1) IDENT(e),
"1..5" stays a range, hex "0x1E3" is untouched, and '_' separators work
in all parts. strtod() already converted exponents, so only the
patterns changed.

Pinned at every layer: syntax_literals.lpc (2.5e2, 2.5e-2, 1.5E+2,
1e3, 1_2e4), the JS tokenizer + generated tmLanguage (with node tests
for the exponent forms and the "1e" non-exponent), the lexical EBNF
(realLiteral | exponent rules, replacing a half-aspirational entry),
and docs/lpc/types/float.md (which also wrongly claimed single
precision -- LPC_FLOAT is a double).

Verified: testsuite x3 + ctest 297 (ASan Debug), RelWithDebInfo full
ctest, lpc-syntax node tests (50).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 20:48:48 -04:00
Yucong Sun
7fe7c5c9a9 docs/lpc: source-file resolution, diagnostics, full preprocessor reference
New pages: lpc/source-files (extension rules, extension-blind object
identity, registry-before-filesystem, portable-code guidance),
lpc/diagnostics (clang-style output, macro expansion notes, include
chains, fix-its, show_error_context), preprocessor/conditionals
(token-based #if with C precedence, defined()/efun_defined()) and
preprocessor/pragma (real pragma table from the driver).

Rewrote preprocessor/index (full directive table, immutable
predefines), define (function-like macros, stringize/paste, rescan,
redefinition-warning semantics) and include (search order, master
get_include_path, trailing text, macro file names); constructs/include
is now a summary pointing at the reference, and constructs/inherit
documents pathname resolution. Dropped the vestigial
preprocessor/README; sidebar gains the new pages plus the previously
unlisted text_blocks.

AGENTS.md now points agents at docs/lpc/ as the authoritative LPC
reference. Validated with a clean docusaurus production build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 20:48:48 -04:00
Yucong Sun
48fe0e9b5a Grammar-driven lexical simplification; minimal token inventory
Lexical decisions move from lexer state into the grammar, where LALR
lookahead already disambiguates:
- Array/mapping opens ({ / ([ are ordinary '(' '{' / '(' '[' token
  pairs the grammar pairs (composite tokens deleted).
- The whole '(: name' first-class-function machinery (dedicated start
  condition, function_flag, one-byte peek, old_func()) becomes two
  grammar productions; %expect documents the intentional conflicts.
- Token diet: dead tokens deleted; single-char operators are plain char
  tokens ('!', '.'); same-precedence families share one value-carrying
  token (L_EQ_NE, L_SHIFT, L_INC_DEC -- the L_ORDER idiom). Release-
  build illegal-char diagnostics made unconditional; CRLF multi-line
  #define fold pinned (Windows).

Includes the merge of current master (docs-only advance).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 20:48:48 -04:00
gesslar
9e5a1f16dd
docs: modernize concepts/general and fix correctness (#1211)
* empty

* docs: modernize concepts/general and fix correctness

Overhaul the docs/concepts/general set: remove MudOS/LPmud/TMI-era
references, correct APIs and config options against the current source,
trim speculative/AI-generated content, and fix the driver overview.

- Rename MudOSdriver.md -> fluffos_driver.md and modernize its content
- lpc: FluffOS branding, note class/struct types, code formatting
- message_doc: correct message()/receive_message signatures and the
  `type` semantics; distinguish the message() path from the catch_tell
  output efuns; replace the speculative "smart client protocol" essay
  with pointers to GMCP/MSDP/MXP/MSP/ZMP
- preprocessor: correct the #pragma list and defaults, document
  #error/#warn; move @/@@ text blocks to lpc/constructs/text_blocks.md
- simul_efun: correct the call mechanism, config option name, and
  function-visibility rules
- socket_efuns: rewrite the 1992 tutorial into a concise, accurate guide
  and document MUD-mode save/restore serialization
- tls: fix fabricated client APIs (socket_connect arity/address, resolve),
  the TLS-version claims and cert-path resolution; de-emoji
- tracing: fix the default/cap, output format, build-gate and overhead
  claims
- websocket: fix config option names (websocket http dir), set_debug_level,
  and the receive_message input/output confusion; trim filler
- global_include_file, oop: minor correctness/typo fixes

Rebased onto the Docusaurus docs: internal links use .md, index files use
extensionless links, and code with braces is fenced. Not built locally;
please verify rendering. See PR notes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 11:12:31 -07:00
Yucong Sun
2063e95436
Fix Docusaurus sidebar, broken links, and gh-pages CI (#1209)
* Reorder sidebar: Driver > CLI > Reference (LPC Language, Apply, EFUN, Concepts)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix Docusaurus build: broken links, duplicate routes, and gh-pages CI

- Strip .html from all markdown link targets (51 files) for Docusaurus URL routing
- Add slug: frontmatter to 4 files whose names match their parent directory
  (interactive.md, objects.md, README.md, build.md) to prevent Docusaurus's
  category-index convention from creating duplicate routes
- Fix one missed .html link in zh-CN/build/index.md
- Move onBrokenMarkdownLinks to markdown.hooks (Docusaurus v4 deprecation)
- Update gh-pages.yml: rename to Docusaurus, use node 22, correct build path
  (docs/build instead of docs/.vitepress/dist)

Build now completes with [SUCCESS] and zero warnings or broken links.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 21:01:21 -07:00
gesslar
819a883c7f
Add nullish coalescing (??) and logical assignment operators (||=, &&=, ??=) (#1149)
* 11-12-adding_nullish_coalescing_operator: 2025-11-12 22:47 - nullish stuff

* 11-12-adding_nullish_coalescing_operator: 2025-11-12 23:28 - adding logical assignment operators

* adding autogen files because grammar has changed

* addressing Codex feedback

* Fix __TREE__ debug output for NODE_NULLISH and NODE_LOGICAL_ASSIGN

The lpc_tree_name array was missing entries for NODE_NULLISH and
NODE_LOGICAL_ASSIGN node types that were added when implementing
the nullish coalescing operator (??). This caused __TREE__ to return
incorrect type names in the debug output.

The fix adds the missing entries "nullish" and "logical assign" to
the lpc_tree_name array at the correct indices to match the parse node
enum definition, allowing the constant_expr.c test to pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* adding autogens

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-05 19:46:11 -08:00
gesslar
3977897e2e
Add simplified syntax for function pointers and invocation (#1148)
* first-class-functions

* first-class-functions

* adding grammar.autogen.cc as requested. did not generate a .h after merging in last update

---------

Co-authored-by: Yucong Sun <1256464+thefallentree@users.noreply.github.com>
2025-11-12 23:53:35 -08:00
gesslar
5bb1538c7c
Add C99-style variable declarations anywhere in function blocks (#1147)
* adding C99-style variable declaration capabilities

* adding autogens as requested
2025-11-12 22:54:19 -08:00
Yucong Sun
978d467992
fix a default argument bug and update docs (#1144) 2025-11-01 02:35:11 -07:00
Yucong Sun
ddf8b9740d
more doc fixes. (#1137)
* docs: Fix multiple documentation issues

This commit addresses several documentation inconsistencies and gaps
identified by comparing docs with actual code implementations:

**Build Documentation Fixes:**
- Updated macOS build instructions to clarify Homebrew paths for both
  Apple Silicon and Intel Macs, emphasizing modern Homebrew defaults
- Added clarity to SQLite version options (=1 vs =2) explaining the
  differences and recommending version 2

**CLI Tool Documentation:**
- Added documentation for the `symbol` utility (docs/cli/symbol.md)
  - Tool for loading and analyzing LPC files
  - Usage: symbol <config> lpc_file
- Added documentation for the `portbind` utility (docs/cli/portbind.md)
  - Privilege-separated port binding for FluffOS
  - Allows binding to privileged ports then dropping privileges

**Apply Documentation:**
- Added mxp_enable apply (docs/apply/interactive/mxp_enable.md)
  - Called when MXP protocol is negotiated with client
- Added mxp_tag apply (docs/apply/interactive/mxp_tag.md)
  - Processes MXP tags from client
- Added zmp_command apply (docs/apply/interactive/zmp.md)
  - Handles ZMP (Zenith Mud Protocol) commands
- Added receive_ed apply (docs/apply/interactive/receive_ed.md)
  - Post-processes ed editor output
- Removed obsolete view_errors.md documentation
  - Apply not found in source code, appears to be deprecated

All new documentation follows existing format conventions and includes
cross-references to related efuns and applies where applicable.

* docs: Add missing apply and efun documentation

This commit adds documentation for previously undocumented applies and
efuns, completing the documentation coverage for recent FluffOS features.

**Apply Documentation:**
- terminal_colour_replace (interactive): Custom color token replacement
  callback for terminal_colour() efun preprocessing
- parser_error_message (master): Custom error message generation for
  parse_sentence() failures

**Efun Documentation:**
- hash() (crypto package): Complete documentation for cryptographic hash
  function with support for modern algorithms (SHA-3, BLAKE2, SM3)
  - Includes security recommendations and version compatibility notes
  - Documents all supported algorithms from legacy (MD5, SHA-1) to modern
    (SHA-3, BLAKE2b512)
  - Created new crypto package documentation directory

**Index Updates:**
- Updated docs/apply/interactive/index.md with new applies:
  mxp_enable, mxp_tag, receive_ed, terminal_colour_replace, zmp
- Updated docs/apply/master/index.md with parser_error_message
- Removed obsolete view_errors reference from master index
- Added crypto section to docs/efun/index.md

These additions address documentation gaps identified by comparing the
source code with existing documentation, particularly for the crypto
package enhancements from commit 33de35c (modern hash algorithms).

* docs: Add CLI tool, config guide, and documentation maintenance guide

This commit adds comprehensive documentation for additional FluffOS
features and creates a maintenance guide for the documentation itself.

**New CLI Documentation:**
- generate_keywords: Development tool for generating keywords.json for IDE
  integration and language server support. Extracts all efun metadata into
  structured JSON format.

**New Configuration Documentation:**
- config.md (driver/): Complete guide to driver configuration file format
  - Documents all configuration options with examples
  - Network setup (telnet, websocket, TLS)
  - Memory management and performance tuning
  - Protocol support (GMCP, MXP, ZMP, MSSP, MSP)
  - Security settings and limits
  - Includes practical examples for development and production

**Documentation Maintenance Guide:**
- CLAUDE.md (docs/): Comprehensive guide for maintaining FluffOS documentation
  - Documentation structure and organization
  - Templates for applies, efuns, and CLI tools
  - Workflow for finding and documenting undocumented features
  - Source code mapping (where to find implementations)
  - Verification and testing procedures
  - Common documentation issues and fixes
  - Package-specific notes and guidelines
  - Quick reference commands for contributors

**Index Updates:**
- Updated docs/driver/index.md to include config.md

These additions make it easier for contributors and AI assistants to
maintain accurate, complete documentation for FluffOS.

* docs: Add socket TLS options and LPC default arguments documentation

This commit documents important FluffOS features that were previously
undocumented, focusing on TLS socket options and LPC language enhancements.

**Socket TLS Options Documentation:**
- socket_set_option() efun - Configure socket TLS/SSL parameters
  - SO_TLS_VERIFY_PEER: Control peer certificate verification
  - SO_TLS_SNI_HOSTNAME: Set Server Name Indication hostname
  - Includes security notes and practical examples
  - Essential for HTTPS and secure socket connections

- socket_get_option() efun - Query socket option values
  - Retrieve TLS verification and SNI settings
  - Useful for validation and debugging
  - Examples for conditional logic and auditing

**LPC Language Feature:**
- Default Arguments documentation (prototypes.md)
  - Comprehensive guide to FluffOS default argument syntax
  - Feature added in commit bcb8e91 (2023) but not documented
  - Rules, examples, and use cases
  - Multiple practical examples showing API design patterns
  - Notes on compile-time behavior and limitations

**Index Updates:**
- Updated docs/efun/index.md with new socket functions
- Updated docs/efun/sockets/index.md with new functions

These features significantly enhance LPC programming capabilities:
1. TLS options enable secure network connections with certificate verification
2. Default arguments reduce code duplication and improve API usability

Both features are production-ready and widely used but were missing
from the documentation.

References:
- Socket options: commit 1fd7f61 (2023)
- Default arguments: commit bcb8e91 (2023)

* docs: Add comprehensive guides for tracing, TLS, and WebSocket support

Added three new concept documentation guides:

1. tracing.md - Complete guide to performance profiling with trace_start/trace_end
   - Chrome DevTools integration
   - Memory management warnings
   - Profiling scenarios (commands, combat, heartbeats)
   - Analysis techniques and best practices

2. tls.md - Comprehensive TLS/SSL support documentation
   - Server port TLS configuration
   - Certificate generation (self-signed and Let's Encrypt)
   - Client TLS socket connections with SO_TLS_VERIFY_PEER and SO_TLS_SNI_HOSTNAME
   - Security best practices
   - Troubleshooting guide

3. websocket.md - Complete WebSocket support guide
   - WebSocket server configuration (ws:// and wss://)
   - Multiple protocols (ASCII, Telnet, Binary, HTTP)
   - Built-in web client documentation and customization
   - GMCP over WebSocket
   - Telnet protocol over WebSocket
   - Security considerations and performance optimization
   - Troubleshooting and advanced usage

Updated concepts/general/index.md to include all three new guides.

* docs: Regenerate documentation indices

Ran update_index.sh to regenerate all documentation indices:

- docs/apply/index.md: Added new applies (mxp_enable, mxp_tag, receive_ed,
  terminal_colour_replace, zmp, parser_error_message), removed obsolete view_errors

- docs/concepts/index.md: Added new concept guides (tls, tracing, websocket)

- docs/driver/index.md: Added config documentation, updated title format

- docs/efun/crypto/index.md: Regenerated to standard format with hash function

- docs/cli/index.md: Generated index for CLI tools (driver, generate_keywords,
  json2o, lpcc, o2json, portbind, symbol)

All indices now correctly reference the new documentation added in previous commits.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-31 14:25:38 -07:00
Michael Programs
b5162afc1e
Update classes.md with casting example (#1067) 2024-06-06 03:06:02 -07:00
gesslar
6415efa14f Update function type documentation 2024-01-11 10:19:17 -05:00
Michael Programs
4472977238
add documentation for lpc type "class" (#1019)
* add documentation for lpc type "class"

* add line return
2023-12-06 08:59:51 -08:00
Yucong Sun
6d8698a09d setup vitepress 2023-12-02 20:00:33 -08:00
Michael Programs
f7802e218d run docs update_index.sh 2023-08-21 23:38:33 -07:00
Yucong Sun
3bca73af50 EFUN: sys_reload_tls(int port_index) 2023-05-29 21:15:09 -07:00
Yucong Sun
3478cc2ed6 EFUN: time_ns() 2023-05-29 11:09:09 -07:00
噢哎哟喂
c0b196c0d6
更新说明文档 (#897)
* 更新说明文档

* update README.md

* update efun docs

* update efun docs, add parsing efun

* 修复network_stats()中文文档错误
2022-07-07 18:39:06 -07:00
Yucong Sun
3093d818e5 Update doc index 2022-05-07 14:04:12 -07:00
噢哎哟喂
44532e0228
add DB, update docs (#807)
* add DB

* update docs

* update DB

* add database code

* update docs
2021-05-09 06:35:59 -07:00
噢哎哟喂
2d2f3c94ee
update zh-CN docs (#801) 2021-04-23 02:35:04 -04:00
jalbright015
cb1faea2f5
Updated documentation for clone_object() and new() efuns (#794)
* Updated documentation for clone_object() and new() efuns

* Updated documentation index

* Updating documentation for new() and clone_object() efuns
2021-04-17 21:57:47 -04:00
jalbright015
b13f0ebd9d
Added perf_counter_ns() documentation (#792)
* Added documentation for defer() efun

* Updated index to reflect defer() documentation

* Added documentation for new efun perf_counter_ns()

* Updated index to reflect perf_counter_ns() documentation

* Added documentation for new efun perf_counter_ns()

* Updated index to reflect perf_counter_ns() documentation
2021-04-16 13:27:16 -04:00
jalbright015
69351d52e6
Added documentation for defer() efun (#791)
* Added documentation for defer() efun

* Updated index to reflect defer() documentation
2021-04-16 11:36:23 -04:00
Yucong Sun
c20ca9a15e Update doc index 2021-04-11 01:45:36 +08:00
Yucong Sun
5eafddf8bb
stdlib: base64encode/base64decode from Gesslar (#770)
* new sefuns base64encode, base64decode

* update docs

* stdlib: base64encode/base64decode

Co-authored-by: Brian Workman <bworkman@frogdice.com>
2021-03-25 05:43:16 +08:00
Yucong Sun
868d2db57b EFUN: strptime() and strftime() 2021-02-23 17:58:02 -08:00
Yucong Sun
f395a8884d fix build errors 2020-12-13 18:10:32 -08:00
Yucong Sun
42b286f067 Some cleanup on the docs 2020-12-10 18:35:31 -08:00
Yucong Sun
f9531c05cc cleanup doc index generation, upgrade to python3 2020-12-10 18:35:31 -08:00
oiuv
e836be493c update docs 2020-12-06 09:49:10 -08:00
Yucong Sun
0650d9e73b Adding EFUN pcre_match_all 2020-11-10 13:36:29 -08:00
Yucong Sun
53e789890f Update doc 2020-11-10 13:36:29 -08:00
oiuv
5023af8875 update docs 2020-10-31 23:38:58 +08:00
Yucong Sun
1c622e5ba4 Upgrade backward-cpp 2020-10-11 18:46:33 -07:00
Yucong Sun
ae71e49c60 apply: virtual_start() for virtual objects 2020-08-25 20:46:10 -07:00
Yucong Sun
553e97ee97 Adding efun: telnet_nop() 2020-08-20 00:11:40 -07:00
Yucong Sun
6833890533 Implement secure_random() efun 2020-07-23 00:18:24 -07:00
oiuv
8b7baa7fb9 update docs 2020-05-09 09:35:19 -07:00
Yucong Sun
34f226a938 Adding an VMTracer that could output to chrome devtools 2020-04-13 23:25:27 -07:00
噢哎哟喂
294cf88049
update docs,fix lpcc (#615)
* update rouge.css

* update build.md

* fix lpcc

* update docs

* update index
2020-03-19 08:56:48 -07:00
oiuv
5d71ec806b update docs 2020-03-18 07:48:45 -07:00