fluffos/testsuite/command/stress_string.lpc
Yucong Sun fb2202546c
lpc-syntax: source-following LPC formatter with printWidth wrapping and full testsuite reformat (#1270)
Rebuilds the LPC formatter (tools/lpc-syntax/format.mjs) on top of the
existing grammar-driven tokenizer as a dependency-free engine (node
only). The supported surfaces are testsuite/format.sh (corpus CLI /
CI check) and the VS Code extension; an ESLint plugin was prototyped
mid-branch and deliberately removed again -- a thin wrapper adding an
npm dependency without using any framework capability (an LSP server
is the planned next editor-integration surface):

- Configurable line-wrap width (`options.printWidth`, default 100 --
  matching ColumnLimit in src/.clang-format) and
  indent size (`options.indentSize`, default 2): a rendered line over
  printWidth gets its outermost splittable bracket group (call args,
  array/mapping literal) broken one element per line, recursively.
  Wrap slices carry their context (mapping element vs not, pending
  ternary '?' count) so key colons stay tight and ternary colons stay
  spaced on every pass.
- Line-break decisions follow the source instead of forcing a
  canonical shape both ways: a `{ ... }` body the source wrote on one
  line stays one line whatever its statement count (and statement
  groups sharing a source line stay merged, single-spaced), if it
  still fits printWidth; a call/condition/declaration already split across
  multiple source lines keeps that layout; a genuinely empty block
  always collapses to `{}`; a brace-less if/while/for/foreach/else
  body the source wrote on its own line keeps that break too --
  including else-if chains and nested dangling-if chains, with an
  `else` re-indenting to the nearest `if` (the one it binds to). The
  tracking works inside anonymous-function bodies nested in call
  arguments (statements there get one line each; nothing glues or
  leaks indent), and `} while` only stays joined for a real do-while
  body. An empty or whitespace-only source (testsuite/clone/inh0.lpc
  is a real intentionally-empty corpus file) formats to an empty
  string, not a manufactured newline.
- NON-empty array/mapping/closure literals (`({ ... })`, `([ ... ])`,
  `(: ... :)`) get one inner padding space; EMPTY `({})`/`([])` stay
  tight -- both confirmed against the pristine, pre-formatting corpus
  rather than an already-reformatted one, since the latter is circular
  (it can only reflect what a prior, possibly-buggy pass already did).
  A mapping's key:value colon is tight before it (`([ "a": 1 ])`,
  matching new()'s class-member-initializer colon), distinguished from
  a ternary colon appearing as the mapping's value via a per-bracket-
  depth `?`/`:` counter; case/default label colons are tight via
  keyword-armed, ternary-aware detection (correct even mid-line and in
  one-lined switches). Bare "::" (the parent/efun bypass call with no
  left-hand qualifier) keeps normal spacing before it; only a
  qualified `identifier::`/`efun::` form is tight on both sides.
- A token-merge safety net in renderLine guarantees no two tokens are
  ever butted together whose concatenation re-lexes differently:
  `a - --b` must not render as `a ---b` (which re-lexes as
  `(a--) - b`), `- -x` must not become the pre-decrement `--x`, and
  `f( ::g() )` must not become `f(::g())`, whose `(:` re-lexes as a
  functional-literal opener. The check asks the real tokenizer
  (cached), so it covers present and future tight-spacing rules.
- The tokenizer handles a `#define` whose `/* */` comment opens on the
  directive line and closes on a LATER physical line (invisible
  whitespace to the directive, not a token boundary), while a QUOTE in
  a directive never extends it past its physical line (`#define Q it'`
  must not swallow the next source line; an unterminated '"' must not
  swallow everything to the next quote in the file). `\`-splices still
  continue directives, including inside a string.
- The formatter never re-spaces a macro argument that a `#define
  NAME(params) body` macro stringizes via `#param` (as opposed to
  plain `##` token paste, which operates on the value and doesn't care
  about spelling) -- `STR(1+2)` keeps stringizing to "1+2", not
  "1 + 2". Detection mirrors the driver's own preprocessing rather
  than naive text matching: directive text is analyzed after folding
  `\`-continuations and stripping comments outside quotes; an ODD-
  length '#' run stringizes (`###x` = paste-then-stringize); flags are
  unioned across every definition in the file (a dead `#if 0`
  redefinition must not strip protection -- the formatter can't
  evaluate #if truth, and over-masking only preserves spacing while
  under-masking corrupts program output); call sites accept keyword-
  shaped macro names (macros resolve before reserved words, so
  `#define string(x) #x` works); and argument boundaries mirror the
  driver's collector, which nests only the '(' character (a comma
  inside `x[...]` really splits driver arguments), with a whole-call
  verbatim freeze when a span would be bracket-unbalanced. No file in
  the corpus is excluded from stringize-aware formatting (the only
  format.sh exclusions are the two raw-byte UTF-8 fixtures).
  testsuite/single/tests/compiler/preprocessor_stringize.lpc pins
  every shape end-to-end through the real driver -- it FAILS when
  formatted with a stringize-naive formatter, so a future
  formatter regression here fails the driver suite, not just the JS
  self-checks.
- Deterministic and idempotent in all cases, including after wrapping,
  under non-default printWidth/indentSize, and across every rule
  above.

testsuite/.gitattributes marks the two deliberately-invalid-UTF8
compiler fixtures `binary`, since the repo root's `* text=auto` rule
can otherwise corrupt them on an unrelated `git checkout` (its CRLF
heuristic misfires on stray bytes inside the invalid sequences).

The full testsuite/**/*.lpc,*.c corpus is reformatted to match,
verified at every step: 0 crashes, 0 token-sequence mismatches against
the pristine pre-formatting corpus, 0 idempotency failures, the two
binary fixtures byte-identical, and the actual FluffOS driver built
and the real LPC testsuite (`driver etc/config.test -ftest`) run
against the reformatted corpus across multiple randomized-order
passes, confirming a clean `Checks succeeded.` with zero regressions.
The driver-suite step (plus an adversarial multi-agent review pass:
state-machine analysis, ~55k-input fuzzing, driver-semantics probing
of the preprocessor, and render-rule review) caught every bug class
the token-equivalence self-check is structurally blind to -- the
directive/comment tokenizer truncation, stringize arguments getting
re-spaced, adjacent-token merges, and compounding indent drift from
dangling-body bookkeeping inside call arguments.

testsuite/format.sh is the corpus auto-formatter: it formats
testsuite/**/*.lpc,*.c in place (--check verifies for CI, exit 1 when
anything is unformatted), needs only node (no npm install), hard-codes
the two malformed-UTF8 fixture exclusions alongside the .gitattributes
protection, and refuses to write any file whose output isn't
token-sequence-equivalent to the input and idempotent
(bin/format-corpus.mjs).

Spacing and layout deliberately mirror the repo's own C++ style
(src/.clang-format: Google base, IndentWidth 2, ColumnLimit 100) on
every language-common rule -- 2-space indent, 100-column wrap,
attached braces with cuddled else/while, `if (` spaced vs call-tight
parens, indented case labels, tight casts including after a keyword
(`return (string)x`), tight unary/`++`/`--`/`[]`/`->`/`::`, spaced
binary/assignment/ternary operators, directives at column 0, no
include sorting, and trailing `//` comments at least two spaces off
the code (SpacesBeforeTrailingComments: 2) with wider hand-aligned
gaps preserved exactly (AlignTrailingComments in spirit, without ever
moving a comment). LPC-specific constructs keep the corpus's own
conventions where C++ has no analogue or the corpus disagrees for a
reason: `type *name` binds the array-marker `*` to the name (~5:1
pristine, opposite of PointerAlignment: Left), literal inner padding
(`({ 1, 2 })`, `([ "a": 1 ])`) with tight empties, call-tight
`catch(`/`new(`, spaced functional-literal bounds (`(: f :)`), and
tight-before default-argument colons. clang-format's line-reflow
canonicalization is deliberately not adopted -- breaks follow the
source (see above).

The VS Code extension (tools/lpc-syntax/vscode/) gets matching
`lpc.format.printWidth`/`lpc.format.indentSize` settings and
regenerated generated-copy files (lib/tokenizer.mjs, lib/format.mjs)
sharing the exact same engine. test.mjs covers the
tokenizer/formatter/linter/generated-VS-Code-asset surface, including
regression tests pinned to every convention and bug fix above. AGENTS.md and the tools/lpc-syntax README document the
testsuite/.gitattributes gotcha, the driver-mirroring stringize
machinery, and the "validate against the real driver, not just
token-equivalence" lesson for future changes to this tooling.


Claude-Session: https://claude.ai/code/session_01HSL1G3iHXu1dd8XhnBQ2fe

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-17 11:55:21 -07:00

92 lines
2.5 KiB
Text

int indexing(string str) {
// Count "e" by using a loop looking at character index in string
int count = 0;
int sz = strlen(str);
for (int i = 0; i < sz; i++) {
if (str[i] == 'e') count++;
}
return count;
}
int exploding(string str) {
// Count "e" using a loop looking at index in an array of strings
int count = 0;
int sz;
string *list;
list = explode(str, "");
sz = sizeof(list);
for (int i = 0; i < sz; i++) {
if (list[i][0] == 'e') count++;
}
return count;
}
string count_chart() {
int w = 100, h = 25;
// w,h are characters, each with 4h 2w dots
// fill below or a dot per column
int *grid = allocate(8 * w * h);
int pre, mid, post, a, b, scale;
string input;
string output = "Each column is 5 chars longer string processed via 2 algorithms, top line means instant\n";
for (int i = 0; i < (2 * w); i++) { // going across
input = sprintf("%*'blargle 's", i * 5, "");
pre = perf_counter_ns();
indexing(input);
mid = perf_counter_ns();
exploding(input);
post = perf_counter_ns();
scale = 2000000;
a = min(({ w - 1, ((mid - pre) / scale) }));
//scale = 1000000;
b = min(({ w - 1, ((post - mid) / scale) }));
grid[i + min(({ w, (a) })) * 2 * w] = 1;
grid[i + min(({ w, (b) })) * 2 * w] = 1;
}
for (int i = 0; i < h; i++) { // going down
for (int j = 0; j < w; j++) { // going across
output += sprintf("%c", 0x2800 +
(grid[i * w * 2 * 4 + j * 2] ? 1 : 0) +
(grid[i * w * 2 * 4 + j * 2 + 1] ? 8 : 0) +
(grid[i * w * 2 * 4 + j * 2 + w * 2] ? 2 : 0) +
(grid[i * w * 2 * 4 + j * 2 + w * 2 + 1] ? 16 : 0) +
(grid[i * w * 2 * 4 + j * 2 + w * 4] ? 4 : 0) +
(grid[i * w * 2 * 4 + j * 2 + w * 4 + 1] ? 32 : 0) +
(grid[i * w * 2 * 4 + j * 2 + w * 6] ? 64 : 0) +
(grid[i * w * 2 * 4 + j * 2 + w * 6 + 1] ? 128 : 0) +
0);
}
output += "\n";
}
return output;
}
string count_x(int x) {
string input;
int pre, mid, post;
input = sprintf("%*'blargle 's", x, "");
pre = perf_counter_ns();
indexing(input);
mid = perf_counter_ns();
exploding(input);
post = perf_counter_ns();
return sprintf(
"Input of %d chars, indexing=%1.3f ms, exploding=%1.3f ms\n",
strlen(input),
(mid - pre) * 0.000001,
(post - mid) * 0.000001
);
}
int main(string arg) {
write(count_chart());
write(count_x(100));
write(count_x(500));
write(count_x(1000));
write(count_x(2000));
write(count_x(5000));
write(count_x(10000));
return 1;
}