mirror of
https://github.com/fluffos/fluffos
synced 2026-08-12 18:26:06 -04:00
* lpc-syntax: wire formatter into vscode extension, fix tokenizer/formatter bugs
- Register a DocumentFormattingEditProvider (Format Document / format-on-save)
backed by format.mjs, gated by a new lpc.format.enabled setting; never lets
a formatter error corrupt or block a save.
- Regenerate the grammar contract (grammar.y already had `ref` = '&' sugar
that lpc-grammar.json/grammar.ebnf hadn't picked up) and make operator-list
generation deterministic (secondary alphabetical sort key instead of
relying on Python's randomized string-hash set ordering).
- tokenizer.mjs: fix template-interpolation brace scanning to skip nested
strings/chars/comments/templates as opaque spans (a stray '}' inside e.g.
`${ ch == '}' }` previously ended the interpolation early); fix char
literals with variable-length \xHH/\NNN escapes being truncated.
- format.mjs: track array/mapping literal braces `({ ... })` separately from
block braces so they don't affect indentation depth; force a flush after a
trailing `//` comment so a second format pass can't swallow following code
into it; fix an off-by-one that mis-indented every nested block; stop
accumulating a blank line on re-format of a source that swallows to EOF.
- language-configuration.json: add onEnterRules for /** */ doc-comment
continuation.
- Extend test.mjs with regression coverage for all of the above (59 checks).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XUhzkBiuWxX2M9RX94BckT
* lpc-syntax: fix heredoc, mapping-literal, case-colon, and indexing spacing in formatter
Verified tokenizer/highlighter already model heredoc (@/@@ text blocks)
correctly per parseHeredoc() in lexer_utils.cc. Found and fixed four real
formatter bugs, all in format.mjs:
- Mapping literals `([ ... ])` never got the array-literal treatment
({ ... }) got last session -- only '{'/'}' was tracked, not '['/']'.
Generalized the brace-tracking into one combined stack covering both,
distinguishing array/mapping literals from blocks/indexing by whether
the bracket is immediately preceded by '('.
- Both array and mapping literals collapsed onto a single line even when
the source spread them across many lines, which mangles real mudlib
data tables. Multi-line literals now preserve their line breaks and
indent one level, while short single-line literals still collapse as
before.
- `case`/`default` labels rendered as "case 1 :" (space before the
colon) -- checked against testsuite convention (843:6 no-space vs
space) and fixed; ternary/mapping colons are unaffected.
- `a[0]`/`b[1..2]` rendered as "a [0]" / "b [1 .. 2]" (space before '['
and around the range operator) -- checked against testsuite
convention (1603:15, 197:7) and fixed; varargs '...' spacing is
unaffected.
Fixing the heredoc terminator to force a line break (matching the
documented @/@@ style, since the driver rescans trailing code after the
terminator on its own) exposed a latent bug: the ';'-triggered flush
computed paren depth over just the current line buffer, which goes
negative (never reaches the expected 0) once a forced mid-statement
flush leaves an unmatched ')' behind. Replaced it with a running
paren-nesting counter across the whole pass.
Re-verified via an independent 723-file sweep of testsuite/ (tokenizer
lossless reconstruction, formatter idempotency + literal-content
preservation, highlighter lossless reconstruction, lint false-positive
check on real files): all clean except one confirmed non-issue
(intentional trailing-whitespace trim on a directive line). Added 6
regression tests (65 total) and regenerated the vscode/lib/format.mjs copy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XUhzkBiuWxX2M9RX94BckT
* lpc-syntax: fix highlighting gaps found by auditing against grammar.y and docs/lpc
Cross-referenced the highlighting pipeline (tokenizer.mjs kind classification,
highlight.mjs, generate_ebnf.py's TextMate grammar generation) against
lexer_utils.cc's reswords[] table and every page under docs/lpc/.
Verified already correct, no change: `inherited` is genuinely not a keyword
(any identifier before `::` is treated uniformly, matching docs/lpc/constructs/
inherit.md's own examples); the full type/modifier keyword lists match
reswords[] exactly; range/spread/optional-chaining/nullish operators already
have distinct scopes; `array` staying highlighted as a keyword despite
ARRAY_RESERVED_WORD being #undef'd by default is a pre-existing, low-impact
gap not worth a schema change to plumb through.
Real gaps fixed, all in generate_ebnf.py/highlight.mjs (never hand-edit the
generated lpc-grammar.json/lpc.tmLanguage.json themselves):
- "struct" was an undocumented reserved word (lexer_utils.cc maps both
"class" and "struct" to L_CLASS, both gated on unconditionally-defined
macros) but TOKEN_SPEC only listed "class" -- struct declarations
highlighted as a plain identifier. Added the second spelling.
- class/struct are type-introducing keywords, not control flow -- split them
out of keyword.control.lpc into their own storage.type.class.lpc scope,
matching how other C-family TextMate grammars color struct/class.
- The function-call heuristic (identifier immediately before '(') had no
guard against matching a reserved word, relying only on TextMate's
same-position rule-order tie-break. Added an explicit negative lookahead
over the full keyword/type/modifier set so `if (`/`new (`/etc. can never
be misscoped as entity.name.function.lpc.
- $1/$2 closure params had no visual distinction in the HTML highlighter
(they intentionally still tokenize as plain 'identifier', since format.mjs
keys spacing off that kind) -- fixed at the highlight.mjs layer with a
dedicated lpc-param class, matching the TextMate grammar's existing
dollar-params rule.
- Illegal/unknown characters rendered with no visual flag in the HTML
highlighter -- added an lpc-unknown class so invalid syntax is visible.
Re-verified via an independent 723-file sweep of testsuite/ (highlighter
lossless reconstruction: 0 crashes, 0 mismatches) and the full test suite
(70 checks, all passing). tokenizer.mjs, lint.mjs, and format.mjs are
untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XUhzkBiuWxX2M9RX94BckT
---------
Co-authored-by: Claude <noreply@anthropic.com>
61 lines
1.9 KiB
JavaScript
61 lines
1.9 KiB
JavaScript
// HTML syntax highlighter over the grammar-driven tokenizer.
|
|
// highlightLPC(source) -> HTML string of <span class="lpc-KIND"> runs.
|
|
|
|
import { tokenize } from './tokenizer.mjs';
|
|
|
|
const CSS_CLASS = {
|
|
comment: 'lpc-comment',
|
|
directive: 'lpc-directive',
|
|
string: 'lpc-string',
|
|
template: 'lpc-string',
|
|
textblock: 'lpc-string',
|
|
char: 'lpc-string',
|
|
number: 'lpc-number',
|
|
keyword: 'lpc-keyword',
|
|
efunkw: 'lpc-keyword',
|
|
type: 'lpc-type',
|
|
modifier: 'lpc-modifier',
|
|
identifier: 'lpc-identifier',
|
|
operator: 'lpc-operator',
|
|
punctuation: 'lpc-punctuation',
|
|
functional: 'lpc-functional',
|
|
unknown: 'lpc-unknown',
|
|
};
|
|
|
|
// $1/$2/... closure params tokenize as plain 'identifier' (format.mjs relies
|
|
// on that kind for call-parenthesis spacing) but still deserve their own
|
|
// visual color, matching the VS Code grammar's dollar-params scope.
|
|
const DOLLAR_PARAM = /^\$[0-9]+$/;
|
|
|
|
const escapeHtml = (s) =>
|
|
s.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>')
|
|
.replace(/"/g, '"');
|
|
|
|
export function highlightLPC(source) {
|
|
let out = '';
|
|
for (const t of tokenize(source)) {
|
|
let cls = CSS_CLASS[t.kind];
|
|
if (t.kind === 'identifier' && DOLLAR_PARAM.test(t.text)) cls = 'lpc-param';
|
|
if (cls === undefined || t.kind === 'whitespace') {
|
|
out += escapeHtml(t.text);
|
|
} else {
|
|
out += `<span class="${cls}">${escapeHtml(t.text)}</span>`;
|
|
}
|
|
}
|
|
return out;
|
|
}
|
|
|
|
// Default stylesheet consumers may embed (kept minimal on purpose).
|
|
export const defaultCss = `
|
|
.lpc-comment { color: #6a737d; font-style: italic; }
|
|
.lpc-directive { color: #b30086; }
|
|
.lpc-string { color: #22863a; }
|
|
.lpc-number { color: #005cc5; }
|
|
.lpc-keyword { color: #d73a49; font-weight: 600; }
|
|
.lpc-type { color: #6f42c1; }
|
|
.lpc-modifier { color: #6f42c1; font-style: italic; }
|
|
.lpc-operator { color: #444; }
|
|
.lpc-functional { color: #e36209; font-weight: 600; }
|
|
.lpc-param { color: #e36209; }
|
|
.lpc-unknown { color: #fff; background: #d73a49; }
|
|
`;
|