fluffos/tools/lpc-syntax/vscode/syntaxes/lpc.tmLanguage.json
Yucong Sun 6b6f169952
lpc-syntax: wire formatter into vscode extension, fix tokenizer/formatter bugs (#1259)
* lpc-syntax: wire formatter into vscode extension, fix tokenizer/formatter bugs

- Register a DocumentFormattingEditProvider (Format Document / format-on-save)
  backed by format.mjs, gated by a new lpc.format.enabled setting; never lets
  a formatter error corrupt or block a save.
- Regenerate the grammar contract (grammar.y already had `ref` = '&' sugar
  that lpc-grammar.json/grammar.ebnf hadn't picked up) and make operator-list
  generation deterministic (secondary alphabetical sort key instead of
  relying on Python's randomized string-hash set ordering).
- tokenizer.mjs: fix template-interpolation brace scanning to skip nested
  strings/chars/comments/templates as opaque spans (a stray '}' inside e.g.
  `${ ch == '}' }` previously ended the interpolation early); fix char
  literals with variable-length \xHH/\NNN escapes being truncated.
- format.mjs: track array/mapping literal braces `({ ... })` separately from
  block braces so they don't affect indentation depth; force a flush after a
  trailing `//` comment so a second format pass can't swallow following code
  into it; fix an off-by-one that mis-indented every nested block; stop
  accumulating a blank line on re-format of a source that swallows to EOF.
- language-configuration.json: add onEnterRules for /** */ doc-comment
  continuation.
- Extend test.mjs with regression coverage for all of the above (59 checks).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XUhzkBiuWxX2M9RX94BckT

* lpc-syntax: fix heredoc, mapping-literal, case-colon, and indexing spacing in formatter

Verified tokenizer/highlighter already model heredoc (@/@@ text blocks)
correctly per parseHeredoc() in lexer_utils.cc. Found and fixed four real
formatter bugs, all in format.mjs:

- Mapping literals `([ ... ])` never got the array-literal treatment
  ({ ... }) got last session -- only '{'/'}' was tracked, not '['/']'.
  Generalized the brace-tracking into one combined stack covering both,
  distinguishing array/mapping literals from blocks/indexing by whether
  the bracket is immediately preceded by '('.
- Both array and mapping literals collapsed onto a single line even when
  the source spread them across many lines, which mangles real mudlib
  data tables. Multi-line literals now preserve their line breaks and
  indent one level, while short single-line literals still collapse as
  before.
- `case`/`default` labels rendered as "case 1 :" (space before the
  colon) -- checked against testsuite convention (843:6 no-space vs
  space) and fixed; ternary/mapping colons are unaffected.
- `a[0]`/`b[1..2]` rendered as "a [0]" / "b [1 .. 2]" (space before '['
  and around the range operator) -- checked against testsuite
  convention (1603:15, 197:7) and fixed; varargs '...' spacing is
  unaffected.

Fixing the heredoc terminator to force a line break (matching the
documented @/@@ style, since the driver rescans trailing code after the
terminator on its own) exposed a latent bug: the ';'-triggered flush
computed paren depth over just the current line buffer, which goes
negative (never reaches the expected 0) once a forced mid-statement
flush leaves an unmatched ')' behind. Replaced it with a running
paren-nesting counter across the whole pass.

Re-verified via an independent 723-file sweep of testsuite/ (tokenizer
lossless reconstruction, formatter idempotency + literal-content
preservation, highlighter lossless reconstruction, lint false-positive
check on real files): all clean except one confirmed non-issue
(intentional trailing-whitespace trim on a directive line). Added 6
regression tests (65 total) and regenerated the vscode/lib/format.mjs copy.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XUhzkBiuWxX2M9RX94BckT

* lpc-syntax: fix highlighting gaps found by auditing against grammar.y and docs/lpc

Cross-referenced the highlighting pipeline (tokenizer.mjs kind classification,
highlight.mjs, generate_ebnf.py's TextMate grammar generation) against
lexer_utils.cc's reswords[] table and every page under docs/lpc/.

Verified already correct, no change: `inherited` is genuinely not a keyword
(any identifier before `::` is treated uniformly, matching docs/lpc/constructs/
inherit.md's own examples); the full type/modifier keyword lists match
reswords[] exactly; range/spread/optional-chaining/nullish operators already
have distinct scopes; `array` staying highlighted as a keyword despite
ARRAY_RESERVED_WORD being #undef'd by default is a pre-existing, low-impact
gap not worth a schema change to plumb through.

Real gaps fixed, all in generate_ebnf.py/highlight.mjs (never hand-edit the
generated lpc-grammar.json/lpc.tmLanguage.json themselves):

- "struct" was an undocumented reserved word (lexer_utils.cc maps both
  "class" and "struct" to L_CLASS, both gated on unconditionally-defined
  macros) but TOKEN_SPEC only listed "class" -- struct declarations
  highlighted as a plain identifier. Added the second spelling.
- class/struct are type-introducing keywords, not control flow -- split them
  out of keyword.control.lpc into their own storage.type.class.lpc scope,
  matching how other C-family TextMate grammars color struct/class.
- The function-call heuristic (identifier immediately before '(') had no
  guard against matching a reserved word, relying only on TextMate's
  same-position rule-order tie-break. Added an explicit negative lookahead
  over the full keyword/type/modifier set so `if (`/`new (`/etc. can never
  be misscoped as entity.name.function.lpc.
- $1/$2 closure params had no visual distinction in the HTML highlighter
  (they intentionally still tokenize as plain 'identifier', since format.mjs
  keys spacing off that kind) -- fixed at the highlight.mjs layer with a
  dedicated lpc-param class, matching the TextMate grammar's existing
  dollar-params rule.
- Illegal/unknown characters rendered with no visual flag in the HTML
  highlighter -- added an lpc-unknown class so invalid syntax is visible.

Re-verified via an independent 723-file sweep of testsuite/ (highlighter
lossless reconstruction: 0 crashes, 0 mismatches) and the full test suite
(70 checks, all passing). tokenizer.mjs, lint.mjs, and format.mjs are
untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XUhzkBiuWxX2M9RX94BckT

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-12 15:42:14 -04:00

211 lines
5.1 KiB
JSON

{
"$comment": "GENERATED from lpc-grammar.json by generate_ebnf.py (CMake target generate_ebnf) -- do not edit by hand.",
"name": "LPC",
"scopeName": "source.lpc",
"fileTypes": [
"lpc"
],
"patterns": [
{
"include": "#comments"
},
{
"include": "#preprocessor"
},
{
"include": "#textblock"
},
{
"include": "#template"
},
{
"include": "#strings"
},
{
"include": "#chars"
},
{
"include": "#numbers"
},
{
"include": "#functional"
},
{
"include": "#class-keyword"
},
{
"include": "#keywords"
},
{
"include": "#types"
},
{
"include": "#modifiers"
},
{
"include": "#dollar-params"
},
{
"include": "#function-call"
},
{
"include": "#operators"
}
],
"repository": {
"comments": {
"patterns": [
{
"name": "comment.line.double-slash.lpc",
"match": "//.*$"
},
{
"name": "comment.block.lpc",
"begin": "/\\*",
"end": "\\*/"
}
]
},
"preprocessor": {
"name": "meta.preprocessor.lpc",
"begin": "^\\s*(#\\s*[A-Za-z_]*)",
"beginCaptures": {
"1": {
"name": "keyword.control.directive.lpc"
}
},
"end": "(?<!\\\\)$",
"patterns": [
{
"include": "#comments"
},
{
"include": "#strings"
},
{
"name": "string.quoted.other.include.lpc",
"match": "<[^>\\n]*>"
},
{
"include": "#numbers"
},
{
"name": "constant.character.escape.line-continuation.lpc",
"match": "\\\\$"
}
]
},
"textblock": {
"name": "string.unquoted.textblock.lpc",
"begin": "(@@?)([A-Za-z_][A-Za-z0-9_]*)$",
"beginCaptures": {
"1": {
"name": "keyword.operator.textblock.lpc"
},
"2": {
"name": "constant.other.terminator.lpc"
}
},
"end": "^\\2(?![A-Za-z0-9_])",
"endCaptures": {
"0": {
"name": "constant.other.terminator.lpc"
}
}
},
"template": {
"name": "string.interpolated.lpc",
"begin": "`",
"end": "`",
"patterns": [
{
"name": "constant.character.escape.lpc",
"match": "\\\\."
},
{
"name": "meta.embedded.interpolation.lpc",
"begin": "\\$\\{",
"end": "\\}",
"beginCaptures": {
"0": {
"name": "punctuation.section.interpolation.begin.lpc"
}
},
"endCaptures": {
"0": {
"name": "punctuation.section.interpolation.end.lpc"
}
},
"patterns": [
{
"include": "$self"
}
]
}
]
},
"strings": {
"name": "string.quoted.double.lpc",
"begin": "\"",
"end": "\"",
"patterns": [
{
"name": "constant.character.escape.lpc",
"match": "\\\\."
}
]
},
"chars": {
"name": "constant.character.lpc",
"match": "'(\\\\.|[^'\\\\])'"
},
"numbers": {
"name": "constant.numeric.lpc",
"match": "\\b(0[xX][0-9A-Fa-f_]+|0[bB][01_]+|[0-9][0-9_]*(\\.(?!\\.)[0-9_]*)?([eE][+-]?[0-9][0-9_]*)?)"
},
"functional": {
"patterns": [
{
"name": "keyword.operator.functional.lpc",
"match": "\\(:"
},
{
"name": "keyword.operator.functional.lpc",
"match": ":\\)"
}
]
},
"class-keyword": {
"name": "storage.type.class.lpc",
"match": "\\b(struct|class)\\b"
},
"keywords": {
"name": "keyword.control.lpc",
"match": "\\b(time_expression|parse_command|__TREE__|continue|default|foreach|inherit|return|sscanf|switch|array|break|catch|while|case|efun|else|for|new|ref|do|if|in)\\b"
},
"types": {
"name": "storage.type.lpc",
"match": "\\b(function|mapping|buffer|object|string|float|mixed|void|int)\\b"
},
"modifiers": {
"name": "storage.modifier.lpc",
"match": "\\b(protected|private|varargs|nomask|nosave|public|static)\\b"
},
"dollar-params": {
"name": "variable.parameter.positional.lpc",
"match": "\\$[0-9]+"
},
"function-call": {
"match": "\\b(?!(?:time_expression|parse_command|protected|__TREE__|function|continue|private|default|foreach|mapping|inherit|varargs|buffer|public|object|sscanf|static|return|struct|nomask|switch|string|nosave|while|class|break|float|array|catch|mixed|else|case|efun|void|ref|new|for|int|do|if|in)\\b)([A-Za-z_][A-Za-z0-9_]*)\\s*(?=\\()",
"captures": {
"1": {
"name": "entity.name.function.lpc"
}
}
},
"operators": {
"name": "keyword.operator.lpc",
"match": "\\&\\&=|\\.\\.\\.|<<=|>>=|\\?\\?=|\\|\\|=|!=|%=|\\&\\&|\\&=|\\(:|\\*=|\\+\\+|\\+=|\\-\\-|\\-=|\\->|\\.\\.|\\.\\?|/=|:\\)|::|<<|<=|==|>=|>>|\\?\\.|\\?\\?|\\^=|\\|=|\\|\\||=|>"
}
}
}