fluffos/docs/efun/buffers/to_buffer.md
Yucong Sun 814c4fa5b9
vm: string foreach/ref fixes, buffers as byte arrays (foreach, strict bytes, to_buffer), thorough ref tests; #1196 docs follow-up (#1250)
* docs/tests: fix constructs index regression from #1196; add ref page to sidebar; extend & ref tests

- Restore extension-less links and the text_blocks entry in
  docs/lpc/constructs/index.md (the PR was recreated from a pre-Docusaurus
  branch and reintroduced .html links, which fail the docs build under
  onBrokenLinks: 'throw', and dropped text_blocks)
- Add lpc/constructs/ref to the hand-authored sidebar in docs/sidebars.ts
- Drop the stale VitePress 'layout: doc' frontmatter from ref.md
- Extend testsuite/single/tests/operators/ref.lpc: & in parameter
  declarations, ref keyword in foreach, and bitwise &/&= non-regression

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

* vm: fix string foreach crash and off-by-one ref reads; support += / -= on string chars

Three related defects around the shared string-codepoint lvalue
(global_lvalue_codepoint), all reproduced on the unfixed binary:

- Nested foreach over strings SEGFAULTED: the iteration cursor lived in
  the shared global, so the inner loop's F_EXIT_FOREACH reset the
  iterator out from under the outer loop (null deref in
  post_index_to_offset). The EGC cursor now lives in each loop's own
  stack slot (the T_NUMBER slot under the loop variable); the ref case
  re-arms the shared codepoint lvalue every iteration, and exit only
  clears the global when it still points at this loop's string slot.

- foreach (int ref c in str) read the WRONG characters: the shared
  index was advanced before the body ran, so every read through the ref
  was off by one ('abc' summed to 197 instead of 294, and the final
  iteration read one past the end). The global index now stays on the
  current character for the whole body. Writes through the ref still go
  to the loop's by-value stack copy and never reach the iterated
  variable -- semantics pinned by tests/operators/foreach.lpc.

- s[i] += n / s[i] -= n threw "Bad Argument 1 to +=()": F_ADD_EQ and
  f_sub_eq handled buffer byte lvalues (T_LVALUE_BYTE) but not string
  codepoint lvalues (T_LVALUE_CODEPOINT), even though ++/--/= worked.
  Both now route through a new codepoint_lvalue_add() helper and produce
  the resulting character as the rvalue.

Regression tests: nested (plain / ref / multi-byte UTF-8) string foreach
and ref-read correctness in tests/operators/foreach.lpc; compound
assignment on string chars (incl. reverse index, rvalue result, and
non-number rhs error) in tests/operators/string_index.lpc. Verified on
the unfixed binary: the nested-foreach test segfaults, the others fail.
Full LPC suite passes 2x on clang ASan/UBSan Debug and 2x on
RelWithDebInfo.

Docs: correct the ref.md note on foreach-over-strings; document the
string-char lvalue rules in AGENTS.md (section 8 + audit checklist 9)
and the ref/& feature in README.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

* vm: foreach ref over strings shares the s[i] char-lvalue logic; thorough ref tests

Arming a string-char lvalue now goes through one shared helper,
aim_lvalue_codepoint(): it validates the target EGC (out-of-bounds and
multi-codepoint error cleanly, same messages as s[i]) and points the
shared codepoint state at it. Both s[i] lvalues (push_indexed_lvalue)
and foreach ref loop variables use it, and every write consumer already
funnels into assign_lvalue_codepoint() -- so ref loop chars follow
exactly the s[i] rules:

- single-codepoint characters (and EGCs up to 4 bytes, which index as
  their first codepoint) keep working: reads deliver the character,
  assignments through the loop variable succeed
- wider EGCs (flag emoji, ZWJ sequences) now raise the catchable
  "Indexed character is multi-codepoint" error when the ref loop
  reaches them, instead of silently reading as -1; the non-ref form
  still iterates and delivers -1 for such clusters

tests/operators/ref.lpc is now a thorough pass-by-reference suite:
ref/& parameter declarations and call arguments, by-value contrast,
ref forwarding through call chains, call-site refs to array elements /
mapping values / string chars (forward and reverse index, write-back),
foreach ref over arrays / mapping values / strings (read correctness,
multi-byte codepoints, by-value write semantics, assignment error
parity, the multi-codepoint error, non-ref contrast), compile-time
rejections via generated sources (ref outside an argument list, ref to
a range), and bitwise &/&= non-regression -- 39 checks.

Full LPC suite passes 2x on RelWithDebInfo and 2x on a clean clang
ASan/UBSan Debug build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

* tests: cover ref across all LPC types

Extend tests/operators/ref.lpc to 58 checks: ref parameters of every
value type (int, float, string, array, mapping, object, function,
buffer, class, mixed) verifying reassignment propagates; the by-value
contrast for reference-typed containers (member writes propagate,
reassignment doesn't); call-site refs to buffer bytes (forward and
reverse index) and class members (both . and -> spellings); foreach
ref over mapping values of mixed types; the compile-time rejection of
a ref mapping KEY in foreach; and the clean runtime error for foreach
over a non-iterable type (buffer).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

* buffers behave like byte arrays: foreach, strict 0..255 bytes, string/array promotion, to_buffer()

Buffers are now first-class byte containers:

- foreach iterates a buffer like an array, delivering each byte as an
  unsigned int 0..255. A ref loop variable mutates the buffer in place;
  each ref carries its OWN T_LVALUE_BYTE (in ref->sv), so nested buffer
  ref loops and b[i] lvalues in the body can't alias each other.
  T_LVALUE_BYTE consumers now read the lvalue's own pointer/subtype
  instead of reaching for the shared global_lvalue_byte (which remains
  only as the scratch instance b[i] arms).

- every LPC byte write path (=, ++, --, +=, -=) range-checks the result:
  a value outside 0..255 raises "Buffer byte value out of range" and
  leaves the byte unchanged, instead of silently truncating/wrapping.
  += / -= on bytes also yield the resulting value as their rvalue.

- strings and arrays of ints 0..255 PROMOTE to buffers: a new
  to_buffer() efun (registered like to_int/to_float) is wrapped around
  the rhs by do_promotions() / rule_expr_assign for 'buffer b = str',
  'b += str', initializers, and 'b + str'; range assignment and the
  runtime + / += paths convert unpromoted (mixed) values through the
  same svalue_to_buffer_bytes() helper. A string contributes its raw
  UTF-8 bytes; an array must hold only ints 0..255 (validated before
  allocation) or the conversion errors with the target unchanged.

- fixed a pre-existing ref_t leak: a foreach ref loop variable reused by
  a re-entered inner loop leaked one ref per outer iteration (flagged by
  the debug memory checker as 'Found temporary block: make_ref').

tests/operators/buffer_bytes.lpc pins the byte-range semantics, + / +=
concatenation, all promotion forms, and the error paths (106 checks);
foreach.lpc and ref.lpc pin buffer iteration and ref-loop independence;
buffer_range_assign.lpc gains range-read pins. New docs for to_buffer
(sidebar regenerated) and a rewritten lpc/types/buffer page.

Full LPC suite passes 2x on RelWithDebInfo and 2x on a clean clang
ASan/UBSan Debug build (no ref-checker warnings); GTest suite 312/312.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Skf9CfHwAngWorDNDzPKWp

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-12 01:39:30 -04:00

1.2 KiB

title
buffers / to_buffer

to_buffer

NAME

to_buffer() - convert a string or an array of ints to a buffer

SYNOPSIS

buffer to_buffer( string | buffer | mixed *value );

DESCRIPTION

Returns a buffer holding the bytes of <value>:

- A string contributes its raw UTF-8 bytes (a multi-byte character
  becomes several bytes).
- An array must contain only ints in the range 0..255; each element
  becomes one byte. Any other item raises an error.
- A buffer is returned unchanged.

This efun is also the implicit promotion used by the compiler, so a
string or an array of ints can be used directly wherever a buffer is
expected in an assignment context:

- `buffer b = "abc";` and `b = ({ 1, 2, 255 });`
- `b += "abc";` and `b + ({ 0 })` (appends the converted bytes)
- `b[i..j] = "abc";` (buffer range assignment)

EXAMPLES

```c
buffer b;

b = to_buffer("é");        // ({ 0xC3, 0xA9 }) -- 2 UTF-8 bytes
b = to_buffer(({ 7, 255 }));
b = to_buffer(({ 300 }));  // error: item out of 0..255
```

SEE ALSO

allocate_buffer(3), bufferp(3), read_buffer(3), write_buffer(3),
to_int(3), to_float(3)