The flip: FUNCTION/XFUNCTION/FUN::fun/delim_check and the module
interfaces take `const UTF8 * const fargs[]`. Double-const is
load-bearing: C++ qualification conversion needs const at both pointer
levels, so builder-side `UTF8 *[]` arrays convert implicitly — the
evaluator, the JIT marshaller, and every owner site need zero casts,
and slot reassignment inside bodies becomes a compile error for free.
The conversions: the flip landed first so the compiler enumerated every
violation; this commit is that inventory worked to zero — ~250 sites
across funceval, funceval2, functions, funmath, help, mail, session,
powers, levels, predicates, conf, walkdb, stringutil, timeutil/
date_scan (regenerated, one-line diff), exp3, and mux_main, each
classified per docs/campaign-2136-const-fargs.md's four recipes.
New idioms (functions.h): trim_space_sep_n() — non-destructive trim for
(pointer, length) consumers, so trim-then-scan sites need no copy at
all; FargVec — the argv counterpart of FargCopy for CS_ARGV handlers.
countwords() and DecodeListOfIntegers() rewritten non-destructive.
The flip deleted more than it added: #2157's fun_munge list1 copy, the
engine_com help-topic copy, fun_index's in-place NUL write, and five
const_casts (process_sex x4, sha1_helper). const_cast budget: zero
added.
Trap recorded in the brief: an old-signature definition doesn't fail
the build — it becomes a C++ overload, and the new-signature symbol
stays undefined until dlopen(RTLD_NOW). delim_check, the conn_bridge
bridges, the dbt_spike stub, and exp3::Call were all silently shadowed;
muxscript was the only host that noticed, because netmux's own net.cpp
resolved the flat-namespace lookup. After any signature flip, grep the
old spelling.
Verified: make test EXPECT_CONFIG="jit=yes" (35 passed / 0 failed) and
make test-scenario, including the new tests/scenario/sidefx_fargs.py
that live-probes the class-3 wrappers smoke never touches (pemit/
trigger/link/tel/wipe/destroy).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Started as the boring half of #1653 -- jit_compiler.cpp held 26 of the 79
remaining raw printf-family sites -- and the guard found a real bug in the
process.
ECALL_ORD wrote into guest memory with no bound at all:
op += sprintf(op, "%ld", static_cast<long>(ch));
It is fun_ord()'s loop transliterated, but the interpreter writes through
safe_chr/safe_ltoa, which stop at the buffer end. Dropping those for raw
pointer arithmetic dropped the only thing bounding the write.
The check above it reads like it bounds the write and does not:
if (in_addr >= ec->memory_size || out_addr >= ec->memory_size - 64)
That guarantees 64 bytes of headroom against a loop emitting up to 8 bytes
per CODEPOINT, and a grapheme cluster is one cluster but any number of
codepoints. Measured on ord() of 'a' followed by N combining acutes, which
is one cluster of player-supplied input:
N=400 1602 bytes written
N=1200 4802 bytes written
N=3800 15202 bytes written
Linear in the input, no cap. It does not crash only because out_addr happens
to sit far enough from the end of guest memory; everything past the result
slot is overwritten either way, and that margin is accidental rather than
checked.
Now bounded by the space that actually exists, stopping cleanly the way the
interpreter does when its LBUF fills. AST and JIT agree byte-for-byte at
N = 5, 60, 400, 1200 and 3800, so the hazard is gone with no behaviour
change.
Worth noting how it surfaced: my own `grep "snprintf("` missed it, because it
is sprintf. The guard's ban list covers the family, so it did not.
The other 25 conversions are mechanical, except that two were also latent:
- memcpy(dst, nbuf, len + 1) where len is snprintf's WOULD-write, so a
truncation would have overrun a small stack buffer
- n += snprintf(..., LBUF_SIZE - n, ...) where an over-long n sends a
negative through a size_t parameter; guarded today only by an explicit
n < LBUF_SIZE - 256 check that exists because of those semantics
Both stop being possible with a did-write return.
Which needed one: mux_snprintf, in libmux, returning the length mux_vsnprintf
already computes and mux_sprintf discards. mail_mod.cpp grew a private
mail_sprintf for exactly this, and copying it here would have made a third --
the same duplication #1667's ADR objects to for append_ljust_field, which is
byte-identical across two modules today. Registered in check_formats.py so
its call sites are checked like any other wrapper.
It is NOT snprintf's return: snprintf answers what it WOULD have written, so
`n >= size` is how callers detect truncation. This answers what it DID
write. Anything ported across that tests the return against the buffer size
has to be re-read rather than renamed -- which is why the conversions above
drop the clamps instead of keeping them.
Legacy raw printf-family sites: 79 -> 53.
make test green; test-lua-jit 1561/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gettext catalogues reorder multi-conversion msgids with %1$s / %2$d.
The sequential path cannot resynchronise a va_list cursor, so when a
format contains %N$:
1. scan every conversion for 1-based index and type
2. pull arguments 1..max from va_list in index order
3. format using the saved values
Sequential formats keep the original single-pass path. Mixed
positional/sequential (other than %%) fails closed (#1429).
Un-fuzzy the seven Korean reorderings, flip run_ko case 4 to require
substitution, drop the temporary check_nls reject of non-fuzzy %N$,
and compare catalogue conversions by argument position.
#1517 landed the same defect an hour before this branch was opened, so master
already carried a fix. git merged both mechanically and the result was clean
but WRONG: it kept my `if (0 != nLen)` wrapper alongside this branch's null
early-return, and left my comment in place claiming things the merged code no
longer does -- that MU_Substitutes[4] is `{ 0, nullptr }` (this branch changes
it to `{ 0, T("") }`), and that a null src with a non-zero count "stays
undefined on purpose" (the early return handles it).
Resolved in favour of this branch's version, which is the better fix:
* Null-tolerance is the established convention among these helpers.
safe_copy_str, safe_copy_str_lbuf and utf8_safe_chr all return early on a
null src; safe_copy_buf was the lone outlier. My length-based guard was
the inconsistent one.
* The early return also covers TrimPartialSequence, which dereferences
p[iStart] whenever n > 0 and sits BEFORE my guard. So "leave null with a
non-zero count undefined" was not merely a stylistic choice -- it left a
null dereference one call away.
* `{ 0, T("") }` removes the null at source. A null in a table of string
literals is a hazard for whoever indexes it next, and the change is
behaviourally identical: safe_copy_buf(T(""), 0) does what
safe_copy_buf(nullptr, 0) did.
My `if (0 != nLen)` wrapper is dropped as redundant once src cannot be null --
memcpy with a valid pointer and a zero count is well defined.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
UBSan reports "null pointer passed as argument 2" at stringutil.cpp:4348.
Passing null to memcpy is UB even when the length is zero -- the standard
requires valid pointers regardless (C17 7.24.1p2).
The caller was not obvious from the report, so I instrumented
safe_copy_buf on Windows to capture a symbolized stack whenever src was
null and ran the smoke suite. It fires 5 times, always with nLen == 0,
always from the same place:
translate_string stringutil.cpp:3643
eval_ecall jit_compiler.cpp:4456
dbt_run dbt.cpp:641
run_cached_program jit_compiler.cpp:2854
jit_eval jit_compiler.cpp:5190
mux_exec ast.cpp:2928
The source is a data table, not a computation. MU_EscapeConvert maps 0x0D
(CR) to substitute 4, and MU_Substitutes[4] was { 0, nullptr } -- the
deliberate "emit nothing" slot. translate_string hands it straight to
safe_copy_buf, which hands it to memcpy. So every CR that passes through
translate_string is one of these, which is why it reproduces readily under
a sanitizer and never misbehaves without one.
Fixed at both ends:
- MU_Substitutes[4] is now { 0, T("") }. A null in a table of string
literals is a trap for anyone else who indexes it; empty says the same
thing and cannot be dereferenced.
- safe_copy_buf returns early on a null src. Its siblings
safe_copy_str, safe_copy_str_lbuf and utf8_safe_chr all already do
this, so tolerating null is the established convention among these
helpers and safe_copy_buf was the lone outlier -- worth closing across
all 63 call sites rather than only the one that trips today. It also
covers TrimPartialSequence, which would dereference src outright had a
caller passed null with a non-zero length.
No behaviour change: both spellings emit zero bytes.
Verified on Windows Server 2022, MSVC 14.51, Release x64. Clean rebuild,
0 warnings, 0 errors. Smoke 316 dispatched / 1487 succeeded / 17 failed --
identical to the pre-change baseline, the 17 being the known build-config
set (exp3, UNIX_DIGEST, REALITY_LVLS).
The behaviour is already pinned by translate_fn.mux TC004, which asserts
translate(%r,p) == %r -- %r is CRLF, so that case consumes substitute 4
(CR, emit nothing) and substitute 3 (LF, "%r"). It still passes, so no new
test is needed. Confirmed live as well: @decompile of an attribute holding
alpha%rbeta%rgamma returns it unchanged.
Windows has no UBSan, so this cannot verify the report itself is gone --
that needs the Linux --enable-sanitizers build from #1449.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last open item on #1458. UBSan on Linux/aarch64 reported
stringutil.cpp:4348:11 null pointer passed as argument 2
against the memcpy in safe_copy_buf. Passing null to memcpy is undefined
even when the count is zero -- the parameter is declared non-null, so the
standard does not care that nothing would be copied, and a compiler is
entitled to infer from the call that src cannot be null and delete a later
check for it.
The other seven findings in that issue are already fixed on master and I
verified each: the three svdhash misaligned uint32_t loads now go through
mux_read_u32, mux_atoi64 accumulates and negates in uint64_t, timeutil's
iHash is uint32_t, and dbt_emit_a64 special-cases rot == 0. This was the
only one left.
## It is reached, and by a caller that is not wrong
Apple clang does not implement this check -- I confirmed a deliberate
memcpy(dst, nullptr, 0) goes unreported here even with
-fsanitize=undefined,nonnull-attribute,builtin -O0 -- so the report cannot
be reproduced on this box. Reachability can be measured instead, and that
is the more useful question anyway.
Instrumenting safe_copy_buf across a full Makesmoke + Smoke run:
2,392,544 calls, 4 with src == nullptr, all with nLen == 0
Backtraces put all four in translate_string(), via the JIT ECALL path
(mux_exec -> jit_eval -> run_cached_program -> dbt_run -> eval_ecall). The
source is
static LITERAL_STRING_STRUCT MU_Substitutes[NUM_MU_SUBS] =
{
...
{ 0, nullptr }, // 4
...
};
-- a deliberate "substitute nothing" entry. A (pointer, length) pair with
length zero is a perfectly good empty string, so the caller is not at fault
and the table should not be changed to carry a dummy T(""). The guard
belongs in the helper, where it also covers the other 58 call sites.
A null src with a NON-zero count deliberately stays undefined: that would be
a caller bug and silencing it here would hide it.
## On the instrumentation
Two of my probes were vacuous before one worked, which is worth recording
because both failures looked like clean results:
* A UBSan probe reported nothing -- but so did its own control, a direct
memcpy(dst, nullptr, 0). Without that control the "clean" run would have
read as evidence the defect was not real.
* A stderr probe printed nothing -- because Smoke deletes netmux.log on
success, not because the path was cold. Writing to a fixed file instead
produced the counts above.
Verified: build clean, smoke 1505 passed / 0 failed / 316 of 316 dispatched,
test-format 1170 call sites plus 31744 differential assertions green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#1435 stopped an unimplemented conversion from aborting the process: the
spec is echoed literally and formatting continues. Echoing is right;
continuing is not.
No va_arg is consumed for a spec the formatter cannot parse, so from
that point the argument list is misaligned and every later conversion in
the same format string reads the wrong argument. Measured on master,
Linux x86-64:
"%d|%d" 11, 22 -> "11|22" baseline
"%s|%+d" "head", 11 -> "head|%+d" safe, nothing follows
"%+d|%d" 11, 22 -> "%+d|11" silently the WRONG argument
"%+d|%s" 11, "tail" -> SIGSEGV
The last one hands an int to the %s path as a UTF8* and dereferences it.
That trades a clean abort for a wild pointer, which is not the
improvement #1429 was for -- a crash that used to name its own file and
line is now an unattributed segfault somewhere else.
There is no way to resynchronise without knowing what the unknown spec
would have taken, and no way to learn that without implementing it. So
recovery now stops at the first spec it cannot parse: the remainder of
the format is echoed verbatim and interpretation ends. Conversions
BEFORE the bad spec are still honoured -- "%s|%+d" still renders its
%s -- so stopping is not the same as discarding the format.
Echoing the whole remainder rather than just the offending spec keeps
the evidence intact: the spec appears in full, with the text that
followed it, while nothing further is interpreted.
Bug-catch: with the formatter reverted to master's continue and these
tests kept, the test binary SEGFAULTS rather than reporting a failure --
the same unambiguous signal astbench gave before #1385.
tests/format 31744 passed, 0 failed
smoke 1497/1497 on both routes, 0 crashes, 315/315 dispatched
Closes#1445.
An unimplemented conversion reached mux_assert(0), and that is an
unconditional abort() in the shipping build -- mux_assert has no NDEBUG
guard and AssertionFailed calls abort() outright. So an ordinary looking
T("%+d") did not produce wrong output, it killed the game.
#1416 implemented %i, %o and the floating-point conversions, which shrank
the surface considerably. This is the other half: the remaining gaps stop
being fatal. Both matter together -- implementing conversions means fewer
gaps, and not aborting means the next gap someone finds is a formatting
bug rather than an outage. That is the pattern #1382 produced twice
already (@list cache and astbench), and both were fixed by rewriting the
call site to dodge %f rather than by making the formatter survive.
The spec is now echoed literally and formatting continues. Emitting it
rather than dropping it leaves the evidence in the output, visible without
a debugger. A bounded formatter already truncates when it runs out of
room, so callers are written against "imperfect output", not against "no
server".
Caveat, documented at the site: no va_arg is consumed for an unimplemented
spec, so a later conversion in the same format string takes a shifted
argument. That is inherent to recovering without fully parsing an unknown
spec, and is still strictly better than terminating.
Added tests/format coverage for the seven forms named in the issue, plus
surrounding text and the %% case so the recovery cannot break what already
worked. snprintf is deliberately not the oracle for these -- echoing
literally is not what printf does -- so they carry explicit expectations.
Verified both directions on Win64. With the fix, 31738 passed / 0 failed.
Against a libmux rebuilt without it, the same test binary:
EXIT=127
C:\tinymux\mux\lib\stringutil.cpp(6701): Assertion failed.
Note that stdout was empty in that run: the abort discards even the
thousands of results that had already passed. That is the cost of the old
behaviour in miniature.
Smoke: 315 dispatched, 1477 succeeded, 17 failed -- the known
build-configuration failures on this box (exp3, UNIX_DIGEST,
REALITY_LVLS). No assertions or crashes in the log.
Scope is mux_vsnprintf only; mux_assert elsewhere in the tree is untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mux_vsnprintf implements printf's conversions by hand, and anything it did
not implement fell through to mux_assert(0) -- so a caller reaching for a
standard C conversion took the server down. That is #1382 in fun_astbench,
and independently the same shape in @list. Both were fixed by rewriting the
call site to avoid %f, which leaves the trap in place for the next caller.
A sweep of every call site through the four wrappers (tprintf,
safe_tprintf_str, mux_sprintf, mux_fprintf) found no third instance today:
986 call sites, 909 with a format, 1749 conversion specs, all supported. But
"no third instance today" is not a property anyone can maintain by reading,
the restriction is invisible at the call site, and the penalty is the whole
process. Better to implement the conversions than to keep forbidding them.
Added: %i (alias of %d), %o, and %f %F %e %E %g %G.
Floating point takes its digits from mux_dtoa -- the same correctly rounded
generator mux_ftoa, fval and NearestPretty already use -- so float output does
not depend on the host libc. mux_ftoa itself is not usable here: it is MUX's
own float rendering and switches to exponent form once the decimal point
passes 18, which %f never does. dtoa suppresses trailing zeros and reports a
decimal-point position, so padding back out to the requested precision, the
%g e-vs-f selection, and the exponent form are assembled here. Note dtoa owns
its buffer and reclaims it on the next call (MULTIPLE_THREADS is not defined),
so callers must not free -- matching fval.
Octal needed mux_utoo/mux_ui64too, and a wider scratch: 64-bit octal is 22
digits, one more than LONGEST_I64 allows for decimal.
Hand-assembled float formatting is exactly the code that looks right and is
wrong at the boundaries, so tests/format compares against the platform
snprintf over 6 conversions x 9 precisions x 5 widths x 3 flags x ~40 values,
plus infinities, NaN, negative zero, ties where round-half-even differs, the
integer/fraction boundary, %i, %o, 64-bit octal, and truncation.
That oracle immediately earned itself: the first run was 31396 passed / 332
failed, all of them %g of values rendering as the empty string. The strip of
trailing zeros ran over the whole buffer instead of the fraction, so it ate
the integer digit -- "%.1g" of 0 produced "" rather than "0". Bounded to the
fraction, 31728 pass and none fail.
Wired into `make test` as test-format. Smoke unchanged at 1488 passed,
314/314 dispatched.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Completes the sweep the issue called for. mux_atol returns long, which
is 32-bit on LLP64, so every caller silently truncated on Windows. Two
of those were real defects (the truthiness family and cf_size, fixed in
the preceding commits); the rest were latent, waiting for a value large
enough to matter.
Rather than audit 290 sites for whether each can reach 2^31 today, use
the 64-bit parser everywhere and remove the class. A dbref cannot
overflow now, but nothing stops a later caller passing that same site a
timestamp or a byte count.
Pure 1:1 substitution: 285 lines changed, and every removed line
contained mux_atol while every added line contains mux_atoi64. No
control flow, no types, no behaviour beyond the wider parse.
This is a NO-OP on LP64 -- long is already 64-bit on Linux and macOS, so
the generated code there is unchanged. It only widens the parse on
Windows. Narrowing destinations are unaffected either way: `int x =
mux_atoi64(s)` truncates exactly as `int x = mux_atol(s)` did, on both
models.
Left alone: mux_atol itself in mathutil, its declaration, and three
comments that name it. Callers that genuinely want 32-bit semantics can
still ask for them; none appear to.
Verified on Windows: full solution builds clean with no new warnings,
smoke is 1418 passed / 16 failed / 0 crashes / 306 of 306 dispatched --
identical to before the sweep, with the same 16 build-configuration
failures (exp3 module not loaded, hmac/digest behind UNIX_DIGEST).
Spot checks after the change: the boolean family returns 1 for multiples
of 2^32, cf_size round-trips 3000000000 and still reads -1 as unlimited,
and arithmetic, string and list functions are unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
mux_vsnprintf is a custom, UTF-8/PUA-color-aware formatter and only
handled d/s/u/x/X/p/c with 'l' length modifiers. Any other specifier
fell through to mux_assert(0) -> AssertionFailed -> abort().
ganl_adapter's NET/STAT line (8c971724c) switched to %zu/%llu/%lld/%ld.
LOG_NET is on by default and log_socket_stats() runs from the main loop
with an unprimed rate limiter, so netmux aborted on the first periodic
pass -- immediately after "GANL: Entering main loop" (exit 134).
Add 'z' as a length modifier mapped onto the existing nLongs tiers by
actual width, so LP64 (size_t == unsigned long) and LLP64 (size_t is
64-bit while long is 32-bit) both marshal the va_arg type the caller
pushed.
This also closes a pre-existing latent class of aborts at other %zu
call sites routed through mux_vsnprintf -- notably lib/alloc.cpp's
pool-budget warning, which would have aborted the server instead of
warning about it: net.cpp:159, alloc.cpp:64, command.cpp (@lua stats),
cque.cpp:1175, mail_mod.cpp:4968.
Verified: netmux survives a 100s run emitting well-formed NET/STAT
lines with zero assertions; smoke suite 1319/1319 pass, 0 crashes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Eliminate the ISOUTOFMEMORY macro that unconditionally aborted on allocation
failure. Each of the 23 call sites now handles OOM appropriately:
- Fatal sites (buffer pools, db array, anum table): mux_assert or OutOfMemory
- Recoverable sites (queue, mail, commands, guests, vattrs, config, restart,
forward lists): log the failure and return gracefully
Also fix g_dump_child_pid portability: volatile pid_t -> volatile sig_atomic_t
with explicit casts in ganl_adapter.cpp.
Close integer overflow issue as false alarm (getstring_noalloc uses a bounded
static buffer, so nBuffer+1 cannot wrap).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
These functions do full Unicode mux_tolower state machine lookups
per character, but ~99% of object names and commands are pure ASCII.
Add a fast loop that compares via mux_tolower_ascii[] table lookup
while both bytes are < 0x80, falling through to the Unicode path
only when non-ASCII is encountered.
Eliminates string_prefix (4.45%) and string_compare (3.71%) from
the top of the perf profile.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add LBuf class to alloc.h: an RAII wrapper around alloc_lbuf/free_lbuf
that moves LBUF_SIZE buffers from the stack to the heap pool. Convert
all 108 non-static UTF8 xxx[LBUF_SIZE] stack arrays across 25 source
files. Static BSS buffers (24) are unchanged.
This eliminates LBUF_SIZE from recursive stack frames, making it safe
to increase LBUF_SIZE without risking stack overflow in the evaluation
pipeline.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The modular architecture (netmux.exe, libmux.dll, engine.dll, etc.)
with static CRT (/MT) gave each module its own CRT heap. FILE* handles
from mux_fopen (in libmux) crashed when used by stdio functions in
other modules — fclose in write_pidfile, fgets in cf_include, etc.
Switch all 11 vcxproj files to /MD (shared CRT DLL) so all modules
share one CRT instance. Ship msvcp140.dll, vcruntime140.dll, and
vcruntime140_1.dll in the binary distribution.
Also add mux_fclose to libmux as good hygiene (pairs with mux_fopen),
and replace all cross-module fclose calls. This change is safe on Unix
where everything links into one process.
Add Startmux.bat as a replacement for Startmux.wsf since Windows
Script Host is no longer associated by default on modern Windows.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New Ragel -G2 functions in color_ops.rl for PUA-to-client rendering:
co_dfa_ascii(p) — approximate one UTF-8 code point to ASCII via
the tr_ascii DFA (e.g., e-acute → 'e', n-tilde → 'n'). Exported
with C linkage for use from both C and C++ code.
co_render_ascii(out, data, len) — strip PUA color codes and
convert all visible Unicode to ASCII in a single Ragel pass.
For NOANSI/ASCII client output.
Retarget C++ ConvertToAscii() in stringutil.cpp to delegate to
co_dfa_ascii() instead of duplicating the DFA driver loop. This
eliminates the duplicate tr_ascii table traversal code.
Unit tests: 11 new test cases in tests/color_ops/ covering empty,
pure ASCII, color+ASCII, Unicode approximation (e-acute, n-tilde,
u-umlaut, c-cedilla), mixed color+Unicode, RGB color, all-color,
and 5000-iteration fuzz verifying all output is 7-bit ASCII.
505/505 smoke tests pass. 305/305 color_ops unit tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add C-linkage wrappers in color_ops.h for FindNearestPaletteEntry(),
which uses CIE97 perceptual distance with K-d tree search through
the xterm palette. Works on both Unix (no mangling) and Windows
(extern "C" prevents MSVC name decoration).
co_nearest_xterm256(rgb) — search full 256-entry palette
co_nearest_xterm16(rgb) — search 16-entry ANSI palette
Update TitanFugue's TrueColor fallback path to use
co_nearest_xterm256() instead of the Euclidean-in-RGB rgb_to_xterm()
approximation. Perceptually more accurate color matching on
256-color terminals.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- color_ops.h: Add LIBMUX_API to all co_* declarations; fix CO_CS_NORMAL
compound literal for MSVC (both C and C++ modes)
- color_ops.c: Remove redundant (co_ColorState) casts that fail on MSVC C
- utf8tables.h/cpp: Add extern "C" wrapping for C/C++ linkage compatibility
- stringutil.h/cpp: extern "C" on utf8_FirstByte; LIBMUX_API on co_console_width
- unicode_tables_c.h: Guard extern declarations with #ifndef __cplusplus
(C++ gets them from utf8tables.h with proper LIBMUX_API and types);
add CO_OTT_CAST macro for string_desc/co_string_desc pointer conversion;
add proper LIBMUX_API fallback with dllexport/dllimport for Windows
- libmux.vcxproj: Add color_ops.c and unicode_tables.c as C compilation units
- ast.cpp: Use QueryPerformanceCounter for WIN32 astbench timing
- engine_com.cpp, functions.h, functions.cpp: Guard JIT-only code with TINYMUX_JIT
- utf/ generators: Add LIBMUX_API to all extern declarations in smutil.cpp,
strings.cpp, pairs.cpp; strings.cpp guards OTT externs with #ifndef __cplusplus;
preamble/postamble updated with CO_OTT_CAST and proper LIBMUX_API fallback
NOTE: Build not yet clean — ubuntu agent needs to regenerate auto-generated
files (utf8tables.h, utf8tables.cpp, unicode_tables_c.h) from updated
utf/ generators to pick up all LIBMUX_API and linkage fixes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the old variable-length per-channel delta encoding (U+F0000-F05FF,
6 blocks of 256, 7-15 bytes per layer) with a fixed-size 2-code-point
encoding (U+F0000-F3FFF, 4 blocks of 4096, always 11 bytes per layer).
CP1 packs R high nibble + G, CP2 packs R low nibble + B. The XTERM
indexed base (3 bytes) is preserved for all 4 output paths.
The DFA shrinks from 37 to 11 states (1536 SMP entries removed from
tr_Color.txt); SMP detection uses a prefix check before the DFA.
Flatfile version bumped to 5 with automatic V4→V5 migration on load.
All 502 smoke tests and 294 standalone color_ops tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch from Y'UV (BT.601 with ad-hoc 1.5x Y weight) to CIE76
(Euclidean distance in CIELAB) for RGB-to-XTERM palette matching.
sRGB->XYZ->CIELAB pipeline with D65 illuminant. Integer-scaled
L*a*b* (x100) for K-d tree operations.
Changes the nearest-neighbor result for 52.7% of the 16.7M RGB
color space. All 502 smoke tests pass.
Files: stringutil.h (LABi replaces YUV), stringutil.cpp (rgb2lab,
diff, tree search, palette table), t5xgame.cpp (convert tool).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove the mux_string class (2400+ lines), mux_words, isEmpty helper,
CursorMin/CursorMax/curAscii statics, and the dead queue_string
mux_string overload from net.cpp. All PUA color handling now uses
co_* Ragel functions and free-standing C/C++ helpers. mux_cursor and
mux_field are retained (used by utf8_next_grapheme and linewrap).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Full color tracking replaces mux_string for center/ljust/rjust/cpad/lpad/rpad:
- parse_fill_chars: absorbs PUA into per-character color states (like mux_string import)
- emit_fill_from_chars: emits fill with proper color transitions and phase offset
- emit_data_with_tracking: tracks color state through content PUA codes
- Trailing emit_transition to CS_NORMAL for clean state at end
Fixes fill patterns containing internal resets (%cn) that previously leaked
through raw PUA passthrough. Removes boundary-reset hack (has_pua_color,
emit_color_reset, emit_fill_columns). Removes debug logging from functions.cpp.
593/593 smoke tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fun_squish: single-char uses co_compress; multi-char falls back to
mux_string::compress().
fun_tr: keep transform_range() for a-z expansion, export expanded
from/to sets, then use co_transform for the actual mapping.
Remove dead mux_string methods: transform(), transform_Ascii(),
compress_Spaces(). 593/593 tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fun_reverse: co_reverse replaces mux_string import/reverse/export.
fun_edit: co_edit with ^/$ prepend/append and \^/%^ escape handling,
ping-pong buffers eliminate mux_string allocation entirely.
fun_trim: single-char trim uses co_trim; multi-char pattern falls
back to mux_string::trim(p,n,...) for cyclic pattern stripping.
Fix edit_fn.mux: TC003/TC004 separator was '--' (not recognized by
unformat.pl), so those tests never ran. Fixed to '-'. Updated TC003
SHA1 for co_edit's color-preserving behavior.
Remove dead mux_string methods: reverse(), edit(),
trim(const UTF8*, bool, bool). 593/593 tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
