tinymux/tests/codiff/runner.cpp

127 lines
4.5 KiB
C++
Raw Permalink Normal View History

test(codiff): run color_ops on every route that executes it, with qemu as oracle color_ops.c is compiled twice -- once into libmux for the host, once into the freestanding rv64 blob -- and the blob is then executed by two engines of our own. That is four implementations of one source, and until now the only differential we had compared the first against itself. #2002 is where that bit. A cursor rewrite of the word-list functions was verified on the host across 560,000 cases with a negative control, and it still broke in the blob; the attempt was reverted with no root cause, because reproducing it needed the blob under an external oracle and no box had run one. This adds that. One freestanding guest binary uses only Linux syscalls 64 and 93 -- exactly what dbt_test.cpp's ELF harness implements -- so qemu-riscv64-static, rv64_interp_run and the DBT all execute the SAME instruction stream rather than three builds of one source. The host leg links the already-built libmux, which makes it the pre-change implementation and therefore the specification, not a hand-written table of expectations. The battery is 16 fixed cases, 5 at the max_words cap, and 200 seeded random cases driven by the same LCG on every route, so inputs are identical by construction. 1,221 transcript lines. The cap cases earn their place: a cap-check off-by-one moves 4 lines and the random leg catches it never, while a common-path off-by-one moves 212. Both controls were run. What it found on first use is #2019 -- the DBT returns a wrong answer for correct RV64 code when block chaining is enabled, intermittently. So the #2002 rewrite was sound and the divergence was ours. repro/ carries that reproducer: the rewrite as a patch (color_ops.c is generated -- a real change belongs in color_ops.rl) and a script that loops it and reports a rate, because a single run passes about half the time. test-codiff joins TEST_TARGETS and passes on master. test-codiff-2019 is deliberately NOT in `make test`: it is expected to fail. Both skip loudly without a RISC-V cross-compiler rather than reporting a pass for a run that compiled nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:33:37 -06:00
/* runner.cpp -- run one RV64 ELF through the interpreter and the DBT with
* full visibility into the DBT's counters and (optionally) its trace.
*
* tests/dbt/dbt_test can already run an ELF both ways, but it prints only
* blocks/hits/misses and offers no way to turn on DBT_TRACE_*. This is the
* same two routes with the internals exposed, so that a divergence can be
* localised to a translation event rather than merely observed.
*
* Env:
* RUN_TRACE=1 DBT_TRACE_TRANSLATE
* RUN_TRACE=3 + DBT_TRACE_EXEC (very loud)
*/
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include "dbt.h"
#include "dbt_interp.h"
#include "dbt_elf64.h"
static int io_ecall(rv64_state_t *state, void *user)
{
rv64_memory_t *mem = static_cast<rv64_memory_t *>(user);
switch (state->x[17]) {
case 93: return static_cast<int>(state->x[10]);
case 64: {
uint64_t buf = state->x[11], len = state->x[12];
if (buf + len > mem->size) { state->x[10] = (uint64_t)-1LL; return -1; }
fwrite(mem->data + buf, 1, len, stdout);
state->x[10] = len;
return -1;
}
default: return -1;
}
}
test(codiff): add the chain-edge bisection that localised #2019 Knowing chaining is the mechanism does not say WHICH chained edge is wrong, and with 70 of them in the reproducer that is the difference between a lead and a location. 2019-chain-bisect.patch adds two scratch knobs to dbt.cpp -- list the chain targets, and suppress named ones -- and bisect.sh binary-searches for the smallest set whose suppression makes the failure go away. It is a debugging patch, not a proposed change: nothing here is meant to be merged into the engine. It converges on one PC out of 70. Skipping that edge alone: 0/40 wrong. Skipping the TAKEN side of the same branch: 19/40. Skipping wc_next's own entry: 28/40. Skipping an arbitrary other edge: 28/40. So the fault is one specific edge rather than chaining being generally fragile here. That edge is the fall-through of a shrink-wrapped early-out: gcc sank wc_next's prologue below the `finished` test, which makes the fall-through target both a mid-function entry point and a PC that is not a branch target in the guest at all -- control simply continues into it. Its sibling, a real branch target, chains correctly. The sampling matters and is documented in the script: at ~50% failure, "0 wrong" over 30 runs is a false clean with probability about 1e-9, and lowering RUNS quietly turns the bisection into a coin flip. runner.cpp gains the two hooks the patch defines. They are declared __attribute__((weak)) and null-checked, because a normal build links the unpatched dbt.cpp and defines neither -- declaring them plainly breaks `make test-codiff` at link time, which is how the first version of this commit was wrong. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 20:01:51 -06:00
/* Hooks supplied only by repro/2019-chain-bisect.patch, a scratch patch to
2026-08-03 20:29:54 -06:00
* dbt.cpp used to bisect chained edges. Weak *definitions*, not weak
* declarations: an undefined weak symbol resolves to null only on ELF, while
* the Mach-O linker rejects it outright, so the declaration form makes a
* normal build fail to link on macOS. An empty weak definition links
* everywhere and is overridden by the patch's strong one when applied. */
extern "C" __attribute__((weak)) void dbt_dump_chained(void) { }
extern "C" __attribute__((weak)) void dbt_dump_heads(void) { }
test(codiff): add the chain-edge bisection that localised #2019 Knowing chaining is the mechanism does not say WHICH chained edge is wrong, and with 70 of them in the reproducer that is the difference between a lead and a location. 2019-chain-bisect.patch adds two scratch knobs to dbt.cpp -- list the chain targets, and suppress named ones -- and bisect.sh binary-searches for the smallest set whose suppression makes the failure go away. It is a debugging patch, not a proposed change: nothing here is meant to be merged into the engine. It converges on one PC out of 70. Skipping that edge alone: 0/40 wrong. Skipping the TAKEN side of the same branch: 19/40. Skipping wc_next's own entry: 28/40. Skipping an arbitrary other edge: 28/40. So the fault is one specific edge rather than chaining being generally fragile here. That edge is the fall-through of a shrink-wrapped early-out: gcc sank wc_next's prologue below the `finished` test, which makes the fall-through target both a mid-function entry point and a PC that is not a branch target in the guest at all -- control simply continues into it. Its sibling, a real branch target, chains correctly. The sampling matters and is documented in the script: at ~50% failure, "0 wrong" over 30 runs is a false clean with probability about 1e-9, and lowering RUNS quietly turns the bisection into a coin flip. runner.cpp gains the two hooks the patch defines. They are declared __attribute__((weak)) and null-checked, because a normal build links the unpatched dbt.cpp and defines neither -- declaring them plainly breaks `make test-codiff` at link time, which is how the first version of this commit was wrong. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 20:01:51 -06:00
test(codiff): run color_ops on every route that executes it, with qemu as oracle color_ops.c is compiled twice -- once into libmux for the host, once into the freestanding rv64 blob -- and the blob is then executed by two engines of our own. That is four implementations of one source, and until now the only differential we had compared the first against itself. #2002 is where that bit. A cursor rewrite of the word-list functions was verified on the host across 560,000 cases with a negative control, and it still broke in the blob; the attempt was reverted with no root cause, because reproducing it needed the blob under an external oracle and no box had run one. This adds that. One freestanding guest binary uses only Linux syscalls 64 and 93 -- exactly what dbt_test.cpp's ELF harness implements -- so qemu-riscv64-static, rv64_interp_run and the DBT all execute the SAME instruction stream rather than three builds of one source. The host leg links the already-built libmux, which makes it the pre-change implementation and therefore the specification, not a hand-written table of expectations. The battery is 16 fixed cases, 5 at the max_words cap, and 200 seeded random cases driven by the same LCG on every route, so inputs are identical by construction. 1,221 transcript lines. The cap cases earn their place: a cap-check off-by-one moves 4 lines and the random leg catches it never, while a common-path off-by-one moves 212. Both controls were run. What it found on first use is #2019 -- the DBT returns a wrong answer for correct RV64 code when block chaining is enabled, intermittently. So the #2002 rewrite was sound and the divergence was ours. repro/ carries that reproducer: the rewrite as a patch (color_ops.c is generated -- a real change belongs in color_ops.rl) and a script that loops it and reports a rate, because a single run passes about half the time. test-codiff joins TEST_TARGETS and passes on master. test-codiff-2019 is deliberately NOT in `make test`: it is expected to fail. Both skip loudly without a RISC-V cross-compiler rather than reporting a pass for a run that compiled nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:33:37 -06:00
struct dctx { uint8_t *memory; size_t size; };
static int dbt_io_ecall(rv64_ctx_t *ctx, void *user)
{
dctx *d = static_cast<dctx *>(user);
switch (ctx->x[17]) {
case 93: return static_cast<int>(ctx->x[10]);
case 64: {
uint64_t buf = ctx->x[11], len = ctx->x[12];
if (buf + len > d->size) { ctx->x[10] = (uint64_t)-1LL; return -1; }
fwrite(d->memory + buf, 1, len, stdout);
ctx->x[10] = len;
return -1;
}
default: return -1;
}
}
int main(int argc, char **argv)
{
if (argc < 2) { fprintf(stderr, "usage: runner <elf>\n"); return 2; }
const char *tr = getenv("RUN_TRACE");
int trace = tr ? atoi(tr) : 0;
/* ---- interpreter ---- */
{
rv64_binary_t bin;
if (rv64_load_elf(argv[1], &bin) != 0) return 2;
rv64_state_t st = {};
st.pc = bin.entry_point;
st.x[2] = bin.stack_top;
rv64_memory_t mem = { bin.memory, bin.memory_size };
printf("--- interp ---\n");
fflush(stdout);
rv64_interp_run(&st, &mem, io_ecall, &mem);
fflush(stdout);
rv64_free_binary(&bin);
}
/* ---- DBT ----
* RUN_REPS>1 repeats the whole init/run/cleanup cycle in one process,
* to test whether the divergence needs process-global DBT state rather
* than anything in this ELF. */
const char *rp = getenv("RUN_REPS");
int reps = rp ? atoi(rp) : 1;
for (int rep = 0; rep < reps; rep++) {
rv64_binary_t bin;
if (rv64_load_elf(argv[1], &bin) != 0) return 2;
dctx d = { bin.memory, bin.memory_size };
dbt_state_t dbt;
if (dbt_init(&dbt, bin.memory, bin.memory_size, dbt_io_ecall, &d) != 0) {
fprintf(stderr, "dbt_init failed\n");
return 2;
}
dbt.trace = trace;
printf("--- dbt rep=%d ---\n", rep);
fflush(stdout);
int rc = dbt_run(&dbt, bin.entry_point, bin.stack_top);
fflush(stdout);
printf("rc=%d blocks=%llu hits=%llu misses=%llu chain_hits=%llu "
"chain_misses=%llu code_full=%llu reclaims=%u code_used=%u "
"blob_end=%u\n",
rc,
(unsigned long long)dbt.blocks_translated,
(unsigned long long)dbt.cache_hits,
(unsigned long long)dbt.cache_misses,
(unsigned long long)dbt.chain_hits,
(unsigned long long)dbt.chain_misses,
(unsigned long long)dbt.code_full,
dbt.reclaims_this_run,
dbt.code_used, dbt.blob_code_end);
2026-08-03 20:29:54 -06:00
/* No null test: the weak definitions above make these no-ops in a
* normal build, and a weak symbol that is always defined would make
* the test vacuously true anyway. */
dbt_dump_chained();
dbt_dump_heads();
test(codiff): run color_ops on every route that executes it, with qemu as oracle color_ops.c is compiled twice -- once into libmux for the host, once into the freestanding rv64 blob -- and the blob is then executed by two engines of our own. That is four implementations of one source, and until now the only differential we had compared the first against itself. #2002 is where that bit. A cursor rewrite of the word-list functions was verified on the host across 560,000 cases with a negative control, and it still broke in the blob; the attempt was reverted with no root cause, because reproducing it needed the blob under an external oracle and no box had run one. This adds that. One freestanding guest binary uses only Linux syscalls 64 and 93 -- exactly what dbt_test.cpp's ELF harness implements -- so qemu-riscv64-static, rv64_interp_run and the DBT all execute the SAME instruction stream rather than three builds of one source. The host leg links the already-built libmux, which makes it the pre-change implementation and therefore the specification, not a hand-written table of expectations. The battery is 16 fixed cases, 5 at the max_words cap, and 200 seeded random cases driven by the same LCG on every route, so inputs are identical by construction. 1,221 transcript lines. The cap cases earn their place: a cap-check off-by-one moves 4 lines and the random leg catches it never, while a common-path off-by-one moves 212. Both controls were run. What it found on first use is #2019 -- the DBT returns a wrong answer for correct RV64 code when block chaining is enabled, intermittently. So the #2002 rewrite was sound and the divergence was ours. repro/ carries that reproducer: the rewrite as a patch (color_ops.c is generated -- a real change belongs in color_ops.rl) and a script that loops it and reports a rate, because a single run passes about half the time. test-codiff joins TEST_TARGETS and passes on master. test-codiff-2019 is deliberately NOT in `make test`: it is expected to fail. Both skip loudly without a RISC-V cross-compiler rather than reporting a pass for a run that compiled nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:33:37 -06:00
dbt_cleanup(&dbt);
rv64_free_binary(&bin);
}
return 0;
}