test(codiff): run color_ops on every route that executes it, with qemu as oracle
color_ops.c is compiled twice -- once into libmux for the host, once into
the freestanding rv64 blob -- and the blob is then executed by two engines
of our own. That is four implementations of one source, and until now the
only differential we had compared the first against itself.
#2002 is where that bit. A cursor rewrite of the word-list functions was
verified on the host across 560,000 cases with a negative control, and it
still broke in the blob; the attempt was reverted with no root cause,
because reproducing it needed the blob under an external oracle and no box
had run one.
This adds that. One freestanding guest binary uses only Linux syscalls 64
and 93 -- exactly what dbt_test.cpp's ELF harness implements -- so
qemu-riscv64-static, rv64_interp_run and the DBT all execute the SAME
instruction stream rather than three builds of one source. The host leg
links the already-built libmux, which makes it the pre-change
implementation and therefore the specification, not a hand-written table
of expectations.
The battery is 16 fixed cases, 5 at the max_words cap, and 200 seeded
random cases driven by the same LCG on every route, so inputs are
identical by construction. 1,221 transcript lines. The cap cases earn
their place: a cap-check off-by-one moves 4 lines and the random leg
catches it never, while a common-path off-by-one moves 212. Both controls
were run.
What it found on first use is #2019 -- the DBT returns a wrong answer for
correct RV64 code when block chaining is enabled, intermittently. So the
#2002 rewrite was sound and the divergence was ours. repro/ carries that
reproducer: the rewrite as a patch (color_ops.c is generated -- a real
change belongs in color_ops.rl) and a script that loops it and reports a
rate, because a single run passes about half the time.
test-codiff joins TEST_TARGETS and passes on master. test-codiff-2019 is
deliberately NOT in `make test`: it is expected to fail. Both skip loudly
without a RISC-V cross-compiler rather than reporting a pass for a run
that compiled nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:33:37 -06:00
|
|
|
/* runner.cpp -- run one RV64 ELF through the interpreter and the DBT with
|
|
|
|
|
* full visibility into the DBT's counters and (optionally) its trace.
|
|
|
|
|
*
|
|
|
|
|
* tests/dbt/dbt_test can already run an ELF both ways, but it prints only
|
|
|
|
|
* blocks/hits/misses and offers no way to turn on DBT_TRACE_*. This is the
|
|
|
|
|
* same two routes with the internals exposed, so that a divergence can be
|
|
|
|
|
* localised to a translation event rather than merely observed.
|
|
|
|
|
*
|
|
|
|
|
* Env:
|
|
|
|
|
* RUN_TRACE=1 DBT_TRACE_TRANSLATE
|
|
|
|
|
* RUN_TRACE=3 + DBT_TRACE_EXEC (very loud)
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#include <cstdio>
|
|
|
|
|
#include <cstdlib>
|
|
|
|
|
#include <cstring>
|
|
|
|
|
#include "dbt.h"
|
|
|
|
|
#include "dbt_interp.h"
|
|
|
|
|
#include "dbt_elf64.h"
|
|
|
|
|
|
|
|
|
|
static int io_ecall(rv64_state_t *state, void *user)
|
|
|
|
|
{
|
|
|
|
|
rv64_memory_t *mem = static_cast<rv64_memory_t *>(user);
|
|
|
|
|
switch (state->x[17]) {
|
|
|
|
|
case 93: return static_cast<int>(state->x[10]);
|
|
|
|
|
case 64: {
|
|
|
|
|
uint64_t buf = state->x[11], len = state->x[12];
|
|
|
|
|
if (buf + len > mem->size) { state->x[10] = (uint64_t)-1LL; return -1; }
|
|
|
|
|
fwrite(mem->data + buf, 1, len, stdout);
|
|
|
|
|
state->x[10] = len;
|
|
|
|
|
return -1;
|
|
|
|
|
}
|
|
|
|
|
default: return -1;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
test(codiff): add the chain-edge bisection that localised #2019
Knowing chaining is the mechanism does not say WHICH chained edge is
wrong, and with 70 of them in the reproducer that is the difference
between a lead and a location.
2019-chain-bisect.patch adds two scratch knobs to dbt.cpp -- list the
chain targets, and suppress named ones -- and bisect.sh binary-searches
for the smallest set whose suppression makes the failure go away. It is
a debugging patch, not a proposed change: nothing here is meant to be
merged into the engine.
It converges on one PC out of 70. Skipping that edge alone: 0/40 wrong.
Skipping the TAKEN side of the same branch: 19/40. Skipping wc_next's
own entry: 28/40. Skipping an arbitrary other edge: 28/40. So the fault
is one specific edge rather than chaining being generally fragile here.
That edge is the fall-through of a shrink-wrapped early-out: gcc sank
wc_next's prologue below the `finished` test, which makes the
fall-through target both a mid-function entry point and a PC that is not
a branch target in the guest at all -- control simply continues into it.
Its sibling, a real branch target, chains correctly.
The sampling matters and is documented in the script: at ~50% failure,
"0 wrong" over 30 runs is a false clean with probability about 1e-9, and
lowering RUNS quietly turns the bisection into a coin flip.
runner.cpp gains the two hooks the patch defines. They are declared
__attribute__((weak)) and null-checked, because a normal build links the
unpatched dbt.cpp and defines neither -- declaring them plainly breaks
`make test-codiff` at link time, which is how the first version of this
commit was wrong.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 20:01:51 -06:00
|
|
|
/* Hooks supplied only by repro/2019-chain-bisect.patch, a scratch patch to
|
test(codiff): make the harness run on macOS, not skip cleanly
Two ELF assumptions kept the harness from running on Darwin, both failing
in the direction this harness exists to avoid -- quietly, and looking like
a fact about the code under test rather than about the toolchain.
The symbol probe used `nm -D --defined-only`, neither of which macOS nm
accepts, and grepped for a bare `co_insert_at` where Mach-O exports
`_co_insert_at`. With 2>/dev/null hiding the flag error, the pipeline
yielded nothing, the probe concluded the library was stale, and the run
exited 0 advising `make install` -- which had already been run. `nm -g`
plus an optional-underscore pattern is the portable spelling.
The bisect hooks were declared weak and left undefined. An undefined weak
symbol resolves to null on ELF; the Mach-O linker rejects it, so a normal
build failed to link here. Empty weak *definitions* link everywhere and
are still overridden by the strong ones in
repro/2019-chain-bisect.patch. Their null tests go with them -- an
always-defined weak symbol makes the test vacuously true.
Verified on macOS arm64: 1221 transcript lines, interp vs host OK, dbt vs
host OK. Negative control (off-by-one in the wc_next max_words cap) exits
1 with exactly 4 differing lines, all cap/ cases. The qemu oracle still
skips here and says so -- Homebrew ships no user-mode riscv64 emulator on
macOS -- so this box gets three of the four routes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 20:29:54 -06:00
|
|
|
* dbt.cpp used to bisect chained edges. Weak *definitions*, not weak
|
|
|
|
|
* declarations: an undefined weak symbol resolves to null only on ELF, while
|
|
|
|
|
* the Mach-O linker rejects it outright, so the declaration form makes a
|
|
|
|
|
* normal build fail to link on macOS. An empty weak definition links
|
|
|
|
|
* everywhere and is overridden by the patch's strong one when applied. */
|
|
|
|
|
extern "C" __attribute__((weak)) void dbt_dump_chained(void) { }
|
|
|
|
|
extern "C" __attribute__((weak)) void dbt_dump_heads(void) { }
|
test(codiff): add the chain-edge bisection that localised #2019
Knowing chaining is the mechanism does not say WHICH chained edge is
wrong, and with 70 of them in the reproducer that is the difference
between a lead and a location.
2019-chain-bisect.patch adds two scratch knobs to dbt.cpp -- list the
chain targets, and suppress named ones -- and bisect.sh binary-searches
for the smallest set whose suppression makes the failure go away. It is
a debugging patch, not a proposed change: nothing here is meant to be
merged into the engine.
It converges on one PC out of 70. Skipping that edge alone: 0/40 wrong.
Skipping the TAKEN side of the same branch: 19/40. Skipping wc_next's
own entry: 28/40. Skipping an arbitrary other edge: 28/40. So the fault
is one specific edge rather than chaining being generally fragile here.
That edge is the fall-through of a shrink-wrapped early-out: gcc sank
wc_next's prologue below the `finished` test, which makes the
fall-through target both a mid-function entry point and a PC that is not
a branch target in the guest at all -- control simply continues into it.
Its sibling, a real branch target, chains correctly.
The sampling matters and is documented in the script: at ~50% failure,
"0 wrong" over 30 runs is a false clean with probability about 1e-9, and
lowering RUNS quietly turns the bisection into a coin flip.
runner.cpp gains the two hooks the patch defines. They are declared
__attribute__((weak)) and null-checked, because a normal build links the
unpatched dbt.cpp and defines neither -- declaring them plainly breaks
`make test-codiff` at link time, which is how the first version of this
commit was wrong.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 20:01:51 -06:00
|
|
|
|
test(codiff): run color_ops on every route that executes it, with qemu as oracle
color_ops.c is compiled twice -- once into libmux for the host, once into
the freestanding rv64 blob -- and the blob is then executed by two engines
of our own. That is four implementations of one source, and until now the
only differential we had compared the first against itself.
#2002 is where that bit. A cursor rewrite of the word-list functions was
verified on the host across 560,000 cases with a negative control, and it
still broke in the blob; the attempt was reverted with no root cause,
because reproducing it needed the blob under an external oracle and no box
had run one.
This adds that. One freestanding guest binary uses only Linux syscalls 64
and 93 -- exactly what dbt_test.cpp's ELF harness implements -- so
qemu-riscv64-static, rv64_interp_run and the DBT all execute the SAME
instruction stream rather than three builds of one source. The host leg
links the already-built libmux, which makes it the pre-change
implementation and therefore the specification, not a hand-written table
of expectations.
The battery is 16 fixed cases, 5 at the max_words cap, and 200 seeded
random cases driven by the same LCG on every route, so inputs are
identical by construction. 1,221 transcript lines. The cap cases earn
their place: a cap-check off-by-one moves 4 lines and the random leg
catches it never, while a common-path off-by-one moves 212. Both controls
were run.
What it found on first use is #2019 -- the DBT returns a wrong answer for
correct RV64 code when block chaining is enabled, intermittently. So the
#2002 rewrite was sound and the divergence was ours. repro/ carries that
reproducer: the rewrite as a patch (color_ops.c is generated -- a real
change belongs in color_ops.rl) and a script that loops it and reports a
rate, because a single run passes about half the time.
test-codiff joins TEST_TARGETS and passes on master. test-codiff-2019 is
deliberately NOT in `make test`: it is expected to fail. Both skip loudly
without a RISC-V cross-compiler rather than reporting a pass for a run
that compiled nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:33:37 -06:00
|
|
|
struct dctx { uint8_t *memory; size_t size; };
|
|
|
|
|
|
|
|
|
|
static int dbt_io_ecall(rv64_ctx_t *ctx, void *user)
|
|
|
|
|
{
|
|
|
|
|
dctx *d = static_cast<dctx *>(user);
|
|
|
|
|
switch (ctx->x[17]) {
|
|
|
|
|
case 93: return static_cast<int>(ctx->x[10]);
|
|
|
|
|
case 64: {
|
|
|
|
|
uint64_t buf = ctx->x[11], len = ctx->x[12];
|
|
|
|
|
if (buf + len > d->size) { ctx->x[10] = (uint64_t)-1LL; return -1; }
|
|
|
|
|
fwrite(d->memory + buf, 1, len, stdout);
|
|
|
|
|
ctx->x[10] = len;
|
|
|
|
|
return -1;
|
|
|
|
|
}
|
|
|
|
|
default: return -1;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
int main(int argc, char **argv)
|
|
|
|
|
{
|
|
|
|
|
if (argc < 2) { fprintf(stderr, "usage: runner <elf>\n"); return 2; }
|
|
|
|
|
const char *tr = getenv("RUN_TRACE");
|
|
|
|
|
int trace = tr ? atoi(tr) : 0;
|
|
|
|
|
|
|
|
|
|
/* ---- interpreter ---- */
|
|
|
|
|
{
|
|
|
|
|
rv64_binary_t bin;
|
|
|
|
|
if (rv64_load_elf(argv[1], &bin) != 0) return 2;
|
|
|
|
|
rv64_state_t st = {};
|
|
|
|
|
st.pc = bin.entry_point;
|
|
|
|
|
st.x[2] = bin.stack_top;
|
|
|
|
|
rv64_memory_t mem = { bin.memory, bin.memory_size };
|
|
|
|
|
printf("--- interp ---\n");
|
|
|
|
|
fflush(stdout);
|
|
|
|
|
rv64_interp_run(&st, &mem, io_ecall, &mem);
|
|
|
|
|
fflush(stdout);
|
|
|
|
|
rv64_free_binary(&bin);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* ---- DBT ----
|
|
|
|
|
* RUN_REPS>1 repeats the whole init/run/cleanup cycle in one process,
|
|
|
|
|
* to test whether the divergence needs process-global DBT state rather
|
|
|
|
|
* than anything in this ELF. */
|
|
|
|
|
const char *rp = getenv("RUN_REPS");
|
|
|
|
|
int reps = rp ? atoi(rp) : 1;
|
|
|
|
|
for (int rep = 0; rep < reps; rep++) {
|
|
|
|
|
rv64_binary_t bin;
|
|
|
|
|
if (rv64_load_elf(argv[1], &bin) != 0) return 2;
|
|
|
|
|
dctx d = { bin.memory, bin.memory_size };
|
|
|
|
|
dbt_state_t dbt;
|
|
|
|
|
if (dbt_init(&dbt, bin.memory, bin.memory_size, dbt_io_ecall, &d) != 0) {
|
|
|
|
|
fprintf(stderr, "dbt_init failed\n");
|
|
|
|
|
return 2;
|
|
|
|
|
}
|
|
|
|
|
dbt.trace = trace;
|
|
|
|
|
printf("--- dbt rep=%d ---\n", rep);
|
|
|
|
|
fflush(stdout);
|
|
|
|
|
int rc = dbt_run(&dbt, bin.entry_point, bin.stack_top);
|
|
|
|
|
fflush(stdout);
|
|
|
|
|
printf("rc=%d blocks=%llu hits=%llu misses=%llu chain_hits=%llu "
|
|
|
|
|
"chain_misses=%llu code_full=%llu reclaims=%u code_used=%u "
|
|
|
|
|
"blob_end=%u\n",
|
|
|
|
|
rc,
|
|
|
|
|
(unsigned long long)dbt.blocks_translated,
|
|
|
|
|
(unsigned long long)dbt.cache_hits,
|
|
|
|
|
(unsigned long long)dbt.cache_misses,
|
|
|
|
|
(unsigned long long)dbt.chain_hits,
|
|
|
|
|
(unsigned long long)dbt.chain_misses,
|
|
|
|
|
(unsigned long long)dbt.code_full,
|
|
|
|
|
dbt.reclaims_this_run,
|
|
|
|
|
dbt.code_used, dbt.blob_code_end);
|
test(codiff): make the harness run on macOS, not skip cleanly
Two ELF assumptions kept the harness from running on Darwin, both failing
in the direction this harness exists to avoid -- quietly, and looking like
a fact about the code under test rather than about the toolchain.
The symbol probe used `nm -D --defined-only`, neither of which macOS nm
accepts, and grepped for a bare `co_insert_at` where Mach-O exports
`_co_insert_at`. With 2>/dev/null hiding the flag error, the pipeline
yielded nothing, the probe concluded the library was stale, and the run
exited 0 advising `make install` -- which had already been run. `nm -g`
plus an optional-underscore pattern is the portable spelling.
The bisect hooks were declared weak and left undefined. An undefined weak
symbol resolves to null on ELF; the Mach-O linker rejects it, so a normal
build failed to link here. Empty weak *definitions* link everywhere and
are still overridden by the strong ones in
repro/2019-chain-bisect.patch. Their null tests go with them -- an
always-defined weak symbol makes the test vacuously true.
Verified on macOS arm64: 1221 transcript lines, interp vs host OK, dbt vs
host OK. Negative control (off-by-one in the wc_next max_words cap) exits
1 with exactly 4 differing lines, all cap/ cases. The qemu oracle still
skips here and says so -- Homebrew ships no user-mode riscv64 emulator on
macOS -- so this box gets three of the four routes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 20:29:54 -06:00
|
|
|
/* No null test: the weak definitions above make these no-ops in a
|
|
|
|
|
* normal build, and a weak symbol that is always defined would make
|
|
|
|
|
* the test vacuously true anyway. */
|
|
|
|
|
dbt_dump_chained();
|
|
|
|
|
dbt_dump_heads();
|
test(codiff): run color_ops on every route that executes it, with qemu as oracle
color_ops.c is compiled twice -- once into libmux for the host, once into
the freestanding rv64 blob -- and the blob is then executed by two engines
of our own. That is four implementations of one source, and until now the
only differential we had compared the first against itself.
#2002 is where that bit. A cursor rewrite of the word-list functions was
verified on the host across 560,000 cases with a negative control, and it
still broke in the blob; the attempt was reverted with no root cause,
because reproducing it needed the blob under an external oracle and no box
had run one.
This adds that. One freestanding guest binary uses only Linux syscalls 64
and 93 -- exactly what dbt_test.cpp's ELF harness implements -- so
qemu-riscv64-static, rv64_interp_run and the DBT all execute the SAME
instruction stream rather than three builds of one source. The host leg
links the already-built libmux, which makes it the pre-change
implementation and therefore the specification, not a hand-written table
of expectations.
The battery is 16 fixed cases, 5 at the max_words cap, and 200 seeded
random cases driven by the same LCG on every route, so inputs are
identical by construction. 1,221 transcript lines. The cap cases earn
their place: a cap-check off-by-one moves 4 lines and the random leg
catches it never, while a common-path off-by-one moves 212. Both controls
were run.
What it found on first use is #2019 -- the DBT returns a wrong answer for
correct RV64 code when block chaining is enabled, intermittently. So the
#2002 rewrite was sound and the divergence was ours. repro/ carries that
reproducer: the rewrite as a patch (color_ops.c is generated -- a real
change belongs in color_ops.rl) and a script that loops it and reports a
rate, because a single run passes about half the time.
test-codiff joins TEST_TARGETS and passes on master. test-codiff-2019 is
deliberately NOT in `make test`: it is expected to fail. Both skip loudly
without a RISC-V cross-compiler rather than reporting a pass for a run
that compiled nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:33:37 -06:00
|
|
|
dbt_cleanup(&dbt);
|
|
|
|
|
rv64_free_binary(&bin);
|
|
|
|
|
}
|
|
|
|
|
return 0;
|
|
|
|
|
}
|