From c15deee4a124ba3d8b3c3f81a47e23793953ed6e Mon Sep 17 00:00:00 2001 From: Rod Boev Date: Mon, 27 Jul 2026 09:31:54 -0400 Subject: [PATCH] docs(benchmarks): remove unreproducible savings claims (#2593) --- README.md | 9 +-------- docs/content/docs/index.mdx | 7 +------ wiki/index.md | 9 --------- 3 files changed, 2 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index f46e40640..91bd892c0 100644 --- a/README.md +++ b/README.md @@ -132,14 +132,7 @@ args = ["mcp", "serve"] ## Proof -**Savings on real agent workloads:** - -| Workload | Before | After | Savings | -|-------------------------------|-------:|-------:|--------:| -| Code search (100 results) | 17,765 | 1,408 | **92%** | -| SRE incident debugging | 65,694 | 5,118 | **92%** | -| GitHub issue triage | 54,174 | 14,761 | **73%** | -| Codebase exploration | 78,502 | 41,254 | **47%** | +See the [benchmark methodology and reproduction instructions](https://headroom-docs.vercel.app/docs/benchmarks), including Compression Performance. **Accuracy preserved on standard benchmarks:** diff --git a/docs/content/docs/index.mdx b/docs/content/docs/index.mdx index 75d3d2a84..a88254e04 100644 --- a/docs/content/docs/index.mdx +++ b/docs/content/docs/index.mdx @@ -72,12 +72,7 @@ Headroom works as a **transparent proxy** (zero code changes), a **Python functi 87.6% fewer tokens. Same answer. The FATAL error was automatically preserved -- not by keyword matching, but by statistical analysis of field variance. -| Scenario | Before | After | Savings | -|---|---|---|---| -| Code search (100 results) | 17,765 | 1,408 | **92%** | -| SRE incident debugging | 65,694 | 5,118 | **92%** | -| Codebase exploration | 78,502 | 41,254 | **47%** | -| GitHub issue triage | 54,174 | 14,761 | **73%** | +See the [benchmark methodology and reproduction instructions](/docs/benchmarks). ## Key Features diff --git a/wiki/index.md b/wiki/index.md index ea294ab48..35bcb334e 100644 --- a/wiki/index.md +++ b/wiki/index.md @@ -288,15 +288,6 @@ Context management is handled automatically inside the pipeline (live-zone-only **87.6% fewer tokens. Same answer.** The FATAL error was automatically preserved — not by keyword matching, but by statistical analysis of field variance. -### Real Workloads - -| Scenario | Before | After | Savings | -|----------|--------|-------|---------| -| Code search (100 results) | 17,765 | 1,408 | **92%** | -| SRE incident debugging | 65,694 | 5,118 | **92%** | -| Codebase exploration | 78,502 | 41,254 | **47%** | -| GitHub issue triage | 54,174 | 14,761 | **73%** | - ### Accuracy Benchmarks | Benchmark | Category | N | Accuracy | Compression |