akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs(benchmarks): remove unreproducible savings claims (#2593)

Browse source
This commit is contained in:
Rod Boev 2026-07-27 09:31:54 -04:00
parent f74d874777
commit c15deee4a1
3 changed files with 2 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

9
README.md
View file

@ -132,14 +132,7 @@ args = ["mcp", "serve"]
## Proof
**Savings on real agent workloads:**
| Workload | Before | After | Savings |
|-------------------------------|-------:|-------:|--------:|
| Code search (100 results) | 17,765 | 1,408 | **92%** |
| SRE incident debugging | 65,694 | 5,118 | **92%** |
| GitHub issue triage | 54,174 | 14,761 | **73%** |
| Codebase exploration | 78,502 | 41,254 | **47%** |
See the [benchmark methodology and reproduction instructions](https://headroom-docs.vercel.app/docs/benchmarks), including Compression Performance.
**Accuracy preserved on standard benchmarks:**

7
docs/content/docs/index.mdx
View file

@ -72,12 +72,7 @@ Headroom works as a **transparent proxy** (zero code changes), a **Python functi
87.6% fewer tokens. Same answer. The FATAL error was automatically preserved -- not by keyword matching, but by statistical analysis of field variance.
| Scenario | Before | After | Savings |
|---|---|---|---|
| Code search (100 results) | 17,765 | 1,408 | **92%** |
| SRE incident debugging | 65,694 | 5,118 | **92%** |
| Codebase exploration | 78,502 | 41,254 | **47%** |
| GitHub issue triage | 54,174 | 14,761 | **73%** |
See the [benchmark methodology and reproduction instructions](/docs/benchmarks).
## Key Features

9
wiki/index.md
View file

@ -288,15 +288,6 @@ Context management is handled automatically inside the pipeline (live-zone-only
**87.6% fewer tokens. Same answer.** The FATAL error was automatically preserved — not by keyword matching, but by statistical analysis of field variance.
### Real Workloads
| Scenario | Before | After | Savings |
|----------|--------|-------|---------|
| Code search (100 results) | 17,765 | 1,408 | **92%** |
| SRE incident debugging | 65,694 | 5,118 | **92%** |
| Codebase exploration | 78,502 | 41,254 | **47%** |
| GitHub issue triage | 54,174 | 14,761 | **73%** |
### Accuracy Benchmarks
| Benchmark | Category | N | Accuracy | Compression |