mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
646e705514
## Description
Fixes a dashboard denominator mismatch in the Token Savings card.
The headline was showing the active attempted-token ratio, while the
same card's sublabel reports total-wire savings. This made sessions show
values like about 17% in the headline and about 1.2% in the total-wire
line for the same saved-token count.
This changes the headline to use `stats.tokens.savings_percent`, with
`proxy_savings_percent` as a fallback, so the headline and card copy use
the same denominator.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change fixes issue)
- [ ] New feature (non-breaking change adds functionality)
- [ ] Breaking change (fix or feature cause existing functionality
change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Updated `headlineSavingsPercent` to prefer the total-wire
`savings_percent` metric.
- Updated the headline tooltip to say `Of total wire input tokens`.
- Added a focused dashboard regression test that prevents the headline
getter from using `active_savings_percent` or `proxy_attempted_tokens`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added new functionality
- [x] Manual testing performed
### Test Output
```text
$ python3 - <<'PY'
from pathlib import Path
html = Path('headroom/dashboard/templates/dashboard.html').read_text(encoding='utf-8')
assert 'stats.tokens?.savings_percent' in html
assert 'Of total wire input tokens' in html
start = html.index('get headlineSavingsPercent()')
end = html.index('get headlineSavingsTitle()', start)
headline = html[start:end]
assert 'active_savings_percent' not in headline
assert 'proxy_attempted_tokens' not in headline
print('dashboard headline denominator check passed')
PY
dashboard headline denominator check passed
$ uv run --extra dev pytest tests/test_dashboard_token_savings.py
============================= test session starts ==============================
platform darwin -- Python 3.13.3, pytest-9.0.3, pluggy-1.6.0
collected 1 item
tests/test_dashboard_token_savings.py::test_token_savings_headline_uses_total_wire_denominator PASSED [100%]
============================== 1 passed in 0.10s ===============================
$ uv run --extra dev ruff check tests/test_dashboard_token_savings.py
All checks passed!
```
## Real Behavior Proof
- Environment: Local Headroom dashboard served from the installed 0.28.0
package on macOS, proxy on `127.0.0.1:8788`, checked against the same
dashboard template logic patched in this PR.
- Exact command / steps: Queried local `/stats?cached=1`, compared
`tokens.active_savings_percent` with `tokens.savings_percent`, patched
the dashboard template locally, then refreshed `/dashboard` and
confirmed the served `headlineSavingsPercent` getter reads
`tokens.savings_percent`.
- Observed result: Local stats showed `active_savings_percent` around
16.78 while `tokens.savings_percent`, `tokens.proxy_savings_percent`,
and agent total savings were around 1.24. Before the patch, the
dashboard headline used the 16.78 active value even though the card text
said total wire. After the local template patch, the served dashboard
getter uses the 1.24 total-wire value.
- Not tested: Full cross-browser visual regression; this PR only changes
the Alpine getter denominator and adds a source-level regression test.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows project's style guidelines
- [x] I performed self-review my code
- [x] I commented my code, particularly in hard-to-understand areas
- [ ] I made corresponding changes documentation
- [x] My changes generate no new warnings
- [x] I added tests prove fix is effective or feature works
- [x] New and existing unit tests pass locally my changes
- [ ] I updated CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A.
## Additional Notes
- Documentation and CHANGELOG are N/A for this narrow dashboard bug fix.
- CI is green and the PR is ready for review.
27 lines
962 B
Python
27 lines
962 B
Python
"""Regression tests for dashboard token savings copy."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import re
|
|
|
|
from headroom.dashboard import get_dashboard_html
|
|
|
|
|
|
def _getter_body(html: str, name: str) -> str:
|
|
match = re.search(rf"get {name}\(\) \{{(?P<body>.*?)\n\s*\}},", html, re.S)
|
|
assert match is not None
|
|
return match.group("body")
|
|
|
|
|
|
def test_token_savings_headline_uses_total_wire_denominator() -> None:
|
|
html = get_dashboard_html()
|
|
|
|
headline_body = _getter_body(html, "headlineSavingsPercent")
|
|
assert "stats.tokens?.savings_percent" in headline_body
|
|
assert "stats.tokens?.proxy_savings_percent" in headline_body
|
|
assert "stats.tokens?.active_savings_percent" not in headline_body
|
|
assert "stats.tokens?.proxy_attempted_tokens" not in headline_body
|
|
|
|
title_body = _getter_body(html, "headlineSavingsTitle")
|
|
assert "Of total wire input tokens" in title_body
|
|
assert "Of compressible tokens attempted" not in title_body
|