mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
a14ab45cf0
## Description
`CostTracker._costs` was initialized but never written to, so
`get_period_cost()` always returned `0` and `check_budget()` always
returned "allowed" — the `--budget` flag was a silent no-op.
`_prune_old_costs()` was dead code with zero callers. This makes budget
enforcement actually work: requests are rejected once the configured
limit is reached.
Closes # <!-- no tracked issue; discovered during a proxy-pipeline audit
-->
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **`headroom/proxy/cost.py`** — `record_tokens()` now computes the
request cost via `estimate_cost()` and appends it to `_costs`,
activating `_prune_old_costs()`. When a call site has no API usage
breakdown (cache/uncached all zero), `tokens_sent` is used as the input
count so input cost is not silently dropped. `COST_RETENTION_HOURS` 24 →
744 so retention covers the longest budget period (monthly sums from the
1st; 24h retention would have under-enforced monthly budgets).
- **`headroom/proxy/outcome.py`** — the request funnel passes
`output_tokens` through to `record_tokens()` so costs include output,
for all providers.
- **`headroom/cli/proxy.py`** — added `--budget-period
[hourly|daily|monthly]` (env `HEADROOM_BUDGET_PERIOD`); it existed in
`ProxyConfig` and the server entry point but was unreachable from the
main CLI. Fixed the `--budget` help text that wrongly said "resets at
midnight UTC".
- **`headroom/cli/main.py`** — minor registration/version plumbing.
- Tests: regression coverage for the full `record_tokens →
get_period_cost → check_budget` chain, the `tokens_sent` fallback, and
the `--budget-period` flag/env wiring.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_cost_tracker_counterfactual.py tests/test_request_outcome.py -q
40 passed
$ ruff check headroom/proxy/cost.py headroom/proxy/outcome.py headroom/cli/proxy.py
All checks passed!
$ mypy headroom/proxy/cost.py headroom/proxy/outcome.py headroom/cli/proxy.py --ignore-missing-imports
Success: no issues found
```
## Real Behavior Proof
- Environment: local macOS, Python 3.11, branch `fix/budget-enforcement`
at the PR head commit.
- Exact command / steps: `pytest
tests/test_cost_tracker_counterfactual.py::test_budget_enforced_after_recording_costs
-v` — sets `CostTracker(budget_limit_usd=0.0001)`, records ~$1.50 of
Sonnet input, then asserts `check_budget()` returns not-allowed with
`remaining == 0`.
- Observed result: budget is now enforced — `get_period_cost()` reflects
real spend and `check_budget()` rejects once the limit is exceeded (the
proxy returns HTTP 429 on that path). On `main` the same test fails
because `_costs` is never populated and `check_budget()` always returns
allowed.
- Not tested: live end-to-end rejection against a running proxy with
real upstream traffic; the running proxy needs a restart on this version
to pick up the fix.
```text
$ pytest tests/test_cost_tracker_counterfactual.py::test_budget_enforced_after_recording_costs \
tests/test_cost_tracker_counterfactual.py::test_budget_input_cost_counted_without_usage_breakdown -v
2 passed
```
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A — CLI/backend change with no UI surface. See **Test Output** and
**Real Behavior Proof** above for terminal evidence.
## Additional Notes
- The `ci.yml` coverage-upload change originally added here (commit
`
|
||
|---|---|---|
| .. | ||
| act | ||
| actions/headroom-e2e-setup | ||
| ISSUE_TEMPLATE | ||
| plugin | ||
| workflows | ||
| copilot-instructions.md | ||
| dependabot.yml | ||
| FUNDING.yml | ||
| PULL_REQUEST_TEMPLATE.md | ||