mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Shreyas S K
|
46dede36f9
|
fix(pricing): resolve MiniMax-M3 (provider prefix + pre-registration) (#1186)
## Description
Fixes the cost dashboard reporting `$0.00` for every call when the
upstream model is `MiniMax-M3` (Anthropic-compatible endpoint served
from the `MiniMax` provider).
Two root causes in `headroom/pricing/litellm_pricing.py`:
1. **`resolve_litellm_model()` had no `minimax/` provider prefix.**
LiteLLM's community pricing database stores MiniMax-M3 only under
`minimax/MiniMax-M3`. The resolver never tried that prefix, so callers
in `proxy/cost.py`, `proxy/savings_tracker.py`, and `perf/analyzer.py`
silently fell back to the unresolved name.
2. **The prefix check was case-sensitive.** MiniMax's model name uses
mixed case (`MiniMax-M3`), but every existing prefix pattern (`claude-`,
`gpt-`, `o1-`, …) was lowercase, so even after adding `"minimax-"` the
bare `MiniMax-M3` wouldn't match.
This PR fixes both, plus adds a `_register_minimax_pricing()` helper
that pre-populates `litellm.model_cost["MiniMax-M3"]` from
`minimax/MiniMax-M3` at module load — a safety net so `estimate_cost()`
(which doesn't know the `minimax/` prefix internally) succeeds even on a
cold resolver cache or if LiteLLM drops the prefixed entry in a future
release.
Net change: **+97 / −1 lines across 2 files** (one production file + one
test file).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- **Production** (`headroom/pricing/litellm_pricing.py`):
- Add `"minimax-": "minimax/"` to the provider-prefix table in
`_resolve_litellm_model_uncached()` so the resolver knows about the
MiniMax provider.
- Compute `model_lower = model.lower()` and match prefixes against it
instead of `model`, so the mixed-case bare name `MiniMax-M3` resolves
correctly. The existing prefixes (`claude-`, `gpt-`, `o1-`, `o3-`,
`o4-`, `gemini-`) are already lowercase patterns matched against
canonical lowercase names (`claude-sonnet-4-5-…`, `gpt-4o`,
`gemini-2.0-flash`) — no regression.
- Add `_register_minimax_pricing()`: if `minimax/MiniMax-M3` is in
`litellm.model_cost` and `MiniMax-M3` is not, copy the pricing dict
under the bare key. No-op on older LiteLLM (entry missing) or when the
user has already customised `MiniMax-M3`.
- Invoke `_register_minimax_pricing()` once at module import.
- **Tests** (`tests/test_pricing_litellm.py`):
- Add `test_litellm_minimax_mixed_case_with_provider_prefix` — verifies
`resolve_litellm_model("MiniMax-M3")` returns `"minimax/MiniMax-M3"` via
the case-insensitive prefix match.
- Add `test_litellm_minimax_preregistration_safety_net` — verifies the
pre-registration populates the bare `MiniMax-M3` key, that
`estimate_cost()` returns the correct dollar figure (`0.84` for 1M in +
100k out), and that a user-customised bare entry is never clobbered.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_pricing_litellm.py -v
============================= test session starts ==============================
platform darwin -- Python 3.11.15, pytest-9.0.3, pluggy-1.6.0
configfile: pyproject.toml
plugins: anyio-4.12.1, langsmith-0.8.0, asyncio-1.3.0, cov-7.0.0
collected 7 items
tests/test_pricing_litellm.py::test_litellm_helpers_when_dependency_is_unavailable PASSED [ 14%]
tests/test_pricing_litellm.py::test_litellm_model_pricing_exact_match_and_defaults PASSED [ 28%]
tests/test_pricing_litellm.py::test_litellm_model_pricing_uses_provider_prefixes PASSED [ 42%]
tests/test_pricing_litellm.py::test_litellm_model_pricing_uses_aliases_and_zero_cost_defaults PASSED [ 57%]
tests/test_pricing_litellm.py::test_litellm_model_pricing_returns_none_for_unknown_models PASSED [ 71%]
tests/test_pricing_litellm.py::test_litellm_minimax_mixed_case_with_provider_prefix PASSED [ 85%]
tests/test_pricing_litellm.py::test_litellm_minimax_preregistration_safety_net PASSED [100%]
============================== 7 passed in 1.07s ===============================
$ uv run ruff check headroom/pricing/litellm_pricing.py tests/test_pricing_litellm.py
All checks passed!
$ uv run mypy headroom/pricing/litellm_pricing.py
Success: no issues found in 1 source file
```
Manual reproducer (matches the PR writeup):
```text
$ uv run python -c "
from headroom.pricing.litellm_pricing import resolve_litellm_model, estimate_cost
import litellm
print('resolve_litellm_model(MiniMax-M3):', resolve_litellm_model('MiniMax-M3'))
print('MiniMax-M3 in litellm.model_cost :', 'MiniMax-M3' in litellm.model_cost)
print('estimate_cost (1M in, 100k out): ', estimate_cost('MiniMax-M3', 1_000_000, 100_000))
"
resolve_litellm_model(MiniMax-M3): minimax/MiniMax-M3
MiniMax-M3 in litellm.model_cost : True
estimate_cost (1M in, 100k out): 0.84
```
## Real Behavior Proof
- Environment: macOS Darwin 25.5.0, Python 3.11.15, `headroom-ai`
installed editable via `uv` from this branch, `litellm` pulled from PyPI
on first run.
- Exact command / steps: after `git checkout fix/minimax-pricing && uv
sync --all-extras --dev`, run (1) `uv run python -c "from
headroom.pricing.litellm_pricing import resolve_litellm_model,
estimate_cost; import litellm;
print(resolve_litellm_model('MiniMax-M3'), 'MiniMax-M3' in
litellm.model_cost, estimate_cost('MiniMax-M3', 1_000_000, 100_000))"`,
then (2) `uv run pytest tests/test_pricing_litellm.py -v`, then (3) `uv
run ruff check headroom/pricing/litellm_pricing.py
tests/test_pricing_litellm.py`, then (4) `uv run mypy
headroom/pricing/litellm_pricing.py`.
- Observed result: (1) `resolve_litellm_model('MiniMax-M3')` returns
`minimax/MiniMax-M3` (was `MiniMax-M3`, unresolved); `'MiniMax-M3' in
litellm.model_cost` is `True` (proves `_register_minimax_pricing()`
ran); `estimate_cost('MiniMax-M3', 1_000_000, 100_000)` returns `0.84`
(matches `$0.60/M in × 1M + $2.40/M out × 0.1M`). (2) All 7 tests in
`tests/test_pricing_litellm.py` pass (5 pre-existing + 2 new
MiniMax-specific). (3) `ruff` reports `All checks passed!`. (4) `mypy`
reports `Success: no issues found in 1 source file`.
- Not tested: end-to-end through the running proxy against a live
`MiniMax-M3` endpoint — no Anthropic-compatible key configured in this
environment. The reproducer exercises the exact code path the proxy's
cost accumulator uses, but I did not point the proxy at a real upstream.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation — *N/A: no
user-facing docs reference `litellm_pricing.py` directly; the only
public API affected (`estimate_cost`) now returns correct values for a
previously-unsupported model.*
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable — *N/A: this repo
doesn't appear to use CHANGELOG.md (not present at repo root).*
## Additional Notes
- **Why both fixes are needed.** `estimate_cost()` calls
`get_model_pricing()` directly, and `get_model_pricing()` has its own
hardcoded prefix list `["openai/", "anthropic/", "google/", "mistral/",
"deepseek/"]` that does **not** include `minimax/`. So the prefix
resolver alone is not enough for `estimate_cost("MiniMax-M3")` to return
a non-`None` number — the pre-registration step is what makes the bare
name resolve. The prefix resolver change matters for the proxy's
cost/savings/perf code paths that call `resolve_litellm_model()` and
then look up the prefixed string themselves.
- **Why case-insensitive matching is safe.** All existing prefixes are
lowercase patterns matched against already-lowercase canonical model
names — lower-casing before `startswith()` is a no-op for them. Only the
new `"minimax-"` entry uses a mixed-case input.
- **Pricing drift note.** `_register_minimax_pricing()` mirrors upstream
LiteLLM (input $0.60/M, output $2.40/M, cache read $0.12/M as of
2026-06). Re-check after LiteLLM updates; the function already
short-circuits when the user has customised the entry.
- **Did not run** the full test suite, only
`tests/test_pricing_litellm.py`. Wider CI will catch anything I missed.
---------
Co-authored-by: Shreyas S K <shreyassk@Shreyass-MacBook-Air.local>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Garm
|
efd2ac1ca4 |
chore: renormalize line endings to LF
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but 74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings, violating that contract. Every macOS/Linux clone reports these files as "modified" on fresh checkout because git's diff engine sees the stored bytes don't match the attribute contract, even though the working tree and index match byte-for-byte. Running `git add --renormalize .` rewrites each affected blob so the stored form matches the attribute declaration. No semantic changes — every affected file's diff is "N insertions, N deletions" with inserts and deletes being the same lines modulo line endings. Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub blame skip this mechanical commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JerrettDavis
|
38bf3e639c |
test: expand coverage across helper slices
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |