mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
fix(cli/proxy): preserve explicit HEADROOM_MIN_TOKENS=0 / MAX_ITEMS=0 (#1886)
## Description
The Click `proxy` command builds two `ProxyConfig` fields like this:
```python
min_tokens_to_crush=_get_env_int_optional("HEADROOM_MIN_TOKENS") or 500,
max_items_after_crush=_get_env_int_optional("HEADROOM_MAX_ITEMS") or 50,
```
`_get_env_int_optional` correctly returns `0` for
`HEADROOM_MIN_TOKENS=0`, but
the trailing `or 500` treats that legitimate `0` as falsy and replaces
it with
the default. `0` is a meaningful setting — `smart_crusher` gates on
`if tokens > self.config.min_tokens_to_crush`, so
`min_tokens_to_crush=0` means
"crush every item with any tokens." The user asking for `0` silently
gets `500`
instead (and `HEADROOM_MAX_ITEMS=0` → `50`).
This is provably unintended: the argparse `headroom proxy` path sets the
**same**
fields via `_get_env_int("HEADROOM_MIN_TOKENS", args.min_tokens)`, a
helper that
preserves `0` — so the two entry points disagree on the identical env
var. And
the adjacent
`protect_recent=_get_env_int_optional("HEADROOM_PROTECT_RECENT")`
line deliberately avoids `or`, showing the distinction was understood.
Closes: no issue filed — found while auditing env-var → config parsing.
## Fix
Add a `_get_env_int(name, default)` helper (mirroring
`headroom.proxy.server._get_env_int`)
that substitutes the default only when the var is unset/empty, and use
it for
both fields:
```python
def _get_env_int(name: str, default: int) -> int:
value = _get_env_int_optional(name)
return default if value is None else value
...
min_tokens_to_crush=_get_env_int("HEADROOM_MIN_TOKENS", 500),
max_items_after_crush=_get_env_int("HEADROOM_MAX_ITEMS", 50),
```
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/cli/proxy.py`: add `_get_env_int(name, default)` and use it
for `min_tokens_to_crush` / `max_items_after_crush` instead of `... or
<default>`.
- `tests/test_cli_proxy_env.py`: regression test asserting
`HEADROOM_MIN_TOKENS=0` / `HEADROOM_MAX_ITEMS=0` reach `ProxyConfig` as
`0`.
- `CHANGELOG.md`: Bug Fixes entry under Unreleased.
## Testing
- [x] New regression test added (`tests/test_cli_proxy_env.py`)
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`)
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uv run ruff check headroom/cli/proxy.py tests/test_cli_proxy_env.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, headroom from this branch.
Importing `headroom` loads the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the helper logic
with a dependency-free script and left the full pytest to CI.
- Exact command / steps: replicated `_get_env_int_optional` + the new
`_get_env_int` in a standalone script (only stdlib) and ran the env
values `"0"`, `"120"`, unset, and empty through both the old `or 500`
expression and the new helper.
- Observed result: `"0"` now yields `0` (the old `or 500` gave `500`),
`"120"` → `120`, unset/empty → the default:
```text
OK: '0' -> 0 (old `or 500` gave 500)
OK: '120' -> 120
OK: unset -> 500 default
OK: empty -> 500 default
ENV-INT LOGIC VERIFIED
```
- Not tested: booting the full proxy with `HEADROOM_MIN_TOKENS=0`
end-to-end (needs the heavy stack); the value now flows through as `0`
and the regression test exercises the whole `proxy` command with
`run_server` mocked. Full local `pytest` deferred to CI (OOM, per
above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- No new dependencies; a small helper plus two call-site swaps and a
test.
- @JerrettDavis tagging you — tiny, contained parity fix with the
argparse path if you have a moment.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
parent
bb112dd176
commit
3a33af1af3
3 changed files with 35 additions and 2 deletions
|
|
@ -48,6 +48,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||||
* **wrap/codex:** `headroom unwrap codex` now removes the Headroom rtk instruction block from the Codex global `AGENTS.md`. `wrap codex` injects it there, but unwrap only restored `config.toml` and MCP state, so a plain `codex` launch kept following the "prefix shell commands with `rtk`" guidance and failed once the managed rtk binary was off PATH. Unwrap now strips the marker-fenced block (preserving the rest of the file), mirroring `unwrap copilot` ([#1421](https://github.com/headroomlabs-ai/headroom/issues/1421)).
|
* **wrap/codex:** `headroom unwrap codex` now removes the Headroom rtk instruction block from the Codex global `AGENTS.md`. `wrap codex` injects it there, but unwrap only restored `config.toml` and MCP state, so a plain `codex` launch kept following the "prefix shell commands with `rtk`" guidance and failed once the managed rtk binary was off PATH. Unwrap now strips the marker-fenced block (preserving the rest of the file), mirroring `unwrap copilot` ([#1421](https://github.com/headroomlabs-ai/headroom/issues/1421)).
|
||||||
* **proxy/auth:** classify real Anthropic OAuth tokens correctly. `classify_auth_mode` matched OAuth on the `sk-ant-oat-` prefix, but real access tokens are `sk-ant-oat01-...` (a version number, no dash after `oat`), so every real subscription/OAuth token fell through to the `sk-` branch and was tagged `PAYG` — enabling aggressive lossy compression, auto `cache_control`, and `prompt_cache_key` injection on subscription-bound requests the classifier is meant to route to the passthrough-prefer path. The prefix is now the dash-less `sk-ant-oat` (still matches the legacy dashed shape). The existing parity tests only passed because they used a synthetic `sk-ant-oat-01-` fixture; a regression test now covers the real `sk-ant-oat01-` format.
|
* **proxy/auth:** classify real Anthropic OAuth tokens correctly. `classify_auth_mode` matched OAuth on the `sk-ant-oat-` prefix, but real access tokens are `sk-ant-oat01-...` (a version number, no dash after `oat`), so every real subscription/OAuth token fell through to the `sk-` branch and was tagged `PAYG` — enabling aggressive lossy compression, auto `cache_control`, and `prompt_cache_key` injection on subscription-bound requests the classifier is meant to route to the passthrough-prefer path. The prefix is now the dash-less `sk-ant-oat` (still matches the legacy dashed shape). The existing parity tests only passed because they used a synthetic `sk-ant-oat-01-` fixture; a regression test now covers the real `sk-ant-oat01-` format.
|
||||||
* **install:** stop leaking a file descriptor on every `headroom install start`. `start_detached_agent()` opened the agent log file and handed it to `subprocess.Popen` but never closed the parent's copy, so each call leaked one fd (and pinned the log file open against rotation). The parent now closes its copy in a `try/finally` once the child has inherited it — the close also runs if `Popen` raises ([#1554](https://github.com/headroomlabs-ai/headroom/issues/1554)).
|
* **install:** stop leaking a file descriptor on every `headroom install start`. `start_detached_agent()` opened the agent log file and handed it to `subprocess.Popen` but never closed the parent's copy, so each call leaked one fd (and pinned the log file open against rotation). The parent now closes its copy in a `try/finally` once the child has inherited it — the close also runs if `Popen` raises ([#1554](https://github.com/headroomlabs-ai/headroom/issues/1554)).
|
||||||
|
* **cli/proxy:** honor `HEADROOM_MIN_TOKENS=0` / `HEADROOM_MAX_ITEMS=0`. The Click `proxy` command built these with `_get_env_int_optional(name) or 500`/`or 50`, so an explicit `0` — a legitimate value (`min_tokens_to_crush=0` means "crush every item") — was treated as falsy and silently replaced with the default. The `headroom proxy` argparse path already preserved `0` via `_get_env_int`, so the two entry points disagreed. The Click path now uses the same None-checking helper.
|
||||||
* **proxy:** include the system prompt, tools, and the response-shaping request fields in the SemanticCache key. `_compute_key` hashed only `{model, messages}`, so two non-streaming requests with identical messages but a different top-level `system` prompt, tool set, sampling config, or output-shaping field collided on one key and the second caller was served the first's cached response — generated under different request semantics, in the default config (`cache_enabled` defaults on). The key now folds the request fields that shape generation — `temperature`/`top_p`/`top_k`/`max_tokens`/`stop`, plus OpenAI `tool_choice`/`response_format`/`parallel_tool_calls`/`seed`/`presence_penalty`/`frequency_penalty`/`logit_bias`/`n`/`logprobs`/`top_logprobs`/`reasoning_effort`/`verbosity`/`modalities` and Anthropic `thinking`/`tool_choice`/`output_config` — canonicalizing `system`/`tools` so a moved `cache_control` breakpoint does not fragment it, and the handlers snapshot the fields once at the cache read and reuse them at write so a body mutated by the pipeline cannot diverge the key. Non-streaming path only.
|
* **proxy:** include the system prompt, tools, and the response-shaping request fields in the SemanticCache key. `_compute_key` hashed only `{model, messages}`, so two non-streaming requests with identical messages but a different top-level `system` prompt, tool set, sampling config, or output-shaping field collided on one key and the second caller was served the first's cached response — generated under different request semantics, in the default config (`cache_enabled` defaults on). The key now folds the request fields that shape generation — `temperature`/`top_p`/`top_k`/`max_tokens`/`stop`, plus OpenAI `tool_choice`/`response_format`/`parallel_tool_calls`/`seed`/`presence_penalty`/`frequency_penalty`/`logit_bias`/`n`/`logprobs`/`top_logprobs`/`reasoning_effort`/`verbosity`/`modalities` and Anthropic `thinking`/`tool_choice`/`output_config` — canonicalizing `system`/`tools` so a moved `cache_control` breakpoint does not fragment it, and the handlers snapshot the fields once at the cache read and reuse them at write so a body mutated by the pipeline cannot diverge the key. Non-streaming path only.
|
||||||
* **learn (verbosity):** `--verbosity --apply --all` now aggregates the savings baseline across every project instead of overwriting it per project (last-project-wins), which previously left the output shaper with a tiny, unrepresentative baseline. The applied verbosity level comes from the project with the most samples ([#1288](https://github.com/headroomlabs-ai/headroom/pull/1288)).
|
* **learn (verbosity):** `--verbosity --apply --all` now aggregates the savings baseline across every project instead of overwriting it per project (last-project-wins), which previously left the output shaper with a tiny, unrepresentative baseline. The applied verbosity level comes from the project with the most samples ([#1288](https://github.com/headroomlabs-ai/headroom/pull/1288)).
|
||||||
* **proxy/anthropic:** restore token-mode compression on continued Claude Code turns with a frozen prefix and deferred CCR tool injection. Token mode now runs request-side compression even when the client did not pre-register `headroom_retrieve`, relying on the existing marker-triggered injection override to keep emitted CCR markers redeemable ([#1487](https://github.com/headroomlabs-ai/headroom/issues/1487)).
|
* **proxy/anthropic:** restore token-mode compression on continued Claude Code turns with a frozen prefix and deferred CCR tool injection. Token mode now runs request-side compression even when the client did not pre-register `headroom_retrieve`, relying on the existing marker-triggered injection override to keep emitted CCR markers redeemable ([#1487](https://github.com/headroomlabs-ai/headroom/issues/1487)).
|
||||||
|
|
|
||||||
|
|
@ -91,6 +91,18 @@ def _get_env_int_optional(name: str) -> int | None:
|
||||||
raise click.ClickException(f"{name} must be an integer, got {val!r}") from None
|
raise click.ClickException(f"{name} must be an integer, got {val!r}") from None
|
||||||
|
|
||||||
|
|
||||||
|
def _get_env_int(name: str, default: int) -> int:
|
||||||
|
"""Return the env var as an int, or ``default`` only when it is unset.
|
||||||
|
|
||||||
|
Unlike ``_get_env_int_optional(name) or default``, an explicit ``0`` is
|
||||||
|
preserved — ``0`` is a legitimate value (e.g. ``HEADROOM_MIN_TOKENS=0``
|
||||||
|
means "crush every item") and ``0 or default`` would silently discard it.
|
||||||
|
Mirrors ``headroom.proxy.server._get_env_int``.
|
||||||
|
"""
|
||||||
|
value = _get_env_int_optional(name)
|
||||||
|
return default if value is None else value
|
||||||
|
|
||||||
|
|
||||||
def _get_env_float_optional(name: str) -> float | None:
|
def _get_env_float_optional(name: str) -> float | None:
|
||||||
val = os.environ.get(name)
|
val = os.environ.get(name)
|
||||||
if val is None or val == "":
|
if val is None or val == "":
|
||||||
|
|
@ -1082,8 +1094,8 @@ def proxy(
|
||||||
rate_limit_requests_per_minute=rpm if rpm is not None else 60,
|
rate_limit_requests_per_minute=rpm if rpm is not None else 60,
|
||||||
rate_limit_tokens_per_minute=tpm if tpm is not None else 100_000,
|
rate_limit_tokens_per_minute=tpm if tpm is not None else 100_000,
|
||||||
compress_user_messages=_get_env_bool("HEADROOM_COMPRESS_USER_MESSAGES", False),
|
compress_user_messages=_get_env_bool("HEADROOM_COMPRESS_USER_MESSAGES", False),
|
||||||
min_tokens_to_crush=_get_env_int_optional("HEADROOM_MIN_TOKENS") or 500,
|
min_tokens_to_crush=_get_env_int("HEADROOM_MIN_TOKENS", 500),
|
||||||
max_items_after_crush=_get_env_int_optional("HEADROOM_MAX_ITEMS") or 50,
|
max_items_after_crush=_get_env_int("HEADROOM_MAX_ITEMS", 50),
|
||||||
exclude_tools=_parse_exclude_tools(None) or None,
|
exclude_tools=_parse_exclude_tools(None) or None,
|
||||||
protect_tool_results=frozenset(_parse_csv_tools(protect_tool_results))
|
protect_tool_results=frozenset(_parse_csv_tools(protect_tool_results))
|
||||||
if protect_tool_results
|
if protect_tool_results
|
||||||
|
|
|
||||||
|
|
@ -254,6 +254,26 @@ class TestCLIProxyEnvVars:
|
||||||
assert result.exit_code == 0, result.output
|
assert result.exit_code == 0, result.output
|
||||||
assert captured_config["config"].min_tokens_to_crush == 120
|
assert captured_config["config"].min_tokens_to_crush == 120
|
||||||
|
|
||||||
|
def test_headroom_min_tokens_zero_is_preserved(self, runner):
|
||||||
|
"""HEADROOM_MIN_TOKENS=0 is a legitimate value ("crush everything") and
|
||||||
|
must not be discarded by an `or 500` fallback (regression)."""
|
||||||
|
captured_config = {}
|
||||||
|
|
||||||
|
def mock_run_server(config, **kwargs):
|
||||||
|
captured_config["config"] = config
|
||||||
|
|
||||||
|
with patch("headroom.proxy.server.run_server", mock_run_server):
|
||||||
|
result = runner.invoke(
|
||||||
|
main,
|
||||||
|
["proxy"],
|
||||||
|
env={"HEADROOM_MIN_TOKENS": "0", "HEADROOM_MAX_ITEMS": "0"},
|
||||||
|
catch_exceptions=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert captured_config["config"].min_tokens_to_crush == 0
|
||||||
|
assert captured_config["config"].max_items_after_crush == 0
|
||||||
|
|
||||||
def test_headroom_budget_from_env(self, runner):
|
def test_headroom_budget_from_env(self, runner):
|
||||||
"""HEADROOM_BUDGET env var should be passed to ProxyConfig."""
|
"""HEADROOM_BUDGET env var should be passed to ProxyConfig."""
|
||||||
captured_config = {}
|
captured_config = {}
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue