mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
2 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tejas Chopra
|
96c25f5181
|
fix(cli): stop the macOS malloc re-exec replacing an embedder's process (#3064)
## Description **`main` cannot currently run its own test suite on macOS.** `pytest tests/` dies at roughly 2% with exit code 2 — no traceback, no summary, no failing test named. The pytest process is simply gone. Two independent defects, both landed today, both invisible to CI. ### 1. The macOS malloc re-exec replaces the calling process `headroom proxy` re-execs itself once on Darwin to apply two libmalloc knobs that libmalloc only reads before `main()` (#2820, PR #2879): ```python os.execv(sys.executable, [sys.executable, "-m", "headroom.cli", *sys.argv[1:]]) ``` That reconstruction is only faithful when the process really *is* the Headroom CLI. Ten-plus test files invoke the `proxy` command in-process through Click's `CliRunner`. There, `os.execv` replaces **pytest** with a Headroom process holding pytest's argv. Run with `-s`, the mechanism is visible: ``` tests/test_agent_savings.py Usage: python -m headroom.cli [OPTIONS] COMMAND [ARGS]... Error: No such command 'tests/test_agent_savings.py::test_proxy_cli_reads_agent_90_profile_env'. ``` Everything after the first such test — roughly 98% of the suite — never runs. The same hazard applies to any application embedding the CLI. **The documented kill switch does not help.** `tests/conftest.py:41` scrubs every `HEADROOM_*` variable for hermeticity, so `HEADROOM_MALLOC_TUNING` is deleted before the guard reads it. Only the private `_HEADROOM_MALLOC_TUNED` survives, because it starts with an underscore. **CI could not have caught this.** The tuning is Darwin-only, and while the repo *does* have macOS jobs (`macos-native-wrapper`, `wrap-native (macos-latest)`), neither runs the Python test suite — the `test` shards are `ubuntu-latest` only. So `sys.platform != "darwin"` returns first everywhere pytest actually runs. #2879 merged with 37 green checks. ### 2. A semantic merge conflict between two green PRs #3051 added `bind_scope(tags, request.scope)` at `gemini.py:325` and updated the three Gemini fakes it knew about. #3035 branched earlier and added a fourth `_FakeRequest` without `.scope`. Each was green against its own base; together they fail: ``` AttributeError: '_FakeRequest' object has no attribute 'scope' ``` Git merged both cleanly. Only running the suite on merged `main` surfaces it. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update ## Changes Made - Added `_process_is_headroom_cli_entrypoint()`: the re-exec now verifies its own precondition — `argv[0]` must be the `headroom` console script or `headroom/cli/__main__.py`. - The embedded path returns **before** stamping `_HEADROOM_MALLOC_TUNED`, so a genuine CLI child inheriting the environment can still apply the tuning. - Gave the Gemini `_FakeRequest` the `.scope` every real Starlette `Request` carries. - `test_reexec_skips_when_operator_already_set_vars` now sets a realistic `argv[0]`, matching its sibling exec test. - New `tests/test_cli_proxy_malloc_reexec_guard.py` asserting the guard's logic on **every** platform, since no CI runner is macOS. ## Testing - [x] Unit tests pass - [x] Linting passes (ruff check + format) - [ ] Type checking passes (`uv run mypy headroom`) — not run - [x] New tests added for new functionality ### Test Output Before, on `main`: ```text $ .venv/bin/python -m pytest tests/ -q collected 11622 items / 8 skipped ... tests/test_agent_savings.py ............................ $ echo $? 2 ``` No summary line — the run does not end, it is replaced. After, on this branch: ```text $ .venv/bin/python -m pytest tests/ -q 3 failed, 11055 passed, 581 skipped, 6034 warnings in 303.69s (0:05:03) ``` All three remaining failures reproduce at ` |
||
|
Abhay Singh
|
a01897c791
|
fix(proxy/gemini): guard CCR continuation usage against present-null counts (#3035)
## Description
On the Gemini native `generateContent` path, a successful (200) response
that triggers a CCR retrieval continuation is masked as a synthetic 502
when the continuation response carries a present-null usage count.
`handle_gemini_generate_content` reads `usageMetadata` at three sites.
The initial-response site and the non-CCR site both guard against Gemini
returning a present-null count (a key present with a JSON `null`, which
`.get(key, default)` returns as `None` rather than the default). The
CCR-continuation site read the continuation's `usageMetadata` with a
bare `.get(key, prior)`:
```python
total_input_tokens = usage.get("promptTokenCount", total_input_tokens)
output_tokens = usage.get("candidatesTokenCount", output_tokens)
cache_read_tokens = usage.get("cachedContentTokenCount", cache_read_tokens)
```
When the continuation turn reports `"promptTokenCount": null`,
`total_input_tokens` becomes `None`, and the following
`uncached_input_tokens = max(0, total_input_tokens - cache_read_tokens)`
and the `total_input_tokens > 0` baseline guard raise `TypeError`. The
method's outer `except Exception` then returns a 502 JSONResponse and
records a provider failure, so a genuinely successful upstream turn is
reported to the client as a 502.
## Fix
Read the continuation usage through the same `_usage_int` guard the two
sibling sites use, keeping the pre-continuation count as the fallback
(`_usage_int(value, default)` returns `default` when `value is None`).
Behavior is otherwise unchanged: a present, valid count is still used,
and an absent count still falls back to the pre-continuation value.
Fixes #3034
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/gemini.py` (`handle_gemini_generate_content`,
CCR-continuation branch): read `promptTokenCount` /
`candidatesTokenCount` / `cachedContentTokenCount` through
`_usage_int(..., prior)` instead of a bare `.get(key, prior)`.
- `tests/test_gemini_ccr_continuation_usage.py`: drive the handler
through a CCR continuation whose `usageMetadata` counts are
present-null; assert the client gets 200 (not 502), no provider failure
is recorded, and the pre-continuation count survives as the fallback.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added
### Test Output
```text
tests/test_gemini_ccr_continuation_usage.py 1 passed
tests/test_gemini_nonjson_status.py tests/test_gemini_compression_offload.py tests/test_proxy_gemini_native_integration.py (all pass; platform-skipped cases skipped)
# uvx ruff@0.15.22 check -> All checks passed!
# uvx mypy@1.20.2 headroom/proxy/handlers/gemini.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.22 and mypy 1.20.2 via uvx.
- Exact command / steps: ran `python -m pytest
tests/test_gemini_ccr_continuation_usage.py -q` (pass-after); proved
fail-before by `git stash`-ing only the `gemini.py` change and
re-running (the test failed with `assert 502 == 200` and the captured
log `TypeError: unsupported operand type(s) for -: 'NoneType' and
'NoneType'` at `gemini.py`), then restored the fix and re-ran green; ran
the surrounding Gemini suite (`test_gemini_nonjson_status.py`,
`test_gemini_compression_offload.py`,
`test_proxy_gemini_native_integration.py`); then `uvx ruff@0.15.22
check` and `uvx mypy@1.20.2 headroom/proxy/handlers/gemini.py`.
- Observed result: with the fix a CCR continuation carrying a
present-null `promptTokenCount` returns 200 to the client and records
the outcome with the pre-continuation count (100) instead of raising
`TypeError` and returning a synthetic 502.
- Not tested: a live Gemini session that both triggers a CCR retrieval
continuation and receives a present-null continuation usage payload
(needs a real safety-blocked continuation). The contract is verified at
the handler with the same stub pattern the existing
`test_gemini_nonjson_status.py` uses.
## Runtime Rollout Safety
- Rollout-managed feature(s): none. This is the always-on Gemini native
`generateContent` request path, not a rollout-channel-gated feature.
- Minimum rollout channel: N/A (no rollout-managed behavior).
- Stable/default behavior changed: yes, as a bug fix. A CCR continuation
with a present-null usage count now returns the real 200 instead of a
synthetic 502; all other cases (present valid count, absent count) are
unchanged.
- Kill switch / disable path: N/A. There is no behavioral toggle; the
change only makes the existing continuation path null-safe.
- Unsafe override required: no.
- Qualification impact: brings the CCR-continuation usage extraction to
parity with the two sibling sites that already guard present-null
counts; no routing, compression, or pricing change.
- Rollback path: revert this PR; the continuation site returns to the
bare `.get(key, prior)` read.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title
## Additional Notes
The unguarded site was introduced in #2253 (native CCR retrieval); the
present-null guard on the sibling sites landed separately and did not
extend to it. The fix reuses the existing `_usage_int` helper so all
three Gemini usage-extraction sites now handle present-null identically.
|