## Description
OpenAI-format `POST /v1/chat/completions` requests routed through
`--backend litellm-vertex` fail when the client includes `max_tokens`.
The proxy currently runs its direct-OpenAI compatibility shim before
backend dispatch, renames `max_tokens` to `max_completion_tokens`, then
the LiteLLM path no longer recognizes that field as standard and sweeps
it into `extra_body`. Vertex rejects the resulting request with
`extra_body: Extra inputs are not permitted`.
This change scopes the rename shim to the direct OpenAI path only.
Backend-routed chat requests now keep `max_tokens`, which LiteLLM
already forwards correctly for the Vertex Anthropic path. Direct GPT-5
and o-series compatibility stays unchanged. Closes#2392.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Thread a backend-owned translation flag into
`_normalize_openai_max_tokens`.
- Skip the legacy-to-completion-token rename on backend-routed OpenAI
chat requests.
- Keep the direct OpenAI compatibility path covered with a backend-owned
translation no-op test.
- Add buffered and streaming handler-level regressions for the exact
`litellm-vertex` request shape, proving the request survives the
`/v1/chat/completions` normalization boundary with vendor fields intact.
## Testing
- [x] Unit tests pass (`uv run pytest
tests/test_proxy/test_openai_backend_path.py
tests/test_openai_streaming_backend.py
tests/test_openai_max_completion_tokens.py
tests/test_litellm_openai_passthrough.py -q`)
- [x] Linting passes (`uv run ruff check
headroom/proxy/handlers/openai.py
tests/test_openai_max_completion_tokens.py
tests/test_litellm_openai_passthrough.py
tests/test_proxy/test_openai_backend_path.py
tests/test_openai_streaming_backend.py`)
- [ ] Type checking passes (`uv run mypy headroom`)
- [x] New tests added for new functionality when applicable
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_proxy/test_openai_backend_path.py tests/test_openai_streaming_backend.py tests/test_openai_max_completion_tokens.py tests/test_litellm_openai_passthrough.py -q
......sss............ [100%]
20 passed, 3 skipped, 1 warning in 42.13s
$ uv run ruff check headroom/proxy/handlers/openai.py tests/test_openai_max_completion_tokens.py tests/test_litellm_openai_passthrough.py tests/test_proxy/test_openai_backend_path.py tests/test_openai_streaming_backend.py
All checks passed!
$ uv run ruff format headroom/proxy/handlers/openai.py tests/test_openai_max_completion_tokens.py tests/test_litellm_openai_passthrough.py tests/test_proxy/test_openai_backend_path.py tests/test_openai_streaming_backend.py --check
5 files already formatted
```
## Real Behavior Proof
- Environment: Windows, synced Headroom development environment, mocked
LiteLLM provider boundary, no paid GCP credentials required
- Exact command / steps: run `uv run pytest
tests/test_proxy/test_openai_backend_path.py
tests/test_openai_streaming_backend.py
tests/test_openai_max_completion_tokens.py
tests/test_litellm_openai_passthrough.py -q`, using the issue payload
shape
`{"model":"claude-sonnet-4-6","max_tokens":32,"messages":[{"role":"user","content":"hi"}],"chat_template_kwargs":{"enable_thinking":false}}`
through `POST /v1/chat/completions`
- Observed result: buffered and streaming `litellm-vertex` requests keep
`max_tokens` as a named backend kwarg, preserve `chat_template_kwargs`
in `extra_body`, omit `max_completion_tokens` from `extra_body`, and
return success through the handler boundary. Direct-path normalization
still renames legacy `max_tokens`.
- Not tested: live Vertex AI request
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [ ] My code follows the project's style guidelines
- [ ] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- `CHANGELOG.md`: N/A, the release pipeline generates it from the
conventional-commit subject.
- Scope is intentionally narrow: this fixes the exact backend-routed
`max_tokens` failure and does not broaden `extra_body` hardening for
unrelated OpenAI fields.
Fix OpenAI streaming with backends and /v1 double-path bug
Add stream_openai_message() to LiteLLM and any-llm backends so
/v1/chat/completions with stream:true returns SSE events instead
of a JSON blob. Clients (Kilo Code, Cursor, etc.) were hanging
because the proxy ignored the stream flag when routing through
a backend.
Also strip trailing /v1 from OPENAI_TARGET_API_URL to prevent
double-path URLs like /v1/v1/models.
