# The bug
Several test modules and two production modules loaded the project `.env`
at *import time*. During pytest collection (where every test module is
imported once), this populated `os.environ` with API keys from `.env`.
The skipif guards in `test_proxy_passthrough_integration.py` (and
others) evaluate at collection time:
@pytest.mark.skipif(not os.environ.get("OPENAI_API_KEY"), reason="...")
If the polluter module was collected *before* the guard, the guard saw
the leaked key, decided not to skip, and the integration tests ran
live against a fake key and failed. In a fresh local-dev venv with
`.env` + full `[dev]` extras, this manifested as ~16 spurious test
failures plus a misleading test runtime of 6+ minutes (live HTTP).
# Why now
CI does not see this (no `.env`). It only manifests when:
1. `litellm` (and friends) are installed — they run `dotenv.load_dotenv()`
on import, populating `os.environ` from `.env`.
2. A `.env` file with real API keys exists locally.
Until the venv was provisioned with the full `[dev]` extras during
recent test work, `pytest.importorskip("litellm")` and
`from headroom.pricing import litellm_pricing` both silently no-op'd
(via try/except ImportError → `LITELLM_AVAILABLE=False`), so the leak
never triggered. With litellm now installed, the latent bug surfaced.
# The fix — three patterns
1. **Production modules** (`headroom/pricing/litellm_pricing.py`,
`headroom/backends/litellm.py`): wrap the eager `import litellm` with
a snapshot/restore of `os.environ`. Any keys litellm's bundled
`python-dotenv` adds during import are deleted immediately. The
module is fully imported and cached in `sys.modules` so subsequent
imports hit the cache without re-running the side effect.
2. **Test modules using `pytest.importorskip("litellm")`**
(`test_backend_bugs.py`, `test_bedrock_region.py`,
`test_cost_tracker_counterfactual.py`): replace with
`tests._dotenv.importorskip_no_env_leak("litellm")`, which does the
same snapshot/restore around `importlib.import_module`.
3. **Test modules that intentionally need `.env` values for skipif
guards** (`test_compression_summary_*.py`, `test_query_echo.py`,
`test_cost_tracker_counterfactual.py`, `test_memory_usage_integration.py`,
`test_bundled_tools_savings.py`): replace module-level
`os.environ.setdefault(...)` / `dotenv.load_dotenv()` with
`tests._dotenv.load_env_overrides()` (returns a local dict — does
NOT mutate `os.environ`) plus `autouse_apply_env(...)` (function-
scoped fixture that applies via `monkeypatch.setenv`, auto-cleaned
at teardown). The skipif still works because
`ANTHROPIC_KEY = os.environ.get(...) or _env_overrides.get(...)`
reads from the local dict as fallback.
# Helper module
New `tests/_dotenv.py` exposes:
- `load_env_overrides() -> dict[str, str]` — read `.env` into a dict.
- `autouse_apply_env(overrides) -> fixture` — function-scoped autouse
fixture that applies via `monkeypatch.setenv`.
- `importorskip_no_env_leak(module) -> module` — drop-in
`pytest.importorskip` substitute that quarantines env mutations.
# Results
Local full-suite (excluding live-LLM and live-feed tests):
- Before: 46 failed, 4830 passed, 387s
- After: 2 failed, 4672 passed, 134s
The remaining 2 failures are unrelated environment-dependent tests
(missing `PIL` / Docker daemon).
The 0.5.18 refactor added api_key forwarding from request headers to
LiteLLM kwargs in all 4 handler methods. This breaks Bedrock (AWS SigV4)
and Vertex AI (Google ADC) which authenticate via env vars, not API keys.
Forwarding a dummy key like sk-ant-dummy overrides AWS credentials.
Fix: skip api_key forwarding for bedrock, vertex_ai, vertex_ai_beta,
and sagemaker providers. Applied to all 4 occurrences.
Bedrock requires role=tool messages immediately after assistant tool_calls.
The previous fix inserted a user text message in between when the message
contained both text and tool_result blocks, breaking the pairing.
Drop text alongside tool_result (Claude Code never sends it in practice).
Added ordering regression tests for the Bedrock constraint.
Fix OpenAI streaming with backends and /v1 double-path bug
Add stream_openai_message() to LiteLLM and any-llm backends so
/v1/chat/completions with stream:true returns SSE events instead
of a JSON blob. Clients (Kilo Code, Cursor, etc.) were hanging
because the proxy ignored the stream flag when routing through
a backend.
Also strip trailing /v1 from OPENAI_TARGET_API_URL to prevent
double-path URLs like /v1/v1/models.
- CLI now reads OPENAI_TARGET_API_URL, GEMINI_TARGET_API_URL, and
HEADROOM_ANYLLM_PROVIDER environment variables
- Add --openai-api-url and --gemini-api-url CLI flags
- Remove --backend choices restriction so litellm-* backends work
- Forward tools/tool_choice through LiteLLM and any-llm backends
- Parse tool call arguments from JSON string to dict (Anthropic format)
- Forward top_p, stop_sequences, tools in streaming paths
- Update Vertex AI model map with Claude 3 through 4.6 (from official docs)
- Bump version to 0.4.1
