fix: Wave 3 — multi-turn live integration tests for A+B realignment

Adds tests/test_realignment_live_multi_turn.py with 9 OPT-IN live tests
that validate the load-bearing claims of the Phase A+B megamerge against
real upstream APIs (Anthropic, OpenAI, Gemini). Each test maps to one or
more realignment PRs:

  1. test_anthropic_cache_hit_across_two_turns          — A2/A6/E
       Identical cache_control'd system+messages on two turns must
       eventually produce cache_read_input_tokens > 0. Guards the cache
       hot zone invariant (I2): proxy must not mutate frozen prefix bytes.
       Uses a bounded retry loop (max 4 attempts) to absorb Anthropic's
       eventually-consistent prompt-cache write latency without masking
       a real "proxy broke cache stability" regression.

  2. test_anthropic_cache_stable_when_live_zone_compresses — B2/B3
       Turn 2 mutates only the LATEST user content (8KB+ JSON tail);
       cache_read on turn 2 must still be > 0 AND the proxy must emit
       compression headers — proving the live-zone block dispatcher
       ran on the new tail without disturbing the cached prefix.

  3. test_anthropic_cache_control_passthrough_byte_faithful — A3/A4
       Wraps proxy._retry_request to snapshot the upstream-bound body
       and assert cache_control on system blocks survives verbatim,
       and user content is not flattened from list to string form.

  4. test_openai_chat_completions_multi_turn_through_proxy — A8/B
       Three-turn conversation through /v1/chat/completions; each
       turn returns valid content, prior assistant turns survive in
       the messages list (proxy doesn't drop them).

  5. test_openai_streaming_sse_chunks_arrive_in_order   — A8 (SSE wire)
       Streams /v1/chat/completions; asserts each event is
       'data: ...\\n\\n', terminator is 'data: [DONE]\\n\\n',
       reassembled content non-empty, no malformed events.

  6. test_gemini_multi_turn_through_proxy               — Gemini reach
       Two-turn conversation through native
       /v1beta/models/{model}:generateContent. Proves Gemini handler
       wiring stayed intact through the megamerge.

  7. test_ccr_marker_round_trip_live                    — B7 (CCR)
       Pre-populates compression_store with a fixture entry, embeds
       a CCR marker on a tool_result, verifies (a) headroom_retrieve
       tool is injected into the upstream tools array (PR-B7
       always-on), and (b) /v1/retrieve returns the original bytes
       by hash with all rows intact. Pre-populating the Python store
       (vs. driving SmartCrusher's internal Rust store) matches the
       established pattern in tests/test_proxy_ccr.py and exercises
       the surface served by /v1/retrieve.

  8. test_memory_tail_injection_does_not_modify_system_prompt_live — B6/A2
       Spins up a memory-enabled proxy with MemoryMode.AUTO_TAIL,
       seeds LocalBackend, captures upstream-bound body. Asserts:
       (a) system prompt byte-identical to input; (b) memory text
       lands on latest user message tail; (c) earlier messages
       untouched. Guards the live-zone-only injection contract.

  9. test_classify_auth_mode_routes_payg_vs_oauth       — Phase F-prep / B5
       NOT a live API call. Sends three header shapes through the
       proxy (x-api-key=..., Bearer sk-ant-oat01-..., Bearer
       sk-ant-api03-...), captures dispatcher headers via a wrap on
       _retry_request, and asserts the canonical auth-mode classifier
       maps each correctly. Codifies the Phase F contract.

Conventions:

  * file-level pytestmark = pytest.mark.live → excluded by default
    via 'pytest -m "not live"'. Adds a 'live' marker registration in
    pyproject.toml's [tool.pytest.ini_options].markers.
  * each test skipif's on the relevant API key — no silent fallbacks,
    no real-API runs against fake keys.
  * uses tests/_dotenv.py helpers (load_env_overrides + autouse_apply_env)
    rather than re-implementing env loading.
  * model IDs and thresholds live in a top-of-file LIVE_CONFIG dict
    (no hardcodes); Anthropic primary/fallback resolves at runtime per
    key entitlement.
  * assertions are direction-only (cache_read > 0, tokens_after <=
    tokens_before) — never tied to upstream pricing/tokenizer drift.
  * shared module-scoped TestClient fixture for performance; CCR and
    memory tests build dedicated proxies for their config-specific paths.

Verification:
  * pytest tests/test_realignment_live_multi_turn.py -v
      → 9 passed, 0 skipped, 0 failed in ~25s (with all keys set)
  * pytest -m "not live" --tb=short -q
      → 4694 passed, 265 skipped, 9 deselected — same baseline as today
  * make ci-precheck → green (rust + python + commitlint)

Per-realignment-plan: REALIGNMENT/04-phase-B-live-zone.md.
This commit is contained in:
chopratejas 2026-05-02 17:38:15 -07:00
parent 2fb905fdb0
commit dcbc921d63
2 changed files with 1141 additions and 0 deletions

View file

@ -318,6 +318,7 @@ asyncio_mode = "auto"
markers = [
"slow: slow tests (model loads, large fixtures)",
"real_llm: tests that hit real LLM APIs; skipped unless explicitly enabled",
"live: opt-in multi-turn tests that hit real upstream APIs; require provider keys",
]
[tool.coverage.run]

File diff suppressed because it is too large Load diff