headroom/tests/test_openai_beta_session_sticky.py
chopratejas aec5ba3253 fix: A6 — anthropic-beta and openai-beta deterministic merge + session-sticky
PR-A6 of the Phase A cache-safety lockdown. Eliminates P5-50 and preps
P0-6 (memory tool injection toggling).

Two cache-killer patterns the merge + tracker defeat:

  1. Mid-session mutation: when memory was enabled the proxy did an
     ad-hoc concat of `context-management-2025-06-27` onto the client
     value (anthropic.py:1244-1248). The order varied with the client
     value, breaking byte-stable headers across turns.

  2. Token drop-out across turns: clients (Claude Code, Codex CLI) MAY
     drop a beta token between turn N and turn N+1 even when the proxy
     mutated turn N to add it. The cache hot zone is positional, so the
     next turn's prefix bytes hash differently and the prefix-cache
     read misses.

Changes
-------

`headroom/proxy/helpers.py`
  * `merge_anthropic_beta` / `merge_openai_beta`: pure, deterministic,
    order-preserving merge. Client tokens first (in their original
    order), then Headroom-required tokens (in the order passed). Dedupe
    is case-insensitive but preserves the original casing of the first
    occurrence. No regex.
  * `SessionBetaTracker`: bounded LRU keyed by (provider, session_id),
    unioning client tokens with previously-seen tokens. OrderedDict
    LRU; threading.RLock for thread safety (mirrors the
    CompressionCache pattern from compression_cache.py).
  * `get_session_beta_tracker` / `_reset_session_beta_tracker_for_test`
    process-wide singleton with test reset.
  * `log_beta_header_merge`: structured log per cache-affecting merge.
  * Env-var knobs (NO HARDCODES):
    - HEADROOM_BETA_HEADER_STICKY=enabled|disabled (default enabled).
    - HEADROOM_BETA_TRACKER_MAX_SESSIONS (default 1000).

`headroom/proxy/handlers/anthropic.py`
  * After `compute_session_id` (line ~744): record client
    `anthropic-beta` against the session tracker, write the sticky
    value back into `headers` if changed. Order matters: sticky-merge
    FIRST so memory-injection has the canonical baseline.
  * Memory-injection site (line ~1244): replace the ad-hoc concat with
    `merge_anthropic_beta(headers["anthropic-beta"], required_tokens)`.

`headroom/proxy/handlers/openai.py`
  * Chat-completions (line ~360): record/merge `openai-beta`.
  * /v1/responses HTTP (line ~1213): compute `_responses_session_id`
    and record/merge `openai-beta`.
  * /v1/responses WS (line ~1711): replace the ad-hoc absent-only
    inject with `merge_openai_beta(sticky, ["responses_websockets=
    2026-02-06"])`. Replaces any case-variants of the existing key.

Tests
-----

`tests/test_anthropic_beta_session_sticky.py` (26 tests):
  * Pure helper: empty inputs, only-client, only-headroom, ordering,
    dedupe casing, deterministic memory-injection order, no-double-
    inject when token already present.
  * Tracker: sticky-on across turns even when client drops, casing
    preservation, provider namespace independence, LRU eviction at
    max_sessions, env-var validation (loud failures), thread safety
    under 16-thread concurrent access, blank-input rejection.

`tests/test_openai_beta_session_sticky.py` (17 tests):
  * Mirror of the anthropic suite for `OpenAI-Beta`.
  * Plus WS-specific coverage: sticky-then-merge of
    `responses_websockets=2026-02-06` against client baseline.

`tests/test_openai_codex_routing.py`
  * Add `session_tracker_store` stub to `_DummyOpenAIHandler` so the
    routing tests still exercise the responses HTTP handler now that
    it computes a session_id for beta-merge.

Notes
-----

Build constraints honored:
  * Configurable: HEADROOM_BETA_HEADER_STICKY,
    HEADROOM_BETA_TRACKER_MAX_SESSIONS.
  * No regex, no hardcodes (env-var bounds), no fallbacks (disabled
    mode is operator opt-in for diagnostics, loud failures on invalid
    values).
  * Structured tracing log via `log_beta_header_merge`.

Acceptance:
  * 43 new tests pass.
  * `cargo test --workspace` green (no Rust changes).
  * `make ci-precheck` green.
2026-05-02 09:53:37 -07:00

226 lines
7.9 KiB
Python

"""Session-sticky `OpenAI-Beta` tests for PR-A6 (P5-50, preps P0-6).
OpenAI's `OpenAI-Beta` follows the same comma-separated convention as
Anthropic's `anthropic-beta`. The proxy auto-injects
`responses_websockets=2026-02-06` on the WS path when absent
(handlers/openai.py:~1711). PR-A6 routes that injection through
`merge_openai_beta` so the client's tokens are preserved and the
auto-injected token is appended deterministically.
The same `SessionBetaTracker` (provider-aware, keyed by
``(provider, session_id)``) backs both providers — one tracker, two
namespaces.
"""
from __future__ import annotations
import threading
import pytest
from headroom.proxy.helpers import (
SessionBetaTracker,
_reset_session_beta_tracker_for_test,
merge_openai_beta,
)
# ---------------------------------------------------------------------------
# Pure helper: `merge_openai_beta`
# ---------------------------------------------------------------------------
def test_merge_helper_pure_function() -> None:
"""Same inputs → same output, no global state."""
a = merge_openai_beta("a,b", ["c"])
b = merge_openai_beta("a,b", ["c"])
assert a == b == "a,b,c"
def test_merge_helper_empty_inputs_returns_empty_string() -> None:
assert merge_openai_beta(None, []) == ""
assert merge_openai_beta("", []) == ""
def test_merge_helper_only_client() -> None:
assert merge_openai_beta("alpha=1,beta=2", []) == "alpha=1,beta=2"
def test_merge_helper_only_headroom() -> None:
assert merge_openai_beta(None, ["responses_websockets=2026-02-06"]) == (
"responses_websockets=2026-02-06"
)
def test_merge_helper_preserves_client_order_appends_headroom() -> None:
out = merge_openai_beta(
"alpha=1,beta=2",
["responses_websockets=2026-02-06"],
)
assert out == "alpha=1,beta=2,responses_websockets=2026-02-06"
def test_merge_helper_no_double_inject_when_already_present() -> None:
out = merge_openai_beta(
"responses_websockets=2026-02-06,alpha=1",
["responses_websockets=2026-02-06"],
)
assert out == "responses_websockets=2026-02-06,alpha=1"
def test_dedupe_case_insensitive_preserves_first_casing() -> None:
# OpenAI tokens are usually `kebab=value` so casing rarely differs in
# practice, but the helper's contract is provider-agnostic.
assert (
merge_openai_beta("Responses_Websockets=2026-02-06", ["responses_websockets=2026-02-06"])
== "Responses_Websockets=2026-02-06"
)
def test_merge_helper_skips_empty_tokens() -> None:
assert merge_openai_beta("a, ,b", []) == "a,b"
assert merge_openai_beta("a", ["", "b", " "]) == "a,b"
def test_test_memory_injection_appends_deterministic_order() -> None:
"""Auto-injected `responses_websockets` appends AFTER client tokens."""
out = merge_openai_beta("client-token", ["responses_websockets=2026-02-06"])
assert out == "client-token,responses_websockets=2026-02-06"
# ---------------------------------------------------------------------------
# `SessionBetaTracker` — provider="openai"
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
def _isolate_tracker(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("HEADROOM_BETA_HEADER_STICKY", raising=False)
monkeypatch.delenv("HEADROOM_BETA_TRACKER_MAX_SESSIONS", raising=False)
_reset_session_beta_tracker_for_test()
yield
_reset_session_beta_tracker_for_test()
def test_beta_seen_turn_1_present_in_turn_2_even_if_client_drops() -> None:
tracker = SessionBetaTracker(max_sessions=10)
out1 = tracker.record_and_get_sticky_betas(
provider="openai",
session_id="s-1",
client_value="responses_websockets=2026-02-06,extra-beta=1",
)
assert out1 == "responses_websockets=2026-02-06,extra-beta=1"
out2 = tracker.record_and_get_sticky_betas(
provider="openai",
session_id="s-1",
client_value="responses_websockets=2026-02-06",
)
assert out2 == "responses_websockets=2026-02-06,extra-beta=1"
def test_client_value_preserved_when_no_injection() -> None:
tracker = SessionBetaTracker(max_sessions=10)
out = tracker.record_and_get_sticky_betas(
provider="openai", session_id="s-1", client_value="alpha,beta"
)
assert out == "alpha,beta"
def test_lru_eviction_at_max_sessions() -> None:
tracker = SessionBetaTracker(max_sessions=2)
tracker.record_and_get_sticky_betas(provider="openai", session_id="s-1", client_value="a")
tracker.record_and_get_sticky_betas(provider="openai", session_id="s-2", client_value="b")
tracker.record_and_get_sticky_betas(provider="openai", session_id="s-1", client_value=None)
tracker.record_and_get_sticky_betas(provider="openai", session_id="s-3", client_value="c")
# s-2 evicted.
out = tracker.record_and_get_sticky_betas(
provider="openai", session_id="s-2", client_value=None
)
assert out == ""
def test_disabled_mode_passes_through(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("HEADROOM_BETA_HEADER_STICKY", "disabled")
tracker = SessionBetaTracker(max_sessions=10)
out1 = tracker.record_and_get_sticky_betas(
provider="openai", session_id="s-1", client_value="alpha"
)
assert out1 == "alpha"
out2 = tracker.record_and_get_sticky_betas(
provider="openai", session_id="s-1", client_value=None
)
assert out2 == ""
def test_thread_safe_concurrent_access() -> None:
tracker = SessionBetaTracker(max_sessions=10)
n_threads = 16
iterations = 50
errors: list[BaseException] = []
def worker(idx: int) -> None:
try:
for i in range(iterations):
tracker.record_and_get_sticky_betas(
provider="openai",
session_id="shared",
client_value=f"o-t{idx}-i{i}",
)
except BaseException as e: # noqa: BLE001
errors.append(e)
threads = [threading.Thread(target=worker, args=(idx,)) for idx in range(n_threads)]
for t in threads:
t.start()
for t in threads:
t.join()
assert errors == []
final = tracker.record_and_get_sticky_betas(
provider="openai", session_id="shared", client_value=None
)
final_tokens = set(final.split(","))
expected = {f"o-t{idx}-i{i}" for idx in range(n_threads) for i in range(iterations)}
assert expected.issubset(final_tokens)
def test_provider_namespaces_are_independent() -> None:
tracker = SessionBetaTracker(max_sessions=10)
tracker.record_and_get_sticky_betas(
provider="openai", session_id="shared", client_value="o-token"
)
tracker.record_and_get_sticky_betas(
provider="anthropic", session_id="shared", client_value="a-token"
)
out_openai = tracker.record_and_get_sticky_betas(
provider="openai", session_id="shared", client_value=None
)
out_anth = tracker.record_and_get_sticky_betas(
provider="anthropic", session_id="shared", client_value=None
)
assert out_openai == "o-token"
assert out_anth == "a-token"
def test_ws_required_token_appended_deterministically() -> None:
"""Mirrors the WS handler logic — record client value, then merge required."""
tracker = SessionBetaTracker(max_sessions=10)
sticky = tracker.record_and_get_sticky_betas(
provider="openai",
session_id="ws-1",
client_value="custom-beta=1",
)
merged = merge_openai_beta(sticky, ["responses_websockets=2026-02-06"])
assert merged == "custom-beta=1,responses_websockets=2026-02-06"
def test_ws_required_token_no_double_when_client_already_has_it() -> None:
tracker = SessionBetaTracker(max_sessions=10)
sticky = tracker.record_and_get_sticky_betas(
provider="openai",
session_id="ws-2",
client_value="responses_websockets=2026-02-06",
)
merged = merge_openai_beta(sticky, ["responses_websockets=2026-02-06"])
assert merged == "responses_websockets=2026-02-06"