headroom/tests/test_openai_beta_session_sticky.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

227 lines
7.9 KiB
Python
Raw Normal View History

fix: A6 — anthropic-beta and openai-beta deterministic merge + session-sticky PR-A6 of the Phase A cache-safety lockdown. Eliminates P5-50 and preps P0-6 (memory tool injection toggling). Two cache-killer patterns the merge + tracker defeat: 1. Mid-session mutation: when memory was enabled the proxy did an ad-hoc concat of `context-management-2025-06-27` onto the client value (anthropic.py:1244-1248). The order varied with the client value, breaking byte-stable headers across turns. 2. Token drop-out across turns: clients (Claude Code, Codex CLI) MAY drop a beta token between turn N and turn N+1 even when the proxy mutated turn N to add it. The cache hot zone is positional, so the next turn's prefix bytes hash differently and the prefix-cache read misses. Changes ------- `headroom/proxy/helpers.py` * `merge_anthropic_beta` / `merge_openai_beta`: pure, deterministic, order-preserving merge. Client tokens first (in their original order), then Headroom-required tokens (in the order passed). Dedupe is case-insensitive but preserves the original casing of the first occurrence. No regex. * `SessionBetaTracker`: bounded LRU keyed by (provider, session_id), unioning client tokens with previously-seen tokens. OrderedDict LRU; threading.RLock for thread safety (mirrors the CompressionCache pattern from compression_cache.py). * `get_session_beta_tracker` / `_reset_session_beta_tracker_for_test` process-wide singleton with test reset. * `log_beta_header_merge`: structured log per cache-affecting merge. * Env-var knobs (NO HARDCODES): - HEADROOM_BETA_HEADER_STICKY=enabled|disabled (default enabled). - HEADROOM_BETA_TRACKER_MAX_SESSIONS (default 1000). `headroom/proxy/handlers/anthropic.py` * After `compute_session_id` (line ~744): record client `anthropic-beta` against the session tracker, write the sticky value back into `headers` if changed. Order matters: sticky-merge FIRST so memory-injection has the canonical baseline. * Memory-injection site (line ~1244): replace the ad-hoc concat with `merge_anthropic_beta(headers["anthropic-beta"], required_tokens)`. `headroom/proxy/handlers/openai.py` * Chat-completions (line ~360): record/merge `openai-beta`. * /v1/responses HTTP (line ~1213): compute `_responses_session_id` and record/merge `openai-beta`. * /v1/responses WS (line ~1711): replace the ad-hoc absent-only inject with `merge_openai_beta(sticky, ["responses_websockets= 2026-02-06"])`. Replaces any case-variants of the existing key. Tests ----- `tests/test_anthropic_beta_session_sticky.py` (26 tests): * Pure helper: empty inputs, only-client, only-headroom, ordering, dedupe casing, deterministic memory-injection order, no-double- inject when token already present. * Tracker: sticky-on across turns even when client drops, casing preservation, provider namespace independence, LRU eviction at max_sessions, env-var validation (loud failures), thread safety under 16-thread concurrent access, blank-input rejection. `tests/test_openai_beta_session_sticky.py` (17 tests): * Mirror of the anthropic suite for `OpenAI-Beta`. * Plus WS-specific coverage: sticky-then-merge of `responses_websockets=2026-02-06` against client baseline. `tests/test_openai_codex_routing.py` * Add `session_tracker_store` stub to `_DummyOpenAIHandler` so the routing tests still exercise the responses HTTP handler now that it computes a session_id for beta-merge. Notes ----- Build constraints honored: * Configurable: HEADROOM_BETA_HEADER_STICKY, HEADROOM_BETA_TRACKER_MAX_SESSIONS. * No regex, no hardcodes (env-var bounds), no fallbacks (disabled mode is operator opt-in for diagnostics, loud failures on invalid values). * Structured tracing log via `log_beta_header_merge`. Acceptance: * 43 new tests pass. * `cargo test --workspace` green (no Rust changes). * `make ci-precheck` green.
2026-05-02 09:53:37 -07:00
"""Session-sticky `OpenAI-Beta` tests for PR-A6 (P5-50, preps P0-6).
OpenAI's `OpenAI-Beta` follows the same comma-separated convention as
Anthropic's `anthropic-beta`. The proxy auto-injects
`responses_websockets=2026-02-06` on the WS path when absent
(handlers/openai.py:~1711). PR-A6 routes that injection through
`merge_openai_beta` so the client's tokens are preserved and the
auto-injected token is appended deterministically.
The same `SessionBetaTracker` (provider-aware, keyed by
``(provider, session_id)``) backs both providers one tracker, two
namespaces.
"""
from __future__ import annotations
import threading
import pytest
from headroom.proxy.helpers import (
SessionBetaTracker,
_reset_session_beta_tracker_for_test,
merge_openai_beta,
)
# ---------------------------------------------------------------------------
# Pure helper: `merge_openai_beta`
# ---------------------------------------------------------------------------
def test_merge_helper_pure_function() -> None:
"""Same inputs → same output, no global state."""
a = merge_openai_beta("a,b", ["c"])
b = merge_openai_beta("a,b", ["c"])
assert a == b == "a,b,c"
def test_merge_helper_empty_inputs_returns_empty_string() -> None:
assert merge_openai_beta(None, []) == ""
assert merge_openai_beta("", []) == ""
def test_merge_helper_only_client() -> None:
assert merge_openai_beta("alpha=1,beta=2", []) == "alpha=1,beta=2"
def test_merge_helper_only_headroom() -> None:
assert merge_openai_beta(None, ["responses_websockets=2026-02-06"]) == (
"responses_websockets=2026-02-06"
)
def test_merge_helper_preserves_client_order_appends_headroom() -> None:
out = merge_openai_beta(
"alpha=1,beta=2",
["responses_websockets=2026-02-06"],
)
assert out == "alpha=1,beta=2,responses_websockets=2026-02-06"
def test_merge_helper_no_double_inject_when_already_present() -> None:
out = merge_openai_beta(
"responses_websockets=2026-02-06,alpha=1",
["responses_websockets=2026-02-06"],
)
assert out == "responses_websockets=2026-02-06,alpha=1"
def test_dedupe_case_insensitive_preserves_first_casing() -> None:
# OpenAI tokens are usually `kebab=value` so casing rarely differs in
# practice, but the helper's contract is provider-agnostic.
assert (
merge_openai_beta("Responses_Websockets=2026-02-06", ["responses_websockets=2026-02-06"])
== "Responses_Websockets=2026-02-06"
)
def test_merge_helper_skips_empty_tokens() -> None:
assert merge_openai_beta("a, ,b", []) == "a,b"
assert merge_openai_beta("a", ["", "b", " "]) == "a,b"
def test_test_memory_injection_appends_deterministic_order() -> None:
"""Auto-injected `responses_websockets` appends AFTER client tokens."""
out = merge_openai_beta("client-token", ["responses_websockets=2026-02-06"])
assert out == "client-token,responses_websockets=2026-02-06"
# ---------------------------------------------------------------------------
# `SessionBetaTracker` — provider="openai"
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
def _isolate_tracker(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("HEADROOM_BETA_HEADER_STICKY", raising=False)
monkeypatch.delenv("HEADROOM_BETA_TRACKER_MAX_SESSIONS", raising=False)
_reset_session_beta_tracker_for_test()
yield
_reset_session_beta_tracker_for_test()
def test_beta_seen_turn_1_present_in_turn_2_even_if_client_drops() -> None:
tracker = SessionBetaTracker(max_sessions=10)
out1 = tracker.record_and_get_sticky_betas(
provider="openai",
session_id="s-1",
client_value="responses_websockets=2026-02-06,extra-beta=1",
)
assert out1 == "responses_websockets=2026-02-06,extra-beta=1"
out2 = tracker.record_and_get_sticky_betas(
provider="openai",
session_id="s-1",
client_value="responses_websockets=2026-02-06",
)
assert out2 == "responses_websockets=2026-02-06,extra-beta=1"
def test_client_value_preserved_when_no_injection() -> None:
tracker = SessionBetaTracker(max_sessions=10)
out = tracker.record_and_get_sticky_betas(
provider="openai", session_id="s-1", client_value="alpha,beta"
)
assert out == "alpha,beta"
def test_lru_eviction_at_max_sessions() -> None:
tracker = SessionBetaTracker(max_sessions=2)
tracker.record_and_get_sticky_betas(provider="openai", session_id="s-1", client_value="a")
tracker.record_and_get_sticky_betas(provider="openai", session_id="s-2", client_value="b")
tracker.record_and_get_sticky_betas(provider="openai", session_id="s-1", client_value=None)
tracker.record_and_get_sticky_betas(provider="openai", session_id="s-3", client_value="c")
# s-2 evicted.
out = tracker.record_and_get_sticky_betas(
provider="openai", session_id="s-2", client_value=None
)
assert out == ""
def test_disabled_mode_passes_through(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("HEADROOM_BETA_HEADER_STICKY", "disabled")
tracker = SessionBetaTracker(max_sessions=10)
out1 = tracker.record_and_get_sticky_betas(
provider="openai", session_id="s-1", client_value="alpha"
)
assert out1 == "alpha"
out2 = tracker.record_and_get_sticky_betas(
provider="openai", session_id="s-1", client_value=None
)
assert out2 == ""
def test_thread_safe_concurrent_access() -> None:
tracker = SessionBetaTracker(max_sessions=10)
n_threads = 16
iterations = 50
errors: list[BaseException] = []
def worker(idx: int) -> None:
try:
for i in range(iterations):
tracker.record_and_get_sticky_betas(
provider="openai",
session_id="shared",
client_value=f"o-t{idx}-i{i}",
)
except BaseException as e: # noqa: BLE001
errors.append(e)
threads = [threading.Thread(target=worker, args=(idx,)) for idx in range(n_threads)]
for t in threads:
t.start()
for t in threads:
t.join()
assert errors == []
final = tracker.record_and_get_sticky_betas(
provider="openai", session_id="shared", client_value=None
)
final_tokens = set(final.split(","))
expected = {f"o-t{idx}-i{i}" for idx in range(n_threads) for i in range(iterations)}
assert expected.issubset(final_tokens)
def test_provider_namespaces_are_independent() -> None:
tracker = SessionBetaTracker(max_sessions=10)
tracker.record_and_get_sticky_betas(
provider="openai", session_id="shared", client_value="o-token"
)
tracker.record_and_get_sticky_betas(
provider="anthropic", session_id="shared", client_value="a-token"
)
out_openai = tracker.record_and_get_sticky_betas(
provider="openai", session_id="shared", client_value=None
)
out_anth = tracker.record_and_get_sticky_betas(
provider="anthropic", session_id="shared", client_value=None
)
assert out_openai == "o-token"
assert out_anth == "a-token"
def test_ws_required_token_appended_deterministically() -> None:
"""Mirrors the WS handler logic — record client value, then merge required."""
tracker = SessionBetaTracker(max_sessions=10)
sticky = tracker.record_and_get_sticky_betas(
provider="openai",
session_id="ws-1",
client_value="custom-beta=1",
)
merged = merge_openai_beta(sticky, ["responses_websockets=2026-02-06"])
assert merged == "custom-beta=1,responses_websockets=2026-02-06"
def test_ws_required_token_no_double_when_client_already_has_it() -> None:
tracker = SessionBetaTracker(max_sessions=10)
sticky = tracker.record_and_get_sticky_betas(
provider="openai",
session_id="ws-2",
client_value="responses_websockets=2026-02-06",
)
merged = merge_openai_beta(sticky, ["responses_websockets=2026-02-06"])
assert merged == "responses_websockets=2026-02-06"