headroom/tests/test_turn_hooks.py
Tejas Chopra ec950f7ef1
feat(proxy): add turn-hook extension point for buffered model turns (#1891)
## Description

Adds a small, neutral **extension point** to the proxy: a "turn hook"
that lets an opt-in extension observe and optionally re-drive a single
buffered model turn, without touching the core request/response flow for
anyone who has no extension installed.

A hook can:
- `on_request(ctx)` — inspect or rewrite the outbound tools/messages
before they go upstream (the extensible counterpart to the built-in
tool-search deferral that already lives at that point).
- `on_response(ctx, response, call_model)` — inspect the model's
response and, if it wants, call the model again (via `call_model`) and
return a **replacement** response — transparently to the client. This is
the capability that can't be done from ASGI middleware: it reuses the
proxy-internal re-call path (the same `api_call_fn` the CCR handler
already drives).

The module is **inert unless a hook is registered**: the runners return
their input unchanged and are gated on the registry, so with no
extension the proxy is byte-identical to today. A failing hook is logged
and skipped — it can never take the proxy down.

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `headroom/proxy/turn_hooks.py`: `TurnContext`, the `TurnHook`
protocol (`on_request` / `on_response`), a module registry
(`register_turn_hook` / `registered_turn_hooks` / `clear_turn_hooks`),
and the runners `run_request_hooks` / `run_response_hooks`. Inert when
empty; never raises.
- Wire it at four seams, each gated so an empty registry is a
byte-identical no-op:
- Anthropic — pre-send (right after the existing tool-search deferral) +
the CCR response seam.
- OpenAI — the Responses tool-shaping point (right after the existing
tool-search deferral, copy-on-write-safe) + the CCR response seam.
- Add `tests/test_turn_hooks.py`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed (no-op regression across CCR/handler
suites)

### Test Output

```text
$ ruff check headroom/proxy/turn_hooks.py tests/test_turn_hooks.py \
      headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py
All checks passed!

$ ruff format --check <same 4 files>
4 files already formatted

$ mypy headroom
Success: no issues found in 408 source files

$ pytest tests/test_turn_hooks.py -q
9 passed in 0.17s

$ pytest tests/test_turn_hooks.py tests/test_ccr_response_handler.py \
      tests/test_ccr_tool_injection.py tests/test_proxy_ccr.py \
      tests/test_openai_tool_search_deferral.py \
      tests/test_openai_responses_compression_units.py \
      tests/test_handler_outcome_tag_invariant.py -q
135 passed  (+ 1 pre-existing cross-file flake in test_proxy_ccr::test_health_endpoint,
             which passes in isolation and in its own file: `pytest tests/test_proxy_ccr.py` -> 19 passed)
```

## Real Behavior Proof

- **Environment:** local macOS, project `.venv` (Python 3.12.6); `ruff`
pinned to CI's `0.15.17` via `uvx ruff@0.15.17`; `mypy` from the venv.
- **Exact command / steps:** branched off `upstream/main`; added the
hook module + wired the four handler seams; ran the
ruff/format/mypy/pytest commands above.
- **Observed result:** The unit tests exercise the whole contract —
registry, `on_request` mutating `ctx.tools`, `on_response` returning a
replacement, the `await call_model(...)` re-drive loop,
replacement-chaining across hooks, and the never-raise guarantee. The
existing CCR + handler suites pass unchanged, which is the point: with
no hook registered the added code is a no-op (the runners short-circuit
on an empty registry).
- **Not tested:** the live interactive re-drive path with a *registered*
hook against a real upstream — no hook ships in this repo, so that path
is covered here only by the unit test's fake `call_model`. The
`on_request` seam fires on the Anthropic pre-send and OpenAI Responses
paths (where the existing tool-search deferral runs); other send paths
(e.g. chat-completions, streaming) are not wired in this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
2026-07-08 17:38:45 -07:00

184 lines
5.3 KiB
Python

"""Turn-hook registry + runners (headroom/proxy/turn_hooks.py).
The hook surface is opt-in: with nothing registered the runners must be exact
no-ops (the property the proxy relies on to stay byte-identical for everyone who
has no extension installed). These tests pin that, plus request-mutation,
response-replacement, the re-drive (``call_model``) loop, and the
never-raise guarantee.
"""
from __future__ import annotations
import pytest
from headroom.proxy.turn_hooks import (
TurnContext,
clear_turn_hooks,
register_turn_hook,
registered_turn_hooks,
run_request_hooks,
run_response_hooks,
)
@pytest.fixture(autouse=True)
def _clean_registry():
clear_turn_hooks()
yield
clear_turn_hooks()
def _ctx(**kw):
base = {"provider": "anthropic", "model": "claude-x", "messages": [], "tools": None}
base.update(kw)
return TurnContext(**base)
async def _noop_call_model(_messages): # pragma: no cover - never invoked in no-op tests
raise AssertionError("call_model must not be invoked when no hook re-drives")
# --- inert-when-empty (the load-bearing guarantee) ---------------------------
def test_request_runner_inert_when_empty():
assert registered_turn_hooks() == []
ctx = _ctx(tools=[{"name": "a"}])
before = ctx.tools
run_request_hooks(ctx) # must not raise, must not touch ctx
assert ctx.tools is before
@pytest.mark.asyncio
async def test_response_runner_returns_input_unchanged_when_empty():
resp = {"id": "orig", "content": []}
out = await run_response_hooks(_ctx(), resp, _noop_call_model)
assert out is resp # same object, untouched
# --- on_request mutation -----------------------------------------------------
def test_on_request_may_mutate_ctx():
class Shrink:
name = "shrink"
def on_request(self, ctx: TurnContext) -> None:
ctx.tools = [t for t in (ctx.tools or []) if t["name"] != "drop_me"]
register_turn_hook(Shrink())
ctx = _ctx(tools=[{"name": "keep"}, {"name": "drop_me"}])
run_request_hooks(ctx)
assert ctx.tools == [{"name": "keep"}]
# --- on_response replacement + re-drive loop ---------------------------------
@pytest.mark.asyncio
async def test_on_response_can_replace_via_call_model():
calls: list[list] = []
async def call_model(messages):
calls.append(messages)
return {"id": "resolved", "content": [{"type": "text", "text": "done"}]}
class ResolveOnce:
name = "resolve"
async def on_response(self, ctx, response, call_model):
if response.get("id") == "needs-work":
return await call_model(ctx.messages + [{"role": "user", "content": "go"}])
return None
register_turn_hook(ResolveOnce())
out = await run_response_hooks(
_ctx(messages=[{"role": "user", "content": "hi"}]), {"id": "needs-work"}, call_model
)
assert out["id"] == "resolved"
assert len(calls) == 1
@pytest.mark.asyncio
async def test_on_response_none_leaves_response_unchanged():
class Observer:
name = "observe"
async def on_response(self, ctx, response, call_model):
return None # observe only
register_turn_hook(Observer())
resp = {"id": "orig"}
out = await run_response_hooks(_ctx(), resp, _noop_call_model)
assert out is resp
@pytest.mark.asyncio
async def test_replacements_chain_across_hooks():
class First:
name = "first"
async def on_response(self, ctx, response, call_model):
return {"id": "after-first", "seen": response["id"]}
class Second:
name = "second"
async def on_response(self, ctx, response, call_model):
return {"id": "after-second", "seen": response["id"]}
register_turn_hook(First())
register_turn_hook(Second())
out = await run_response_hooks(_ctx(), {"id": "orig"}, _noop_call_model)
assert out == {"id": "after-second", "seen": "after-first"} # Second saw First's output
# --- a failing hook must never break the proxy -------------------------------
def test_failing_on_request_is_swallowed():
class Boom:
name = "boom"
def on_request(self, ctx: TurnContext) -> None:
raise RuntimeError("kaboom")
register_turn_hook(Boom())
run_request_hooks(_ctx()) # must not raise
@pytest.mark.asyncio
async def test_failing_on_response_is_skipped_and_original_survives():
class Boom:
name = "boom"
async def on_response(self, ctx, response, call_model):
raise RuntimeError("kaboom")
class Good:
name = "good"
async def on_response(self, ctx, response, call_model):
return {"id": "recovered"}
register_turn_hook(Boom())
register_turn_hook(Good())
out = await run_response_hooks(_ctx(), {"id": "orig"}, _noop_call_model)
assert out == {"id": "recovered"} # Boom skipped, Good still ran
# --- hooks with only one method defined --------------------------------------
@pytest.mark.asyncio
async def test_hook_without_on_response_is_skipped():
class OnlyRequest:
name = "only-request"
def on_request(self, ctx: TurnContext) -> None:
pass
register_turn_hook(OnlyRequest())
resp = {"id": "orig"}
out = await run_response_hooks(_ctx(), resp, _noop_call_model)
assert out is resp