mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
TenderDeve
|
a4bd2e62a5
|
fix(proxy): gate mid-turn message coalescing to Claude Code clients (#1643)
## Description
`headroom wrap opencode` (and any other `@ai-sdk/anthropic` client)
can't use subagents. The subagent is spawned, receives the prompt, and
never responds; OpenCode throws `invalid_union / "No matching
discriminator" / discriminator: "type"`.
Root cause is headroom's mid-turn message coalescing. It keys concurrent
streaming requests by `md5(model:system[:500])` (`_get_session_key`,
`handlers/streaming.py`). An OpenCode subagent runs concurrently with
the main agent on the same model and same first-500-char system prefix,
so it produces the **same** session key and collides with the
still-active main stream. Two things then break it:
1. `handlers/anthropic.py` sees the key in `_active_streams` and answers
the subagent's request with a bare `202 headroom_queued` instead of
forwarding it — so the subagent never gets a response.
2. When the main stream ends, `handlers/streaming.py` emits a
non-standard `event: headroom_pending_messages` SSE event.
`@ai-sdk/anthropic`'s SSE parser keys its Zod union on `type`, and
`headroom_pending_messages` isn't a valid Anthropic event type — hence
the error.
The 202 reply and the `headroom_pending_messages` event are a Claude
Code-only protocol (nothing else consumes them). This gates coalescing
to Claude Code clients; every other harness streams normally.
Closes #1608
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `handlers/streaming.py`: only register a stream in `_active_streams`
when `classify_client(headers) == "claude-code"`, and only emit the
`headroom_pending_messages` SSE event for Claude Code.
- `handlers/anthropic.py`: only take the queue-and-`202` branch when the
client is Claude Code (in addition to the existing `session_key in
_active_streams` check).
- Regression tests in `tests/test_mid_turn_steering.py` for all four
cases (active-stream registration and pending-event emission, each for a
Claude Code vs. a non-Claude-Code client).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_mid_turn_steering.py -q
9 passed in 0.46s
$ ruff check headroom/proxy/handlers/streaming.py headroom/proxy/handlers/anthropic.py tests/test_mid_turn_steering.py
All checks passed!
$ ruff format --check <same files>
3 files already formatted
$ mypy headroom/proxy/handlers/streaming.py headroom/proxy/handlers/anthropic.py --ignore-missing-imports
Success: no issues found in 2 source files
```
## Real Behavior Proof
- Environment: macOS (arm64), Python 3.14 venv, editable install of this
branch.
- Exact command / steps: ran `pytest tests/test_mid_turn_steering.py` —
the new tests drive `_stream_response` with a queued mid-turn message
under an `opencode/1.0` User-Agent vs. a `claude-code/1.2.3` User-Agent
and assert the streamed bytes. Also ran the streaming + anthropic
handler suites (`pytest tests/test_mid_turn_steering.py
tests/test_proxy_streaming_* tests/test_anthropic_*
tests/test_streaming_usage_parser.py`).
- Observed result: with the `opencode/1.0` client the session is never
added to `_active_streams` and the response contains no
`headroom_pending_messages` event; with `claude-code/1.2.3` both still
happen (protocol preserved). Handler suites: 155 passed, 3 skipped.
Before this change the non-Claude client received the
`headroom_pending_messages` event (the exact byte string the OpenCode
parser rejects).
- Not tested: end-to-end against a live OpenCode + real subagent run —
reproduced deterministically at the proxy layer instead (the emitted SSE
bytes are the direct source of the reported `invalid_union` error).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Additional Notes
Gating on `classify_client == "claude-code"` (User-Agent `claude-code/`
/ `claude-cli/`) is the same client identification used elsewhere in the
proxy. Unidentified clients (no recognized User-Agent) are treated as
non-Claude-Code and stream normally, which is the safe default for this
feature.
## Maintainer Update (2026-07-21)
- Removed the manual `CHANGELOG.md` entry so release-please remains the
source of changelog updates; pushed `
|
||
|
wzy-del
|
c365c7ff81
|
fix(proxy): only queue mid-turn messages for opt-in clients with explicit session header (#1951)
## Description
Mid-turn steering wrongly queues **concurrent independent streams**.
When two streaming `/v1/messages` requests share the same model + system
prompt and arrive concurrently (no `x-headroom-session-id` header), the
proxy misclassifies the second as a "mid-turn message", returns `202
{"event":"headroom_queued"}`, and never forwards it upstream. A standard
Anthropic SDK client that made a *streaming* call receives a non-SSE 202
→ empty event stream → `AssertionError` (`assert
self.__final_message_snapshot is not None` in
`anthropic/lib/streaming/_messages.py`), and fails after retries.
**Root cause.** Without an `x-headroom-session-id` header,
`_get_session_key()` falls back to `md5(model + system[:500])` (mirrors
`prefix_tracker.compute_session_id`). That key is intentionally coarse
and cannot distinguish genuinely concurrent, independent streams that
share a model + system prompt (e.g. a main conversation plus its
background / parallel requests), so the second stream hits `session_key
in self._active_streams` and gets queued.
A queued message is only ever drained back to the client via the custom
`headroom_pending_messages` SSE event, which a standard Anthropic SDK
does not understand — so mid-turn steering is effectively a private
protocol for clients that **opt in** via `x-headroom-session-id`. A
client that never sends the header can never participate in the queue;
for it, the 202 is simply a broken streaming response.
Note: "send a unique header per request" is **not** a workaround — the
same header also drives `prefix_tracker.compute_session_id()`, so
unique-per-stream ids break prompt caching while a shared id keeps
colliding.
Closes #1949
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Add `StreamingMixin._should_queue_mid_turn()` helper that gates
mid-turn queuing behind an explicit `x-headroom-session-id` header.
- Header-less concurrent streams are now forwarded upstream normally;
only opt-in (header-bearing) callers can be queued.
- Prefix-tracker / cache-alignment behavior is untouched — the header
still drives `compute_session_id()` exactly as before.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ pytest tests/test_mid_turn_steering.py
6 passed
```
New test `test_should_queue_only_with_explicit_session_header`: a
header-less concurrent stream must not queue; an explicit-header opt-in
must. All existing `test_mid_turn_steering.py` cases pass an explicit
header and are unaffected.
## Real Behavior Proof
- Environment: macOS, `headroom-ai` 0.30.0 (installed via `uv tool`),
proxy running `headroom proxy --port 8799 --no-http2 --mode cache`,
upstream = an Anthropic-compatible gateway. Client = Hermes Agent
(Anthropic SDK, streaming) driving a main conversation plus concurrent
background/parallel requests that share the same model + system prompt.
- Exact command / steps:
1. Reproduce on stock 0.30.0: concurrent streaming requests without
`x-headroom-session-id` → second stream returns `202
{"event":"headroom_queued"}` → client raises `AssertionError` in
`anthropic/lib/streaming/_messages.py`.
2. Correlate logs: count of `AssertionError` in the client error log vs
count of `202` in the proxy access log for the window — **48 == 48**,
timestamps line up 1:1.
3. Apply this patch to the running package, restart the proxy, re-run
the same concurrent workload.
- Observed result: after the fix, **0 × 202 / all requests 200**, no new
`AssertionError`, and `cache_hit_pct` stayed ~99% (prefix caching
intact). Header-bearing opt-in clients still queue mid-turn as before.
- Not tested: behavior under a client that deliberately sends a
*changing* `x-headroom-session-id` per request (out of scope —
documented as a caching anti-pattern, not a supported mode).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Docs / CHANGELOG unchanged: this is a proxy-internal correctness fix
with no user-facing config surface.
- The fix is deliberately minimal and conservative — it only narrows
*when* queuing engages (explicit opt-in header), leaving the
prefix-tracker, cache-alignment, and body-rewrite paths byte-for-byte
identical.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Rod Boev
|
b09f027062
|
fix(proxy): queue mid-turn user messages on non-Bedrock streaming path (#1377)
## Description When a user types a follow-up message while Claude Code is working mid-turn, the proxy silently drops it on the standard non-Bedrock Anthropic path. `_stream_response` (`streaming.py:794`) opens a single upstream connection per request with no mechanism to detect concurrent requests for the same conversation. Mid-turn POSTs get forwarded to Anthropic, which rejects them because the prior turn is still in-flight. The message is silently lost. This PR adds a per-session `asyncio.Queue` on `StreamingMixin` keyed by session identity. When a new POST arrives while a stream is active for the same conversation, the message is queued and a 202 response with `event: headroom_queued` is returned. After `message_stop`, the queue is drained and an `event: headroom_pending_messages` frame is emitted with the buffered content. PR #1080 addresses the Bedrock SSE path; this covers the standard non-Bedrock path. Closes #902 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/streaming.py`: add `_mid_turn_queues` and `_active_streams` class-level state on `StreamingMixin`; register/deregister active streams in `_stream_response`; drain queue after `message_stop` and emit `headroom_pending_messages`; add `_queue_mid_turn_message` helper - `headroom/proxy/handlers/anthropic.py`: in the non-Bedrock request handler, check `_active_streams` before calling `_stream_response`; queue and return 202 if session is already streaming - `tests/test_mid_turn_steering.py`: new file with three tests covering queue creation, message buffering, and no-op when no stream is active - `CHANGELOG.md`: bug fix entry ## Testing - [x] Unit tests pass (`uv run pytest tests/test_mid_turn_steering.py -v`) - [x] Linting passes (`uv run ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) — N/A: repo does not enforce mypy in CI - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # paste actual pytest -v output here after running ``` ## Real Behavior Proof - Environment: headroom proxy, Python 3.11+, no live API key required for unit tests - Exact command / steps: construct `StreamingMixin`, register a session key in `_active_streams`, call `_queue_mid_turn_message`, inspect `_mid_turn_queues` - Observed result: message body is present in the queue for the session key; `_mid_turn_queues` and `_active_streams` class attributes exist on `StreamingMixin` - Not tested: actual SSE event emission under a live streaming connection; interaction with Bedrock path (separate, handled by PR #1080); queue TTL eviction under load; `yield` inside `finally` block for pending-messages event under client disconnect (existing codebase pattern, not a new concern) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The Bedrock streaming path (`_stream_response_bedrock` at `streaming.py:1344`) is separate and already scoped to PR #1080 (MrAshRhodes). This PR only touches the standard non-Bedrock path. The `_active_streams` set and `_mid_turn_queues` dict use session keys derived from the `x-headroom-session-id` header (matching `prefix_tracker.py:339`) or a fallback hash of model+system, so they are conversation-scoped and won't cross-contaminate unrelated sessions. Full end-to-end testing requires a running proxy with a live Anthropic API key and a Claude Code client that sends mid-turn messages. The unit tests validate the queue mechanism in isolation. --------- Co-authored-by: JD Davis <mxjerrett@gmail.com> |