Commit graph

1 commit

Author SHA1 Message Date
Focused Instability
4dab254d52
fix: emit SSE ping before message_start on Bedrock streaming path (issue #902) (#1080)
## Description

Closes #902

Mid-turn user interjections (steering) silently dropped through the
Bedrock streaming path. The _stream_response_bedrock code path
reconstructs Anthropic SSE events from parsed StreamEvent objects
instead of passing raw bytes through, so SSE-level ping keepalives are
never forwarded to Claude Code. Claude Code relies on ping events to arm
its mid-turn steering / interruptible state; without them, queued
interjections are discarded instead of sent.

Root cause (confirmed):
- Standard direct-Anthropic path does a raw yield-chunk passthrough —
pings flow unchanged.
- Bedrock path (_stream_response_bedrock.generate()) reconstructs events
from litellm/anyllm
stream_message() output, which only yields semantic events
(message_start, content_block_*,
  message_delta, message_stop, error). No pings, ever.

Fix: emit a synthetic 'event: ping / data: {}' at stream start (before
the first message_start)
so downstream clients see the same ping-then-content cadence as a real
Anthropic stream.

Note: periodic pings for very long responses (>~25s) may be needed if
steering disarms on a timer.
This commit arms it at turn start; follow-up if reporters confirm
steering still drops on long turns.

The causal link (ping → steering) is the reporter's hypothesis from
hands-on debugging.
The observable defect (zero pings in stream) is confirmed and fixed.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- headroom/proxy/handlers/streaming.py: yield ping event before the
event loop in _stream_response_bedrock.generate()
- tests/test_proxy/test_bedrock_sse_ping.py: 3 new tests asserting ping
appears before message_start

## Testing

- [x] Unit tests pass (pytest)
- [x] Linting passes (ruff check .)
- [x] New tests added for new functionality

### Test Output

```
tests/test_proxy/test_bedrock_sse_ping.py::test_bedrock_stream_emits_ping_before_message_start PASSED
tests/test_proxy/test_bedrock_sse_ping.py::test_bedrock_stream_ping_has_empty_data PASSED
tests/test_proxy/test_bedrock_sse_ping.py::test_bedrock_stream_contains_message_stop PASSED
tests/test_backend_streaming_cache_metrics.py (4 tests) PASSED
7 passed in 4.41s
```

## Real Behavior Proof

- Environment: macOS, Python 3.11, headroom unit tests
- Exact command / steps: pytest
tests/test_proxy/test_bedrock_sse_ping.py -v
- Observed result: 3 new tests pass; ping appears before message_start
in Bedrock stream
- Not tested: end-to-end against live Bedrock + Claude Code (no Bedrock
credentials available)

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] Code follows project style guidelines
- [x] Code is commented where non-obvious
- [x] No new warnings
- [x] Tests added and passing

## Additional Notes

The Rust proxy files mentioned in the issue (sse/framing.rs,
sse/anthropic.rs) are NOT part of this fix.
Those drops are in a telemetry-only tee task that never affects the
client byte path — the Rust proxy
does a raw bytes passthrough for all responses. The defect is
Python-only, confined to the Bedrock path.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-08-04 21:41:58 -05:00