mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
9 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Abhay Singh
|
77b26c093c
|
fix(proxy/streaming): tolerate malformed content in _response_to_sse (#2481)
## Description
`StreamingMixin._response_to_sse` rebuilds an Anthropic SSE stream from
a buffered response dict. It iterated the content and read usage with no
type guards:
```python
for idx, block in enumerate(response.get("content", [])):
if block.get("type") == "text":
...
...
"usage": {"output_tokens": response.get("usage", {}).get("output_tokens", 0)},
```
`response` here is provider- and reconstruction-controlled.
`.get("content", [])` only falls back when the key is absent, so a
present-but-null `content` returns `None` and `enumerate(None)` raises
`TypeError`. A non-list `content` (e.g. a bare string) makes
`block.get(...)` raise `AttributeError`, and a null element inside the
list hits the same `AttributeError`. `response.get("usage",
{}).get(...)` breaks the same way on `usage: null`.
This matters because the Anthropic buffered CCR path calls it inside an
`except ValueError` guard only:
```python
try:
sse_events = self._response_to_sse(resp_json, "anthropic")
except ValueError as sse_err:
...
```
A `TypeError`/`AttributeError` from any of the shapes above escapes that
guard and 500s the streamed request. The sibling
`_record_ccr_feedback_from_response` in the same class already guards
`content` for list-ness and skips non-dict blocks, so this closes the
asymmetry.
## Fix
Coerce `content` to a list before iterating (non-list becomes empty),
skip any non-dict block, and coerce a non-dict `usage` to `{}` before
reading `output_tokens`. Well-formed responses render byte-for-byte as
before.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/streaming.py`: list-guard `content`, skip
non-dict blocks, and dict-guard `usage` in `_response_to_sse`.
- `tests/test_sse_thinking_blocks.py`: regression rendering responses
with null/non-list content, a null block element, and null usage, plus a
check that a valid block alongside a null element still renders.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_sse_thinking_blocks.py -q
14 passed
# with the fix reverted, the new test fails with
# TypeError: 'NoneType' object is not iterable
$ uvx ruff@0.15.17 check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/streaming.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: called
`StreamingMixin()._response_to_sse(response, "anthropic")` with four
malformed bodies (`content: null`, `content: "not-a-list"`, `content:
[null, {text}]`, `usage: null`); then reverted `streaming.py` and re-ran
the same inputs.
- Observed result: with the fix each body produces a well-formed SSE
envelope (message_start ... message_stop) and the valid block alongside
a null element still emits its text_delta; with the fix reverted the
`content: null` body raises `TypeError: 'NoneType' object is not
iterable` and the others raise `AttributeError`. Ran against the actual
module via `tests/test_sse_thinking_blocks.py`.
- Not tested: a live upstream returning a malformed buffered response
end to end through the CCR path.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
Parideboy
|
8c8fae0d0b
|
fix(proxy): reassemble server_tool_use.input from streamed partial_json (#2449)
## Description
Under `--target-ratio 0.4` a session died mid-run with a fatal Anthropic
400:
```
messages.13.content.0.server_tool_use.input: Input should be an object
```
Root cause is not compression of the request: the request path passes
structured blocks through byte-for-byte. It is **SSE stream
reconstruction**. When the proxy rebuilds a full Anthropic message from
the streamed response (non-stream retry, buffered, and CCR round-trip
paths), the `content_block_stop` handler parsed the accumulated
`_partial_json` into `input` only for blocks whose type was exactly
`tool_use`. A `server_tool_use` block streams its input identically via
`input_json_delta`, so its input was never reassembled: the block kept
the empty start-event `input: {}` and leaked the internal
`_partial_json` scratch key. That reconstructed block becomes assistant
history, and on the next turn the client replays it, so Anthropic
rejects `server_tool_use.input`. `--target-ratio` only makes the
buffered/reconstructed path more likely; it does not itself rewrite the
block.
Refs #2438 (Finding 2). Findings 1 (prompt-cache regression) and 3
(compression not engaging) are architectural and tracked separately.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/streaming.py` (`_parse_sse_to_response`):
gate the `content_block_stop` `_partial_json` → `input` parse on the
presence of `_partial_json`, not `type == "tool_use"`, so
`server_tool_use` (and any future tool-ish block) is reassembled. Always
strip the scratch key; `input` is always a parsed object (`{}` on
malformed/empty JSON).
- `headroom/ccr/response_handler.py`
(`StreamingCCRHandler._reconstruct_anthropic_response`): same
stop-handler fix, and relax the `input_json_delta` accumulator that was
likewise gated on `type == "tool_use"` so server_tool_use partial JSON
is accumulated at all.
- Regression tests in `tests/test_sse_thinking_blocks.py` and
`tests/test_ccr_response_handler_extra.py`: a `server_tool_use` whose
input arrives via `input_json_delta` must reconstruct to the parsed
object with no `_partial_json` leak.
- Leave `CHANGELOG.md` untouched, release-please generates it.
## Testing
- [x] Unit tests pass (`python -m pytest
tests/test_sse_thinking_blocks.py
tests/test_ccr_response_handler_extra.py -q`)
- [x] Linting passes (`ruff check`, `ruff format --check` on the four
changed files)
- [x] Type checking passes (`mypy headroom/proxy/handlers/streaming.py
headroom/ccr/response_handler.py --ignore-missing-imports`)
- [x] New tests added for new functionality
### Test Output
```text
$ python -m pytest tests/test_sse_thinking_blocks.py tests/test_ccr_response_handler_extra.py -q
26 passed in 3.36s
$ ruff check <changed files>
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.13, local dev checkout on a branch
off upstream/main
- Exact command / steps: Fed a synthetic Anthropic SSE stream with a
`server_tool_use` block whose `input` arrives as `input_json_delta`
partial JSON through both reconstructors (`_parse_sse_to_response`,
`_reconstruct_anthropic_response`); then temporarily restored the `type
== "tool_use"` guard and re-ran.
- Observed result: With the fix, the reconstructed block has `input ==
{"query": ...}` and no `_partial_json` key. With the old guard the test
fails, `input` stays `{}` and the scratch key leaks, reproducing the
malformed block that Anthropic rejects on replay.
- Not tested: End-to-end multi-turn `--target-ratio` session against the
live Anthropic API from this environment, reproduced at the
reconstruction seam instead; the reporter observed the 400 on real
traffic.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Abhay Singh
|
6decbd1e6e
|
fix(proxy/streaming): preserve non-standard content-block fields on SSE reconstruction (#2271)
## Description
When the proxy reconstructs a full response from an Anthropic SSE
stream, it silently drops the payload of any content block that isn't
`text` / `tool_use` / `thinking` / `redacted_thinking`.
`_parse_sse_to_response` builds each block on `content_block_start`:
```python
current_block = {"type": btype, "index": block_index}
if btype == "text":
current_block["text"] = block.get("text", "")
elif btype == "tool_use":
current_block["id"] = block.get("id")
current_block["name"] = block.get("name")
current_block["input"] = {}
elif btype == "thinking":
...
elif btype == "redacted_thinking":
...
blocks_by_index[block_index] = current_block
```
There's no branch for other block types. A `server_tool_use` or
`web_search_tool_result` block (Anthropic server-side tools) therefore
reconstructs as a bare `{"type": ..., "index": ...}`, losing its `id`,
`name`, `input`, and content.
This reconstructed response is what `has_memory_tool_calls` and the CCR
feedback recorder inspect, so a stream that used a server-side tool
feeds detection a gutted block. The sibling reconstructor
`_reconstruct_anthropic_response` (in
`headroom/ccr/response_handler.py`) already handles this correctly with
`elif btype: current_block = dict(block)` — this path just wasn't
updated.
## Fix
Add an `elif btype:` branch that copies through all of the block's
fields (except `type`, already set), mirroring the sibling:
```python
elif btype:
for _k, _v in block.items():
if _k != "type":
current_block[_k] = _v
```
Standard blocks are untouched; non-standard blocks keep their fields.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/streaming.py`: add the non-standard-block
field copy in `_parse_sse_to_response`'s `content_block_start` handler.
- `tests/test_sse_thinking_blocks.py`: new test asserting a
`server_tool_use` block keeps `id` / `name` / `input`.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/streaming.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the block-construction logic with a dependency-free script
and left the full pytest to CI.
- Exact command / steps: ran a `server_tool_use` content_block_start
through the OLD (special-cases only) and NEW (`elif btype:` copy) logic,
plus a `text` block as a control.
- Observed result: OLD produces `{"type": "server_tool_use", "index":
0}` (id/name/input gone); NEW keeps `id`/`name`/`input`; the `text`
block is identical under both.
- Not tested: a live server-tool stream end-to-end; full local `pytest`
deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new test reuses the
existing `_Parser(StreamingMixin)` harness in
`tests/test_sse_thinking_blocks.py`, so it runs under the normal CI
pytest job; behaviour is additionally verified by the standalone proof
above.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Matt Van Horn
|
d05802b620
|
fix: emit unknown Anthropic content blocks verbatim in buffered-to-SSE conversion (#1825)
## Description
Unknown Anthropic content block types are now emitted verbatim inside
`content_block_start` during buffered-to-SSE conversion instead of
raising `ValueError`. The block-start loop in
`StreamingMixin._response_to_sse`
(`headroom/proxy/handlers/streaming.py`) previously handled only `text`,
`tool_use`, `thinking`, and `redacted_thinking`; any other type fell
through to a hard raise, which turned a fully-generated upstream
response into an HTTP 502.
Closes #1806
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Emit unknown content block types, including `server_tool_use`,
`server_tool_result`, `mcp_tool_use`, and future Anthropic block types,
verbatim in `content_block_start` with no delta.
- Preserve main's explicit `server_tool_use` support and newer buffered
CCR/thinking regression coverage after merging current main.
- Keep `content_block_delta` generation gated on known delta-capable
block types, so unknown blocks do not produce spurious deltas.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
uv run --with pytest python -m pytest tests/test_sse_thinking_blocks.py -q
12 passed, 1 warning
uv run --with ruff==0.15.17 ruff check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows worktree `C:\git\headroom-governance-main`, PR
head `
|
||
|
Alex Ander
|
f663894f60
|
fix(ccr): preserve Anthropic re-stream shape (#1854)
## Description Buffered Anthropic CCR re-streaming now preserves response shape instead of normalizing newer Anthropic/Fable fields away during SSE reconstruction. Related upstream traffic checked before opening: - #1451 added the direct streaming CCR buffered path and already preserves thinking/signature/citation fields in `StreamingMixin._response_to_sse`. - #1825 / #1806 cover unknown Anthropic content block types such as `server_tool_use`; this PR does not duplicate that fix. - No open or closed issue/PR search result mentioned `stop_details`, `signature_delta thinking_delta`, `refusal stop_reason`, `Fable CCR`, or `re-stream thinking` as this exact gap. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Preserve `thinking`, `redacted_thinking`, `signature_delta`, `citations_delta`, `stop_details`, and verbatim `stop_reason` while parsing Anthropic SSE in `StreamingCCRHandler`. - Reuse the shared proxy Anthropic SSE renderer for the legacy `StreamingCCRHandler` output path so it preserves the same shape as the direct buffered streaming CCR path. - Preserve `stop_details` and stop defaulting missing `stop_reason` to `end_turn` in `StreamingMixin._response_to_sse`. - Add focused regressions for empty thinking blocks, signatures, redacted thinking data, `refusal`, and `stop_details`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_ccr_response_handler_extra.py tests/test_sse_thinking_blocks.py -q 18 passed in 0.29s $ uv run --frozen --extra dev pytest tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py -q 4 passed, 1 warning in 10.25s $ uv run --frozen --extra dev ruff check headroom/ccr/response_handler.py headroom/proxy/handlers/streaming.py tests/test_ccr_response_handler_extra.py tests/test_sse_thinking_blocks.py All checks passed! $ uv run --frozen --extra dev ruff format --check headroom/ccr/response_handler.py headroom/proxy/handlers/streaming.py tests/test_ccr_response_handler_extra.py tests/test_sse_thinking_blocks.py 4 files already formatted ``` ## Real Behavior Proof - Environment: local worktree based on current `origin/main` after `git fetch origin --prune && git rebase origin/main`. - Exact command / steps: parse and re-emit a synthetic Anthropic SSE stream containing an empty `thinking` block, `signature_delta`, `redacted_thinking.data`, `message_delta.stop_reason = "refusal"`, and `message_delta.stop_details`. - Observed result: the reconstructed response and re-emitted SSE retain the thinking/signature/redacted data plus `refusal` and `stop_details`; a missing `stop_reason` is no longer rewritten to `end_turn`. - Not tested: live upstream Fable/Opus traffic against the proxy. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Rod Boev
|
ede085cc11
|
fix(ccr): preserve thinking blocks in buffered stream re-synthesis (#1897)
## Description Closes #1876. When CCR forces `stream: false` upstream (the buffered path for `headroom_retrieve`), the proxy re-synthesizes an SSE stream for the client from the buffered JSON response via `StreamingMixin._response_to_sse`. The reported symptom was extended-thinking responses arriving corrupted: text blocks missing, and duplicate empty `thinking` blocks with the same timestamp/requestId. Tracing the two functions the issue pointed at: - `_response_to_sse()` already handles `thinking`, `redacted_thinking`, `citations`, and `server_tool_use` blocks explicitly (added across #1451 and #1826) — a direct thinking → text → tool_use round trip through it reconstructs correctly, so that half of the reported pointer no longer applies on current `main`. - `_parse_sse_to_response()`'s `content_block_stop` handling still had the bug: it deduped appended blocks with `target not in response["content"]`, plain whole-dict equality. That has two failure modes: (1) two genuinely distinct blocks that happen to accumulate identical values (e.g. two separate empty `thinking` blocks) could collapse into one, and (2) a redelivered `content_block` lifecycle for the *same* index (e.g. from the proxy's own HTTP/2 stream-reset retry path) whose accumulated content differs from the first delivery — a truncated vs. complete `thinking` block, say — produced **two** dict-unequal entries for one logical block, i.e. exactly the "duplicated" symptom reported. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/streaming.py`: `_parse_sse_to_response()` now dedupes appended content blocks by block index (falling back to object identity for the legacy no-index path) instead of whole-dict equality. One `content_block_stop` per index is honored; a redelivered lifecycle for an already-appended index is dropped rather than appended as a second entry. - `tests/test_sse_thinking_blocks.py`: added three focused regressions — two distinct empty `thinking` blocks at different indices both survive; a redelivered block at the same index with *different* accumulated content collapses to one entry (this one fails on `main` before the fix — `assert 2 == 1`); and an end-to-end `_response_to_sse` → `_parse_sse_to_response` round trip for a buffered CCR extended-thinking response (`thinking` → `text` → `tool_use`) confirming all three block types survive intact and the thinking block isn't duplicated. Adjacent open PR #1854 touches the same files for a different symptom (preserving `stop_details`/`refusal` shape through the legacy test-only `StreamingCCRHandler`, which isn't wired into any real request path); this PR doesn't overlap with that change. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ .venv/Scripts/python.exe -m pytest tests/test_sse_thinking_blocks.py -v 10 passed in 0.28s $ .venv/Scripts/python.exe -m pytest tests/ -k "streaming or ccr or sse" -q 737 passed, 39 skipped, 7569 deselected in 145.23s (2 pre-existing, unrelated failures reproduce identically on unmodified main: a CRLF/LF checkout difference in test_owned_asset_encoding.py, and an order-dependent CCR-store state flake in test_proxy_ccr.py that passes in isolation on both main and this branch.) $ .venv/Scripts/python.exe -m ruff check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py All checks passed! $ .venv/Scripts/python.exe -m ruff format --check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py 2 files already formatted ``` ## Real Behavior Proof - Environment: local worktree on current `origin/main`. - Exact command / steps: `git stash` the `streaming.py` fix, run `pytest tests/test_sse_thinking_blocks.py::test_redelivered_block_same_index_different_content_collapses_to_one_entry`, then `git stash pop` and rerun. - Observed result: on unmodified `main` the test fails — `assert 2 == 1`, with `response["content"]` holding `[{'type': 'thinking', 'index': 0, 'thinking': 'partial'}, {'type': 'thinking', 'index': 0, 'thinking': 'full retried text'}]` — two entries for one logical block index. With the fix, the same scenario produces exactly one entry. This is the mechanism behind the reported "duplicate empty thinking blocks" symptom. - Not tested: a live Claude Code session reproducing the exact reported transcript signature end-to-end (requires the CCR/retrieval infrastructure and an extended-thinking model live). The fix is verified at the unit level against the two functions the issue traced the corruption to. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Rod Boev
|
4ac54934cb
|
fix(streaming): preserve server_tool_use sse blocks (#1826)
## Description Buffered Anthropic responses currently fail late when they contain a `server_tool_use` block. `_response_to_sse()` raises after the upstream response is already fully buffered, so callers wait through the whole generation and then receive a 502 instead of the completed response. This adds explicit `server_tool_use` support in the buffered-to-SSE replay path, while keeping the existing rejection for truly unsupported Anthropic block types. Closes #1806. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a `server_tool_use` branch in the Anthropic buffered-response SSE conversion loop. - Emitted the full `server_tool_use` block in `content_block_start` instead of raising during replay. - Added a focused regression that proves buffered `server_tool_use` blocks convert to SSE and round-trip with the block type intact. - Kept the existing reject-unknown test so unsupported future block types still fail loudly. - Applied the pinned Ruff formatter to three pre-existing files on the current base so the repo-wide lint job passes unchanged semantics. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_sse_thinking_blocks.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_sse_thinking_blocks.py -q 7 passed, 1 warning in 0.19s uv run ruff check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py All checks passed! uv run ruff check . All checks passed! uv run ruff format --check . 1046 files already formatted ``` ## Real Behavior Proof - Environment: Windows, project `uv` environment, focused handler-level regression. - Exact command / steps: run `tests/test_sse_thinking_blocks.py` on `origin/main` with the new `server_tool_use` regression present, then rerun the same file on this branch. - Observed result: base raises `Unsupported Anthropic content block type for SSE conversion: 'server_tool_use'`; head passes the focused file and preserves the `server_tool_use` block type through buffered SSE reconstruction, while the existing reject-unknown test still passes. - Not tested: live proxy traffic against Anthropic server-side tools. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes No changelog entry is needed for this internal handler fix. The issue suggested a broader accept-all fallback, but this PR stays intentionally narrower: it handles the proven `server_tool_use` case and keeps the existing rejection for truly unsupported Anthropic block types. The extra formatting-only diff comes from the current base failing the pinned full-repo Ruff format check. |
||
|
Vinay Gupta
|
d337e3b828
|
fix(proxy): handle streaming CCR retrieval (#1451)
## Description Fixes Anthropic-compatible streaming requests that can emit the internal `headroom_retrieve` CCR tool. When a `stream: true` request includes the CCR retrieve tool and response handling is enabled, Headroom now buffers the upstream call as `stream: false`, lets the existing CCR response handler retrieve and continue, and returns the final result as Anthropic SSE so streaming clients do not see the internal tool call. Closes #1450 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Detect direct Anthropic-compatible `stream: true` requests where `headroom_retrieve` is available and CCR response handling is enabled. - Route those requests through the existing buffered/non-stream CCR response handler, then convert the final response back to `text/event-stream`. - Fail closed with a 502 SSE error if a buffered response still contains `headroom_retrieve` after CCR handling, instead of leaking the internal tool to the client. - Preserve Anthropic `thinking`, `redacted_thinking`, signatures, and citations when converting response JSON back to SSE. - Add regression coverage for handled CCR retrieval, unused CCR tool availability, normal streaming passthrough, mixed client/CCR tool fail-closed behavior, and SSE conversion preservation. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ rtk gh pr checks 1451 --repo headroomlabs-ai/headroom CI Checks Summary: [ok] Passed: 20 [FAIL] Failed: 0 Relevant CI commands from .github/workflows/ci.yml: - ruff check . - ruff format --check . - mypy headroom --ignore-missing-imports - pytest tests scripts/tests $ rtk python3 -m py_compile headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/streaming.py tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_sse_thinking_blocks.py # passed, no output $ rtk pytest tests/test_sse_thinking_blocks.py -q Pytest: 6 passed $ MACOSX_DEPLOYMENT_TARGET=15.0 rtk uv run --python 3.13 pytest tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py -q Failed before test collection while building the local editable package: esaxx-rs build failed with fatal error: 'cstdint' file not found. ``` ## Real Behavior Proof - Environment: GitHub Actions CI on PR #1451 plus local macOS worktree `fix/1450-ccr-streaming-retrieve`. - Exact command / steps: CI ran lint, type checking, build, unit-test shards, dashboard tests, extras tests, and e2e jobs; locally ran syntax checks and the SSE conversion regression tests. - Observed result: CI passed 20 checks with 0 failures; local syntax checks passed; `tests/test_sse_thinking_blocks.py` passed with 6 tests. - Not tested: the new proxy-level regression test was not run locally because the local native extension build fails in `esaxx-rs` before proxy tests can collect; it is included in the CI-tested suite. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - Scope: this handles the direct Anthropic-compatible HTTP `/v1/messages` path. The configured Bedrock/backend streaming path does not share this CCR continuation machinery in this PR. - Documentation, CHANGELOG, code-comment, and local-full-test checklist items are N/A for this narrow bug fix or not true locally. |
||
|
chopratejas
|
148ded392a |
fix: A8 — SSE delta arms, UTF-8 buffer, phase preservation, request-id, 413
Eliminates the Python wire-format hotfix bugs gated on Phase A's
lockdown so the proxy is safe through Phase H's Python retirement.
Bugs retired:
- P0-7 / P4-44: Codex `phase` field is now explicitly preserved
through the Responses-API ↔ Chat-Completions round-trip; multi
text-part rebuild collapses to a single text part (no more
content doubling).
- P1-8: Bytes-level SSE event splitter
`parse_sse_events_from_byte_buffer`; emoji/CJK split across
chunks survive intact. Buffer is `bytearray`; UTF-8 decode happens
only AFTER the `\n\n` event terminator is located in bytes.
Invalid UTF-8 in a *complete* event raises (operator-visible
diagnostic, not silent corruption).
- P1-9: `_parse_sse_to_response` handles all delta types per
Anthropic guide §5.1: `thinking_delta`, `signature_delta`,
`citations_delta`. Block map keyed by `index` so out-of-order
events reconstruct correctly. `redacted_thinking.data` preserved.
- P4-47: Unknown Responses-API item types now log a structured
`unknown_responses_item_type` warning so operators see new
Codex item types in flight before they break.
- P5-57: Rust proxy captures upstream `request-id` (Anthropic) and
`x-request-id` (OpenAI); surfaced as `headroom-upstream-request-id`
on the response and as a tracing span field. Distinct from the
proxy's own `x-request-id`.
- P5-59: Body-too-large now returns 413 (was 400). Pre-checks
`Content-Length` and rejects without consuming the body when
present; chunked uploads still buffer-then-fail with 413.
Configurability (no hardcodes):
- HEADROOM_SSE_BUFFER_MAX_BYTES (default 1 MiB) — per-event cap.
- HEADROOM_PROXY_BODY_TOO_LARGE_STATUS (default 413) — operator
override for body-too-large status.
A7 follow-up: `_DummyAnthropicHandler._retry_request` accepts the
A3 byte-faithful kwargs (`original_body_bytes`, `body_mutated`,
`mutation_reasons`, `request_id`, `forwarder_name`, `path_for_log`)
so the existing 20 backpressure tests stay green against the real
handler signature.
The project-wide grep
git grep 'errors="ignore"\|errors="replace"' headroom/proxy/handlers/ headroom/ccr/
returns nothing; the single remaining lossy-decode site (response-
body diagnostics, not SSE) routes through `safe_decode_for_logging`
in `headroom/proxy/helpers.py`.
Tests:
- tests/test_sse_thinking_blocks.py (4 tests)
- tests/test_sse_utf8_split.py (3 tests)
- tests/test_proxy_responses_phase_preservation.py (4 tests)
- crates/headroom-proxy/tests/integration_request_id.rs (2 tests)
- crates/headroom-proxy/tests/integration_body_size.rs (2 tests)
|