Commit graph

1 commit

Author SHA1 Message Date
Parideboy
ce8ce8313f
fix(ccr): resolve <<ccr:...>> markers inline when no retrieve-tool path exists (#2512)
## Description

Fixes #2509. CCR marker resolution today depends entirely on the model
calling `headroom_retrieve` back a tool-call round-trip. Callers with no
such round-trip (e.g. Headroom running as a LiteLLM guardrail/proxy hop,
per the issue's repro) never get an offered path to redeem a marker, so
raw `<<ccr:HASH,type,size>>` text leaks straight to the agent.

This adds an explicit, opt-in fallback: `--ccr-inline-resolve` /
`HEADROOM_CCR_INLINE_RESOLVE`. When set, the proxy resolves markers
directly from the compression store on the response path instead of
waiting for a tool call. Off by default, guessing "this caller can't use
tools" is fragile, so operators opt in explicitly for guardrail/proxy
deployments.

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change
- [ ] Documentation update

## Changes Made

- `headroom/ccr/marker_resolution.py` (new): `resolve_markers_in_text` /
`resolve_markers_in_response` regex-match `<<ccr:HASH,...>>`, look up
the hash in `CompressionStore`, splice the original content back in. A
miss (expired/evicted hash) leaves the marker in place with the miss
reason appended, since there's no tool-call round-trip to report it back
to the model.
- `headroom/proxy/models.py`: `ProxyConfig.ccr_resolve_markers_inline:
bool = False`.
- `headroom/cli/proxy.py`: `--ccr-inline-resolve` flag /
`HEADROOM_CCR_INLINE_RESOLVE` env, wired into `ProxyConfig`.
- `headroom/proxy/handlers/anthropic.py`,
`headroom/proxy/handlers/openai.py`:
call `resolve_markers_in_response` on the finalized response JSON, right
after existing CCR tool-call handling, at all three non-streaming
response sites (Anthropic Messages, OpenAI Chat Completions backend
path, OpenAI Responses API). Streaming responses are out of scope for
this PR, tracked as follow-up, noted in the module docstring's scope.

## Testing

- [x] Added new tests
- [x] All tests pass locally

```
$ python -m pytest tests/test_ccr_marker_resolution.py -q
============================= test session starts =============================
collected 6 items

tests\test_ccr_marker_resolution.py ......                               [100%]

============================== 6 passed in 0.45s ==============================

$ python -m pytest tests/test_ccr.py tests/test_ccr_response_handler.py tests/test_ccr_response_handler_extra.py tests/test_ccr_response_handler_openai_responses.py tests/test_proxy/test_openai_responses_ccr.py tests/test_proxy/test_anthropic_ccr_raise.py -q
======================= 83 passed, 1 warning in 34.50s ========================
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.11, local headroom repo
(`G:\Programmi Aggiuntivi\headroom`)
- Exact command / steps: `python -m pytest
tests/test_ccr_marker_resolution.py tests/test_ccr.py
tests/test_ccr_response_handler.py
tests/test_ccr_response_handler_extra.py
tests/test_ccr_response_handler_openai_responses.py
tests/test_proxy/test_openai_responses_ccr.py
tests/test_proxy/test_anthropic_ccr_raise.py -q`
- Observed result: 89 passed, 0 failed (6 new + 83 existing CCR tests,
no regressions). `ruff check`, `ruff format --check`, and `mypy
--ignore-missing-imports` all clean on every changed/new file.
- Not tested: the actual Docker Compose / LiteLLM guardrail deployment
from the issue's repro steps (no such environment available here);
streaming response paths (out of scope, see Changes Made).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-12 00:18:43 -05:00