akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
2630 commits 410 branches 169 tags 142 MiB
Commit graph

3 commits

Author SHA1 Message Date
Abhay Singh
d6d121e399
fix(ccr): re-inject headroom_retrieve when history references it on the sessionless path (#2440) (#2533) 
## Description

Fixes #2440. `apply_session_sticky_ccr_tool` bypasses the
`SessionCcrTracker` when `session_id` is `None` (WS / pre-session paths)
and drives injection purely off the per-turn
`has_compressed_content_this_turn` flag:

```python
if not session_id:
    if not has_compressed_content_this_turn:
        ...  # skip: tool NOT re-declared
        return tools_out, False
    ...
```

If an earlier turn emitted a `headroom_retrieve` tool_use into history
but the current turn produced no fresh compression marker, the tool
definition is not re-declared in `tools`, while the forwarded history
still references it. The provider then rejects the whole request:

```
API Error: 400 Tool reference 'headroom_retrieve' not found in available tools.
```

Without a session the tracker can't remember the earlier turn's CCR, so
this is unique to the sessionless path.

## Fix

Add `history_references_ccr_tool(messages)` which detects an existing
`headroom_retrieve` call in the forwarded messages — both the Anthropic
assistant `tool_use` content block and the OpenAI assistant
`tool_calls[].function.name` shapes, fully null-guarded. On the
sessionless path, injection now fires when
`has_compressed_content_this_turn` **or** history already references the
tool, so the definition is re-declared and the request validates. The
decision is logged as a new `inject_history_reference` outcome. Both
handlers pass the signal computed from `optimized_messages` (the bytes
actually forwarded). Behavior with a real `session_id` (the sticky
tracker path) is unchanged.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/helpers.py`: add `history_references_ccr_tool`; add a
`history_has_ccr_reference` parameter to `apply_session_sticky_ccr_tool`
and OR it into the sessionless injection decision.
- `headroom/proxy/tool_injection_logging.py`: add the
`inject_history_reference` decision literal.
- `headroom/proxy/handlers/anthropic.py`,
`headroom/proxy/handlers/openai.py`: pass
`history_references_ccr_tool(optimized_messages)` into the sticky-tool
call.
- `tests/test_ccr_tool_always_on.py`: regressions for the detector (both
provider shapes + malformed inputs) and for sessionless re-injection
when history references the tool.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_ccr_tool_always_on.py -q
14 passed

# with just the `or history_has_ccr_reference` condition reverted, the new
# sessionless re-injection test fails (tool not injected -> would 400)

$ uvx ruff@0.15.17 check headroom/proxy/helpers.py headroom/proxy/tool_injection_logging.py headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py tests/test_ccr_tool_always_on.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/helpers.py headroom/proxy/tool_injection_logging.py
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: called `history_references_ccr_tool` on
Anthropic `tool_use` and OpenAI `tool_calls` histories (plus
null/non-list shapes), and
`apply_session_sticky_ccr_tool(session_id=None,
has_compressed_content_this_turn=False,
history_has_ccr_reference=True)`; then temporarily reverted only the `or
history_has_ccr_reference` condition and re-ran the regression.
- Observed result: the detector returns `True` for both provider shapes
and `False`/no-crash for malformed input; with the fix the sessionless
call injects the tool (`was_injected=True`, tool present) even with no
fresh compression; with the condition reverted the same call returns
`was_injected=False` (the tool is dropped — exactly the 400 path). Ran
against the actual module via `tests/test_ccr_tool_always_on.py`.
- Not tested: a live sessionless multi-turn WS request reproducing the
upstream 400 end to end.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
 2026-08-13 11:45:51 -05:00
Tejas Chopra
c2fc4d3753
fix(ccr): make headroom_retrieve a hash-only full-content lookup (#1532) 
The optional `query` parameter on headroom_retrieve routed retrieval
through CompressionStore.search(), which BM25-scored the items inside a
single cached blob and dropped everything below a 0.3 relevance floor.
On small per-blob corpora with conversational queries this returned an
empty result the large majority of the time, so the LLM saw "nothing
found" for content that was actually present — pushing users to turn
compression off entirely.

Retrieval is fundamentally a hash lookup (this already matches the Rust
proxy's CCR store, which is put/get only — "no BM25 search"). Remove the
query/search path end to end and always return the full original
content:

Core (Python proxy):
- tool schemas (anthropic/openai/google) drop the `query` property
- parse_tool_call returns the hash (str | None) instead of (hash, query)
- response handler, proxy POST/GET/tool-call handlers, the MCP retrieve
tool, and the streaming feedback recorders retrieve by hash only
- proactive context-tracker expansion always restores full content
- delete CompressionStore.search() and its BM25 machinery (the bm25
module stays — it is still used by relevance/)
- CCRToolCall.query, CCRToolResult.was_search, and
ExpansionRecommendation.expand_full/search_query are removed

Plugins (advertised a now-defunct query param to the LLM):
- hermes (Python), openclaw + opencode (TypeScript) retrieve tools drop
`query` from their schemas, signatures, request URLs, and tests

Benchmarks/docs:
- ccr_regression + adversarial benchmarks switch from store.search() to
full hash retrieval (search input-injection tests repurposed to the
hash, the only remaining input surface)
- wiki/ARCHITECTURE.md, wiki/ccr.md, docs/content/docs/ccr.mdx,
config.py and store docstrings updated to describe hash-only retrieval

Tests updated to assert full-content retrieval and guard the removed
surface; the full CCR/proxy/store/TOIN suite passes. ruff + mypy clean.

## Description

<!-- Briefly explain the change and why it is needed. -->

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- 

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
# Paste relevant command output or artifact links here
```

## Real Behavior Proof

- Environment:
- Exact command / steps:
- Observed result:
- Not tested:

## Review Readiness

- [ ] I have performed a self-review
- [ ] This PR is ready for human review

## Checklist

- [ ] My code follows the project's style guidelines
- [ ] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

Add screenshots to help explain your changes.

## Additional Notes

<!-- Mention any N/A checklist items, tradeoffs, follow-ups, or
maintainer context. -->
 2026-06-28 10:32:43 -07:00
chopratejas
00902b8fea fix: B7 — CCR hardening: persistent backends + always-on tool 
P2-25, P2-26: CCR (Compress-Cache-Retrieve) used an in-memory store
that fragmented across uvicorn workers and was wiped on restart, and
the `headroom_retrieve` tool was registered/unregistered per-request
based on whether the latest body happened to contain compression
markers — every flip busted the prompt cache. Both are sticky
side-channels: once a session has done CCR, the tool list bytes and
the retrieval store must stay stable. This PR fixes both.

Rust:
* Split `ccr.rs` into `ccr/` with `backends/` submodule
  (`in_memory.rs`, `sqlite.rs`, `redis.rs` cfg-gated).
* `SqliteCcrStore` (production default): WAL mode, prepared upsert,
  lazy TTL purge on read, persistent across worker restarts and
  shareable across workers on the same host via SQLite file locking.
* `RedisCcrStore` (cfg-gated behind `feature = "redis"`): SETEX with
  startup PING smoke-test, no key-prefix collision risk, no sticky
  session required at the LB.
* `CcrBackendConfig::{InMemory, Sqlite, Redis}` + `from_config(...)`
  factory — every init failure surfaces (no silent fallback per
  `feedback_no_silent_fallbacks.md`).
* `ccr::compute_key` (BLAKE3 → first 24 hex chars) and
  `ccr::marker_for("HASH") -> "<<ccr:HASH>>"` centralize the hash +
  marker format; one definition for the live-zone dispatcher and the
  Python regex (`headroom/ccr/tool_injection.py:211`).
* `compress_anthropic_live_zone_with_ccr` accepts
  `Option<&dyn CcrStore>`. When wired, every accepted compression
  puts the original bytes into the backend and appends `<<ccr:HASH>>`
  to the compressed string. The token-validation gate runs on the
  marker-augmented string so the `compressed_tokens >=
  original_tokens` rejection stays honest.

Python:
* `SessionCcrTracker` + `apply_session_sticky_ccr_tool` mirror the
  PR-A7 `SessionToolTracker` / `apply_session_sticky_memory_tools`
  pattern: once a session has done CCR, every subsequent request
  injects the recorded golden tool-definition bytes. Tool list bytes
  are byte-stable across turns (snapshot test pins them).
* `headroom/ccr/tool_injection.py::inject_tool_definition` accepts a
  new `session_has_done_ccr` kwarg per the PR-B7 spec change at line
  302-328. The legacy per-request path stays intact for callers that
  don't yet thread a session id (e.g. Google handler).
* Anthropic + OpenAI handlers route their CCR tool-list updates
  through `apply_session_sticky_ccr_tool`, keyed off the existing
  `session_tracker_store.compute_session_id(...)` plumbing.

Backend selection model: `CcrBackendConfig::Sqlite { path }` is the
production default — single host, persistent, multi-worker safe with
sticky session. `CcrBackendConfig::Redis { url }` is the multi-host
scale-out option — no stickiness needed. `InMemory` is for tests
and single-worker dev only. RUST_DEV.md "Multi-worker deployment —
CCR fragmentation" rewritten around this matrix.

Tests:
* `crates/headroom-core/tests/ccr_backends.rs` — 7 tests covering
  SQLite round-trip, TTL purge, proxy-restart survival, cross-backend
  byte-equal keys, `from_config` paths, and the no-redis-feature
  loud-failure check (+ 2 redis tests gated behind the feature).
* `crates/headroom-core/tests/live_zone_ccr.rs` — confirms
  `<<ccr:HASH>>` marker injection, store population, and
  no-marker-when-no-store invariants end-to-end.
* `tests/test_ccr_tool_always_on.py` — 12 tests pinning the
  always-on behaviour, session/provider isolation, LRU bound, no-
  session-id fallback, and (per-acceptance-criterion) the byte-stable
  tool-definition snapshot.

Per-PR-B7 plan: REALIGNMENT/04-phase-B-live-zone.md.
 2026-05-02 16:52:33 -07:00