akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
1872 commits 410 branches 169 tags 142 MiB
Commit graph

2 commits

Author SHA1 Message Date
chopratejas
967b0db439 fix: B1 — retire ICM, RollingWindow, scoring, relevance + dependents 
Phase B step 1 of the live-zone-only realignment. Removes ~10K LOC of
"drop messages from history" machinery that became unreachable after
PR-A1 made `/v1/messages` a passthrough on the proxy. Live-zone-only
compression (PR-B2..B7) operates on content blocks within messages;
message-list mutation no longer happens in the pipeline.

Python deletes:
- headroom/transforms/intelligent_context.py (1077 LOC)
- headroom/transforms/rolling_window.py (395 LOC)
- headroom/transforms/progressive_summarizer.py (508 LOC)
- headroom/transforms/scoring.py (459 LOC)
- headroom/transforms/tool_crusher.py (338 LOC)
- 5 corresponding tests/test_transforms/* and tests/test_proxy_intelligent_context.py

Rust deletes:
- crates/headroom-core/src/context/* (manager, config, workspace,
  candidate, ccr_drop, strategy/, mod) + safety.rs replaced
- crates/headroom-core/src/scoring/* (mod, score, scorer, traits, weights)
- MessageScorerComparator from crates/headroom-parity (PR #338/#343
  becomes deletable; sunk cost stays sunk)
- 13 message_scorer fixtures + record_message_scorer.py

Rust adds (move + rewrite):
- crates/headroom-core/src/transforms/safety.rs — `tool_pair_indices`
  preserves the OpenAI/Anthropic tool_use ↔ tool_result pairing rule
  the live-zone dispatcher (PR-B2) needs. No IcmConfig dependency.

Surface refactors:
- HeadroomConfig: drop `tool_crusher`, `rolling_window`,
  `intelligent_context` fields; hoist `output_buffer_tokens` to top
  level (used by client.py).
- ProxyConfig: drop `intelligent_context*` fields.
- `headroom wrap` proxy server: retire IntelligentContextManager
  and RollingWindow imports + branch; pipeline is CacheAligner →
  ContentRouter (smart_routing) or CacheAligner → SmartCrusher
  (legacy).
- CLI: drop `--no-intelligent-context`, `--no-intelligent-scoring`,
  `--no-compress-first` flags.
- LangChain memory integration: rename `_apply_rolling_window` →
  `_apply_compression`, drop RollingWindowConfig dep. Threshold is
  now advisory — B6 will rework the contract.
- TransformPipeline.create_pipeline now takes only cache_aligner_config.
- headroom/__init__.py + headroom/transforms/__init__.py: strip
  exports of deleted symbols.

Bug fixes uncovered by full pytest sweep:
- providers/copilot/wrap.py: `environ or os.environ` collapsed
  empty-dict to falsy → callers passing `environ={}` accidentally
  pulled from os.environ. Use `environ if environ is not None else
  os.environ`.

Test correctness fixes:
- _DummyAnthropicHandler._retry_request gains **_kwargs to match
  the real handler signature post-A8.
- test_ws_http_fallback extracts JSON from `content=` (post-A3
  byte-faithful) rather than the obsolete `json=` kwarg.
- test_ccr_response_handler_extra fixture joins SSE events with
  `\n\n` per spec (post-A8 byte-buffer parser requirement).
- test_proxy_responses_phase_preservation: capture via direct
  handler attached to the named logger, so the assertion is
  order-independent (proxy `_setup_file_logging` flips
  `headroom.propagate=False` once any earlier test triggers it).
- conftest.py autouse fixture resets `headroom.propagate=True`
  before each test as a defensive measure for the same pollution.
- test_wrap_copilot_translated_backend_still_requires_byok:
  monkeypatch.delenv every provider key so the BYOK error
  actually fires.
- test_native_installers: skip when system bash < 4.3 (macOS ships 3.2).
- TestGeminiEmbedContent / TestGeminiBatchEmbedContents:
  pytest.mark.skip — proxy currently has no :embedContent route;
  feature gap, not regression.

Acceptance:
- cargo build --workspace + cargo clippy + cargo fmt --check: green.
- cargo test --workspace --exclude headroom-py: 777 passed.
- pytest: 4892 passed, 240 skipped, 0 failed.
- git grep returns only intentional comments referencing the deletion.

Per-PR-B1 plan: REALIGNMENT/04-phase-B-live-zone.md.
 2026-05-02 12:23:17 -07:00
chopratejas
95fd6d8688 Add multi-provider batch API support with CCR post-processing 
This commit adds comprehensive batch API support for all three major LLM
providers (Anthropic, OpenAI, Google/Gemini) with integrated CCR
(Compress-Cache-Retrieve) functionality for asynchronous batch processing.

## Batch CCR Post-Processing Architecture

When batch APIs are used, responses are processed asynchronously. If the
model calls the CCR retrieval tool (`headroom_retrieve`) within a batch
response, the system now handles this automatically:

1. **Batch Submit**: Request context (messages, tools, model) is stored
   in BatchContextStore keyed by batch_id
2. **Batch Results**: When results are retrieved, CCR tool calls are
   detected in the responses
3. **Continuation**: For each CCR tool call, the system executes local
   retrieval and makes a continuation API call to complete the response
4. **Result Update**: The batch result is updated with the complete
   response, transparent to the caller

## New Components

- `headroom/ccr/batch_store.py`: TTL-based context storage for batch
  requests, enabling CCR retrieval during result processing
- `headroom/ccr/batch_processor.py`: Processes batch results, detects
  CCR tool calls across all provider formats, executes continuations

## Provider Support

### Anthropic
- POST /v1/messages/batches (create with compression)
- GET /v1/messages/batches (list)
- GET /v1/messages/batches/{id} (status)
- GET /v1/messages/batches/{id}/results (with CCR post-processing)

### OpenAI
- POST /v1/batches (create)
- GET /v1/batches (list)
- GET /v1/batches/{id} (status)
- Batch file upload/download support

### Google/Gemini
- Native API support: /v1beta/models/{model}:generateContent
- Batch API: /v1beta/models/{model}:batchGenerateContent
- Token counting: /v1beta/models/{model}:countTokens
- OpenAI-compatible endpoint support

## CCR Enhancements

- Added Google/Gemini format support to response_handler.py
- Extended tool_injection.py with multiple marker patterns for different
  compressors (SmartCrusher, TextCompressor, LogCompressor, etc.)
- Added Google functionCall/functionResponse handling

## Proxy Server Updates

- Added Gemini native API handlers alongside OpenAI-compatible endpoints
- Integrated batch context storage on submission
- Added batch result processing with CCR continuation
- Rate limiting and metrics tracking for all providers

## Test Coverage

Added comprehensive integration tests (all skip gracefully without API keys):
- test_proxy_batch_integration.py: Anthropic and OpenAI batch APIs
- test_proxy_gemini_integration.py: Gemini via OpenAI-compatible endpoint
- test_proxy_gemini_native_integration.py: Gemini native API
- test_proxy_count_tokens_integration.py: Token counting endpoint
- test_proxy_openai_responses_integration.py: OpenAI responses API
- test_proxy_passthrough_integration.py: Passthrough endpoints

## Compression Results (Real API Testing)

- Token savings: 83-98% on tool result content
- CCR tool injection: Working across all providers
- Model behavior: OpenAI gpt-4o-mini successfully called headroom_retrieve
  when presented with compressed data, proving end-to-end CCR functionality
 2026-01-24 11:41:18 -08:00