docs: sync README + benchmarks with code (drop retired IntelligentContext/RollingWindow) (#1545)

## Description

Sync the docs with the code after the live-zone realignment. The
`IntelligentContextManager` (ICM), `RollingWindow`, and scoring modules
were deleted in PR #350 (May 2026), but the README and benchmark
docstrings still advertised them as live, and an example still imported
the deleted module (broken on run). This fixes the README + benchmarks
and removes the dead example.

I validated the README against the code with three parallel
static-analysis sub-agents (features/architecture,
CLI/extras/wrap-matrix, public API/integrations). Most of the README
checked out accurate; only the items below were stale/wrong.

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- README: removed the `IntelligentContext` bullet and
`IntelligentContext / RollingWindow` from the transforms list (both
deleted in PR #350).
- README: standardized `Kompress-base` -> `Kompress-v2-base` to match
the HF model id `chopratejas/kompress-v2-base` and the existing badges
(diagram re-aligned).
- README: corrected the CodeCompressor language list to match the
`CodeLanguage` enum (added TS, C, Perl).
- README: softened the unanchored "6 algorithms" tagline to
"content-aware compressors".
- README: Cortex Code is library-mode only — there is no `headroom wrap
cortex`, so the compatibility-matrix row no longer shows a wrap
checkmark.
- Deleted `examples/test_intelligent_context_toin_ccr.py` — it imported
the deleted `IntelligentContextManager` (ImportError on run) and is
unreferenced.
- Removed stale `RollingWindow` mentions from benchmark
docstrings/comments (`benchmarks/__init__.py`, `bench_transforms.py`,
`bench_latency.py`, `scenarios/conversations.py`); the accurate PR-B1
retirement comment is kept.

## Testing

- [ ] Unit tests pass (`pytest`) — N/A, docs/docstring + example
deletion only
- [x] Linting passes — `ruff check` clean on all changed benchmark files
- [ ] Type checking passes — N/A (no type-relevant changes)
- [ ] New tests added — N/A
- [x] Manual testing performed — see Real Behavior Proof

### Test Output

```text
$ ruff check benchmarks/__init__.py benchmarks/bench_transforms.py benchmarks/bench_latency.py benchmarks/scenarios/conversations.py
All checks passed!

# stale refs remaining in README/benchmarks (excluding accurate retirement notes):
$ grep -rn "IntelligentContext|RollingWindow|Kompress-base" README.md benchmarks/ | grep -v retire
(only benchmarks/bench_transforms.py:362 — the accurate PR-B1 retirement comment)

# deleted example is unreferenced anywhere:
$ grep -rn "test_intelligent_context_toin_ccr" --include=*.md --include=*.yml --include=*.py .
(no hits)
```

## Real Behavior Proof

- Environment: macOS (darwin, arm64), Python 3.12 `.venv`, ruff 0.14.x,
repo at branch `docs/sync-readme-with-code` off latest `main`.
- Exact command / steps: (1) three parallel sub-agents
grep/Read-validated README claims vs `headroom/`, `pyproject.toml`,
`sdk/typescript/`; (2) directly verified each flagged mismatch
(`CodeLanguage` enum, `HF_MODEL_ID`, absence of
`IntelligentContext`/`RollingWindow` classes); (3) confirmed the example
imports a deleted module and is unreferenced; (4) `ruff check` on
changed benchmark files; (5) re-grepped README + benchmarks for any
remaining stale refs.
- Observed result: README and benchmark docstrings now match the code;
the only surviving `RollingWindow` string is the accurate retirement
comment; the broken example is removed; ruff passes; the ASCII
architecture diagram still aligns after the `Kompress-v2-base` rename.
- Not tested: rendering of the README on GitHub/PyPI (text-only change);
the separate `docs/content/` and `wiki/` doc sets (see Additional Notes
— out of scope for this PR).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective — N/A
(docs/example cleanup)
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md — N/A (Release Please
auto-generates from the conventional commit)

## Additional Notes

**Larger related finding (NOT in this PR):** the published docs site
(`docs/content/docs/*.mdx`) and the `wiki/*.md` set still document
`IntelligentContextManager`, `RollingWindow`, `RollingWindowConfig`,
`IntelligentContextConfig`, and `ScoringWeights` as live API — with
`from headroom import RollingWindow` / `from headroom.transforms import
IntelligentContextManager` code examples that would `ImportError`. It is
half-migrated (a couple of `.mdx` files already note "removed in 0.9.x"
while neighbors still teach it as current). This is ~15 files and the
fixes require rewriting examples to the live-zone model, not just
deletions — recommended as a focused follow-up PR rather than bundling
it here.
This commit is contained in:
Tejas Chopra 2026-06-28 22:36:41 -07:00 committed by GitHub
parent a7d3360a05
commit 10251b65ca
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
23 changed files with 98 additions and 988 deletions

View file

@ -8,7 +8,7 @@
The context compression layer for AI agents
</pre></div>
<p align="center"><strong>6095% fewer tokens · library · proxy · MCP · 6 algorithms · local-first · reversible</strong></p>
<p align="center"><strong>6095% fewer tokens · library · proxy · MCP · content-aware compressors · local-first · reversible</strong></p>
<p align="center">
<a href="https://github.com/chopratejas/headroom/actions/workflows/ci.yml"><img src="https://github.com/chopratejas/headroom/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
@ -67,7 +67,7 @@ Headroom compresses everything your AI agent reads — tool outputs, logs, RAG c
│ CacheAligner → ContentRouter → CCR │
│ ├─ SmartCrusher (JSON) │
│ ├─ CodeCompressor (AST) │
│ └─ Kompress-base (text, HF)
│ └─ Kompress-v2-base (text, HF) │
│ │
│ Cross-agent memory · headroom learn · MCP │
└────────────────────────────────────────────────────┘
@ -77,7 +77,7 @@ Headroom compresses everything your AI agent reads — tool outputs, logs, RAG c
```
- **ContentRouter** — detects content type, selects the right compressor
- **SmartCrusher / CodeCompressor / Kompress-base** — compress JSON, AST, or prose
- **SmartCrusher / CodeCompressor / Kompress-v2-base** — compress JSON, AST, or prose
- **CacheAligner** — stabilizes prefixes so provider KV caches actually hit
- **CCR** — stores originals locally; LLM calls `headroom_retrieve` if it needs them
@ -204,7 +204,7 @@ shows an **Output Tokens Saved** card next to input compression, labelled
| Goose | ✅ | starts proxy + launches |
| OpenHands | ✅ | starts proxy + launches |
| Mistral Vibe | ✅ | starts proxy + launches |
| Cortex Code | ✅ | 6065% savings · library mode |
| Cortex Code | Library only | 6065% savings (library mode; no `wrap`) |
Any OpenAI-compatible client works via `headroom proxy`. MCP-native: `headroom mcp install`.
Undo durable wrapping with `headroom unwrap <tool>` (supports: `claude`, `copilot`, `codex`, `opencode`, `openclaw`).
@ -273,11 +273,10 @@ Platform support note: macOS auth reuse via Copilot CLI Keychain storage has bee
<summary><b>What's inside</b></summary>
- **SmartCrusher** — universal JSON: arrays of dicts, nested objects, mixed types.
- **CodeCompressor** — AST-aware for Python, JS, Go, Rust, Java, C++.
- **Kompress-base** — our HuggingFace model, trained on agentic traces.
- **CodeCompressor** — AST-aware for Python, JS/TS, Go, Rust, Java, C/C++, Perl.
- **Kompress-v2-base** — our HuggingFace model, trained on agentic traces.
- **Image compression** — 4090% reduction via trained ML router.
- **CacheAligner** — stabilizes prefixes so Anthropic/OpenAI KV caches actually hit.
- **IntelligentContext** — score-based context fitting with learned importance.
- **CCR** — reversible compression; LLM retrieves originals on demand.
- **Cross-agent memory** — shared store, agent provenance, auto-dedup.
- **SharedContext** — compressed context passing across multi-agent workflows.
@ -292,7 +291,7 @@ Headroom exposes one stable request lifecycle across `compress()`, the SDK, and
`Setup``Pre-Start``Post-Start``Input Received``Input Cached``Input Routed``Input Compressed``Input Remembered``Pre-Send``Post-Send``Response Received`
- **Transforms** do the work: CacheAligner, ContentRouter, SmartCrusher, CodeCompressor, Kompress-base, IntelligentContext / RollingWindow.
- **Transforms** do the work: CacheAligner, ContentRouter, SmartCrusher, CodeCompressor, Kompress-v2-base.
- **Pipeline extensions** observe or customize lifecycle stages via `on_pipeline_event(...)`.
- **Compression hooks** sit alongside the canonical lifecycle as an additional extension seam.
- **Proxy extensions** remain the server/app integration seam for ASGI middleware, routes, and startup policy.
@ -325,7 +324,7 @@ npm install headroom-ai # TypeScript / Node
docker pull ghcr.io/chopratejas/headroom:latest
```
Granular extras: `[proxy]`, `[mcp]`, `[ml]` (Kompress-base), `[code]`, `[memory]`, `[vector]` (optional HNSW backend — needs a C++ toolchain, not in `[all]`), `[relevance]`, `[image]`, `[agno]`, `[langchain]`, `[evals]`, `[pytorch-mps]` (Apple-GPU memory-embedder offload — set `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps`). Requires **Python 3.10+**.
Granular extras: `[proxy]`, `[mcp]`, `[ml]` (Kompress-v2-base), `[code]`, `[memory]`, `[vector]` (optional HNSW backend — needs a C++ toolchain, not in `[all]`), `[relevance]`, `[image]`, `[agno]`, `[langchain]`, `[evals]`, `[pytorch-mps]` (Apple-GPU memory-embedder offload — set `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps`). Requires **Python 3.10+**.
> **Note**: `[all]` covers the core stack but excludes framework adapters. Install them separately: `pip install "headroom-ai[langchain]"` (also `[agno]`, `[strands]`, `[anyllm]`, `[bedrock]`).

View file

@ -16,7 +16,6 @@ Usage:
Performance Targets:
- SmartCrusher: < 10ms for 1000 items
- CacheAligner: < 1ms for date extraction
- RollingWindow: < 5ms for 200 turns
- BM25Scorer: < 1ms for 100 items
- HybridScorer: < 50ms for 100 items (with embeddings)
"""

View file

@ -586,7 +586,7 @@ def generate_scenarios(content_types: list[str] | None = None) -> list[Scenario]
msgs = generate_agentic_conversation(
turns=turns, tool_calls_per_turn=2, items_per_tool_response=items
)
# Set a model_limit that forces IntelligentContext to kick in
# Set a model_limit large enough to exercise compression on big agentic contexts
limit = max(50_000, turns * 2_000)
scenarios.append(
Scenario(

View file

@ -3,7 +3,6 @@
This module contains performance benchmarks for Headroom transforms:
- SmartCrusher: Statistical tool output compression
- CacheAligner: Cache-aligned prefix optimization
- RollingWindow: Token budget management
Performance Targets:
SmartCrusher:
@ -15,10 +14,6 @@ Performance Targets:
- Date extraction: < 1ms
- Hash computation: < 0.5ms
RollingWindow:
- 50 turns: < 5ms
- 200 turns: < 20ms
Run with:
pytest benchmarks/bench_transforms.py --benchmark-only -v
"""
@ -374,7 +369,7 @@ class TestTransformPipelineBenchmarks:
"""Benchmarks for full transform pipeline.
Tests the complete flow:
CacheAligner -> SmartCrusher -> RollingWindow
CacheAligner -> SmartCrusher
Expected performance:
- Simple conversation: < 5ms

View file

@ -3,8 +3,8 @@
This module provides generators for realistic conversation patterns that
exercise Headroom transforms:
- Agentic conversations: Multi-turn with tool calls (SmartCrusher, RollingWindow)
- RAG conversations: Large context injection (CacheAligner, RollingWindow)
- Agentic conversations: Multi-turn with tool calls (SmartCrusher)
- RAG conversations: Large context injection (CacheAligner)
These generators produce conversations that mirror real-world usage patterns
from production agentic systems.

View file

@ -302,25 +302,9 @@ config = CacheAlignerConfig(
</Tab>
</Tabs>
### RollingWindowConfig
### Context management
<Tabs groupId="lang" items={['TypeScript', 'Python']}>
<Tab value="TypeScript">
<TypeTable type={{
enabled: { type: 'boolean', description: 'Enable/disable rolling window' },
keepSystem: { type: 'boolean', description: 'Never drop system messages' },
keepLastTurns: { type: 'number', description: 'Always keep last N turns' },
outputBufferTokens: { type: 'number', description: 'Reserve tokens for model output' },
}} />
</Tab>
<Tab value="Python">
Context management is now automatic. Headroom handles rolling window behavior internally based on the model's context limit and the `headroom_keep_turns` / `headroom_output_buffer_tokens` per-request overrides. The `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes no longer exist as public API.
</Tab>
</Tabs>
Context management is now handled automatically inside the pipeline (live-zone-only compression). Headroom never drops messages from the conversation history; it compresses only the newest content blocks (latest user message, latest tool result) and keeps the cache hot zone — system prompt, tools, and older turns — untouched. Use the `headroom_keep_turns` / `headroom_output_buffer_tokens` per-request overrides to tune behavior. The `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes are no longer part of Headroom.
### HeadroomConfig
@ -331,10 +315,8 @@ Context management is now automatic. Headroom handles rolling window behavior in
storeUrl: { type: 'string', description: 'Storage URL for metrics database' },
defaultMode: { type: 'HeadroomMode', description: 'Default compression mode' },
modelContextLimits: { type: 'Record<string, number>', description: 'Override context limits per model' },
toolCrusher: { type: 'ToolCrusherConfig', description: 'Tool crusher configuration' },
smartCrusher: { type: 'SmartCrusherConfig', description: 'Smart crusher configuration' },
cacheAligner: { type: 'CacheAlignerConfig', description: 'Cache aligner configuration' },
rollingWindow: { type: 'RollingWindowConfig', description: 'Rolling window configuration' },
cacheOptimizer: { type: 'CacheOptimizerConfig', description: 'Cache optimizer configuration' },
ccr: { type: 'CCRConfig', description: 'CCR (Compress-Cache-Retrieve) configuration' },
prefixFreeze: { type: 'PrefixFreezeConfig', description: 'Prefix freeze configuration' },
@ -353,7 +335,7 @@ from headroom import HeadroomConfig
config = HeadroomConfig()
config.smart_crusher.min_tokens_to_crush = 100
config.cache_aligner.enabled = True
# Note: rolling_window was removed in 0.9.x — use headroom_keep_turns per-request instead
# Note: rolling_window has been removed — use headroom_keep_turns per-request instead
```
</Tab>
@ -556,15 +538,6 @@ aligner = CacheAligner()
result = aligner.align(messages)
```
### RollingWindow
```python
from headroom import RollingWindow
window = RollingWindow(config)
result = window.apply(messages, max_tokens=100000)
```
### TransformPipeline
```python
@ -573,7 +546,6 @@ from headroom import TransformPipeline
pipeline = TransformPipeline([
SmartCrusher(),
CacheAligner(),
RollingWindow(),
])
result = pipeline.transform(messages)

View file

@ -103,7 +103,7 @@ Compression pays for itself in latency for 11 of 12 tested scenarios against Cla
| `text_compressor` | 32.0ms | 576ms | Text compression (Kompress ONNX) |
| `initial_token_count` | 2.9ms | 16ms | Token counting (tiktoken) |
ContentRouter accounts for 91--98% of pipeline cost on average. CacheAligner and RollingWindow are sub-millisecond.
ContentRouter accounts for 91--98% of pipeline cost on average. CacheAligner is sub-millisecond.
## Production Telemetry

View file

@ -159,7 +159,7 @@ Check the effective setting at `/v1/retrieve/stats` under
## Message-level CCR
> **Retired:** The "Message-level CCR via IntelligentContext" feature (where `IntelligentContext` would store dropped messages in CCR with a retrieval marker) was part of the `IntelligentContextConfig` API that was removed in 0.9.x. Context management is now handled automatically by the pipeline without a separate configurable IntelligentContext stage. Tool-output CCR via SmartCrusher and ContentRouter remains fully supported.
> **Retired:** The "Message-level CCR via IntelligentContext" feature (where `IntelligentContext` would store dropped messages in CCR with a retrieval marker) was part of the `IntelligentContextConfig` API that has since been removed. Context management is now handled automatically by the pipeline without a separate configurable IntelligentContext stage. Tool-output CCR via SmartCrusher and ContentRouter remains fully supported.
## CCR-enabled components

View file

@ -184,7 +184,7 @@ response = client.chat.completions.create(
)
```
The `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes were removed in 0.9.x. Context management now happens automatically inside the pipeline.
The `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes are no longer part of Headroom. Context management now happens automatically inside the pipeline (live-zone-only compression).
## Proxy Configuration

View file

@ -1,55 +1,24 @@
---
title: Context Management
description: Intelligent importance-based context management that scores messages by learned patterns, with rolling window fallback and output buffer reservation.
description: Automatic live-zone-only context management that compresses the newest content blocks while preserving the provider cache hot zone.
---
When conversations grow beyond a model's context window, Headroom decides which messages to keep and which to drop. Instead of naively removing the oldest messages, the pipeline scores every message by learned importance and drops the least valuable ones first.
Context management is now handled automatically inside the pipeline (live-zone-only compression). Headroom **never drops messages** from the conversation history and does not do position-based or score-based context management.
## How It Works
The context management stage analyzes your conversation, assigns an importance score to each message, and removes low-scoring messages until the conversation fits within the token budget.
Headroom compresses only the **newest content blocks** — the latest user message and the latest tool result / tool output. Compression is type-aware and reversible via [CCR](/docs/ccr), so the LLM can retrieve the original content on demand.
Dropped messages are not lost -- they are stored in [CCR](/docs/ccr) for on-demand retrieval by the LLM.
The **cache hot zone** — the system prompt, tool definitions, and older turns — is never mutated. Leaving the prefix untouched preserves provider prompt caching, so cache hit rates stay stable across turns.
```
100-message conversation (50K tokens) with a 32K budget
-> Score each message by importance
-> Drop 60 lowest-scoring messages
-> Cache dropped messages in CCR (hash=def456)
-> Insert marker: "60 messages dropped, retrieve: def456"
-> Final context: 40 messages within budget
Conversation with a large latest tool result
-> Identify the live zone (newest user message + latest tool output)
-> Compress the live zone type-aware, cache original in CCR (hash=def456)
-> Insert marker: "compressed, retrieve: def456"
-> Older turns, tools, and system prompt are forwarded byte-for-byte
```
## Scoring weights
Each message receives a weighted score from six factors:
| Weight | Default | Description |
|---|---|---|
| `recency` | 0.20 | Exponential decay from the end of the conversation. Recent messages score higher. |
| `semantic_similarity` | 0.20 | Embedding cosine similarity to recent context. Messages related to the current topic score higher. |
| `toin_importance` | 0.25 | TOIN retrieval rate -- messages matching patterns that users frequently retrieve via CCR are scored higher. Learned across all users. |
| `error_indicator` | 0.15 | TOIN field semantics error detection. Messages containing error patterns (learned, not hardcoded) are preserved. |
| `forward_reference` | 0.15 | Count of later messages that reference this one. Messages that other messages depend on are kept. |
| `token_density` | 0.05 | Unique tokens divided by total tokens. Dense, information-rich messages score higher than repetitive ones. |
<Callout type="info" title="No hardcoded patterns">
Error detection does not rely on keyword matching like "error" or "fail". Instead, it uses TOIN's learned `field_semantics.inferred_type` to identify error-bearing messages -- this adapts to your specific data patterns across sessions and users.
</Callout>
Weights are automatically normalized to sum to 1.0, so you can set relative values without worrying about exact proportions.
## Rolling window fallback
If IntelligentContext is disabled or scoring data is unavailable, Headroom falls back to a **rolling window** strategy:
- Drop the oldest messages first
- Always keep the system prompt
- Always keep the last N user/assistant turns
- Drop tool calls and their responses as atomic pairs (no orphaned tool data)
This provides a safe baseline that works without any learned data.
## Protection rules
Headroom enforces several protections to ensure model output quality:
@ -111,16 +80,4 @@ response = client.chat.completions.create(
</Tab>
</Tabs>
> **Note:** The `IntelligentContextConfig`, `ScoringWeights`, and `RollingWindowConfig` classes were removed in 0.9.x. Context management is now handled automatically inside the pipeline.
## How scoring improves over time
IntelligentContext integrates with TOIN (Tool-Output Intelligence Network) to learn from real usage:
1. Messages are dropped based on current scores
2. Dropped messages are stored in CCR
3. If the LLM retrieves a dropped message, TOIN records that pattern
4. Future conversations score similar message patterns higher
5. Drop accuracy improves across all users, not just within one session
This feedback loop means the system gets smarter the more it is used. Error messages that users frequently need are automatically preserved, while verbose success messages that nobody retrieves are dropped more aggressively.
> **Note:** The `IntelligentContextConfig`, `ScoringWeights`, and `RollingWindowConfig` classes are no longer part of Headroom. Context management is now handled automatically inside the pipeline (live-zone-only compression).

View file

@ -26,7 +26,7 @@ using a newer Python, force a supported interpreter.
pip install headroom-ai
```
The core package includes the `compress()` function, SmartCrusher, CacheAligner, and IntelligentContext. No heavy dependencies.
The core package includes the `compress()` function, SmartCrusher, and CacheAligner. No heavy dependencies.
### Extras

View file

@ -52,7 +52,7 @@ Headroom includes an AST-aware CodeCompressor (tree-sitter, 8 languages) but it
**Why this is the right default**: Code is almost always fetched because the user wants to work with it. Compressing function bodies would remove exactly what they need.
**Where code savings come from**: The IntelligentContextManager drops old code messages that are no longer relevant (scoring-based), which is a better strategy than stripping function bodies.
**Where code savings come from**: Headroom compresses code in the live zone — the newest tool outputs and content blocks — with the AST-aware CodeCompressor, while keeping recent and analysis-context code fully intact. It never drops messages from the conversation history or strips function bodies.
**Override**: Set `protect_analysis_context=False` in `ContentRouterConfig` for aggressive code compression. Requires `headroom-ai[code]` for tree-sitter.

View file

@ -104,7 +104,7 @@ HeadroomStrandsModel.stream()
Provider API (Bedrock, etc.)
```
The model wrapper uses the full Headroom pipeline (CacheAligner, ContentRouter, IntelligentContext). The hook provider uses SmartCrusher directly for fast JSON compression.
The model wrapper uses the full Headroom pipeline (CacheAligner, ContentRouter). The hook provider uses SmartCrusher directly for fast JSON compression.
## Structured output

View file

@ -1,339 +0,0 @@
#!/usr/bin/env python3
"""Scale test for IntelligentContextManager TOIN + CCR integration.
This tests that:
1. Dropped messages are stored in CCR
2. Drops are recorded to TOIN
3. The marker includes CCR reference
4. TOIN patterns accumulate across multiple compressions
"""
import json
import os
# Set API key from environment or use provided key
if not os.environ.get("OPENAI_API_KEY"):
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "")
from headroom.cache.compression_store import get_compression_store
from headroom.config import IntelligentContextConfig
from headroom.telemetry import get_toin
from headroom.tokenizer import Tokenizer
from headroom.tokenizers import EstimatingTokenCounter
from headroom.transforms.intelligent_context import IntelligentContextManager
def create_large_conversation(num_turns: int = 50) -> list[dict]:
"""Create a large conversation with varied content."""
messages = [{"role": "system", "content": "You are a helpful coding assistant."}]
for i in range(num_turns):
# Vary content to create different importance levels
if i % 10 == 0:
# Error messages (should be preserved)
messages.append(
{"role": "user", "content": f"I'm getting an error: TypeError at line {i * 10}"}
)
messages.append(
{
"role": "assistant",
"content": f"The TypeError at line {i * 10} is caused by a type mismatch. "
f"Here's the fix:\n```python\n# Fix for error {i}\ndef fix_{i}():\n pass\n```",
}
)
elif i % 7 == 0:
# Tool calls (should stay atomic)
messages.append({"role": "user", "content": f"Search for files matching pattern_{i}"})
messages.append(
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": f"call_{i}",
"type": "function",
"function": {
"name": "search_files",
"arguments": f'{{"pattern": "pattern_{i}"}}',
},
}
],
}
)
messages.append(
{
"role": "tool",
"tool_call_id": f"call_{i}",
"content": json.dumps([f"file_{i}_a.py", f"file_{i}_b.py", f"file_{i}_c.py"]),
}
)
else:
# Regular conversation (lower priority)
messages.append(
{"role": "user", "content": f"Question {i}: Can you explain how feature_{i} works?"}
)
messages.append(
{
"role": "assistant",
"content": f"Feature_{i} is a component that handles processing. "
f"It works by iterating through the data and applying "
f"transformations. Here's a brief overview of the key aspects "
f"and how they interact with other parts of the system. "
f"The main entry point is the process() method which takes "
f"input data and returns the transformed output.",
}
)
return messages
def test_toin_ccr_integration():
"""Test TOIN + CCR integration with IntelligentContextManager."""
print("=" * 70)
print("TOIN + CCR Integration Test for IntelligentContextManager")
print("=" * 70)
# Get TOIN and CCR store
toin = get_toin()
store = get_compression_store()
# Record initial state
initial_patterns = len(toin._patterns) if hasattr(toin, "_patterns") else 0
# CCR store uses a backend, not direct _store
if hasattr(store, "_backend") and hasattr(store._backend, "_store"):
initial_store_size = len(store._backend._store)
else:
initial_store_size = 0
print("\nInitial state:")
print(f" TOIN patterns: {initial_patterns}")
print(f" CCR store entries: {initial_store_size}")
# Create manager with TOIN
config = IntelligentContextConfig(
enabled=True,
keep_system=True,
keep_last_turns=3,
output_buffer_tokens=2000,
use_importance_scoring=True,
)
manager = IntelligentContextManager(config=config, toin=toin)
tokenizer = Tokenizer(EstimatingTokenCounter())
# Run multiple compression cycles to accumulate TOIN patterns
print("\n" + "-" * 70)
print("Running compression cycles...")
print("-" * 70)
all_ccr_refs = []
for cycle in range(5):
# Create fresh conversation each cycle
messages = create_large_conversation(num_turns=30 + cycle * 5)
tokens_before = tokenizer.count_messages(messages)
# Set a tight limit to force dropping
model_limit = tokens_before // 2
result = manager.apply(
messages,
tokenizer,
model_limit=model_limit,
output_buffer=1000,
)
# Extract CCR reference from marker if present
ccr_ref = None
for marker in result.markers_inserted:
if "ccr_retrieve" in marker and "reference '" in marker:
start = marker.find("reference '") + len("reference '")
end = marker.find("'", start)
ccr_ref = marker[start:end]
all_ccr_refs.append(ccr_ref)
print(f"\nCycle {cycle + 1}:")
print(f" Messages: {len(messages)}{len(result.messages)}")
print(
f" Tokens: {result.tokens_before}{result.tokens_after} "
f"({100 * (1 - result.tokens_after / result.tokens_before):.1f}% reduction)"
)
print(f" Transforms: {result.transforms_applied}")
print(f" CCR reference: {ccr_ref or 'None'}")
# Check final state
final_patterns = len(toin._patterns) if hasattr(toin, "_patterns") else 0
if hasattr(store, "_backend") and hasattr(store._backend, "_store"):
final_store_size = len(store._backend._store)
else:
final_store_size = 0
print("\n" + "-" * 70)
print("Final state:")
print("-" * 70)
print(
f" TOIN patterns: {initial_patterns}{final_patterns} (+{final_patterns - initial_patterns})"
)
print(
f" CCR store entries: {initial_store_size}{final_store_size} (+{final_store_size - initial_store_size})"
)
print(f" CCR references created: {len(all_ccr_refs)}")
# Test retrieval from CCR
if all_ccr_refs:
print("\n" + "-" * 70)
print("Testing CCR retrieval...")
print("-" * 70)
ref = all_ccr_refs[-1] # Use the most recent reference
entry = store.retrieve(ref)
if entry:
# Parse the retrieved content from the CompressionEntry
try:
dropped_messages = json.loads(entry.original_content)
print(f" Retrieved {len(dropped_messages)} dropped messages from CCR")
print(f" First message role: {dropped_messages[0].get('role', 'unknown')}")
print(f" Content preview: {str(dropped_messages[0].get('content', ''))[:100]}...")
print(" Entry metadata:")
print(f" - Tool: {entry.tool_name}")
print(f" - Original tokens: {entry.original_tokens}")
print(f" - Compressed tokens: {entry.compressed_tokens}")
except json.JSONDecodeError:
print(f" Retrieved content (not JSON): {entry.original_content[:200]}...")
else:
print(f" WARNING: Could not retrieve CCR reference {ref}")
# Debug: check what's in the store
print(f" Store backend type: {type(store._backend)}")
if hasattr(store._backend, "_store"):
print(f" Backend store keys: {list(store._backend._store.keys())[:5]}...")
# Print TOIN statistics
print("\n" + "-" * 70)
print("TOIN Statistics:")
print("-" * 70)
stats = toin.get_stats()
print(f" Total patterns: {stats.get('total_patterns', 0)}")
print(f" Total compressions: {stats.get('total_compressions', 0)}")
print(f" Total retrievals: {stats.get('total_retrievals', 0)}")
print(f" Retrieval rate: {stats.get('retrieval_rate', 0):.1%}")
# Check for intelligent_context_drop patterns
drop_patterns = (
[
p
for p in toin._patterns.values()
if hasattr(p, "tool_name") and "intelligent_context" in str(getattr(p, "tool_name", ""))
]
if hasattr(toin, "_patterns")
else []
)
print(f" IntelligentContext drop patterns: {len(drop_patterns)}")
print("\n" + "=" * 70)
print("TEST COMPLETE")
print("=" * 70)
# Assertions
assert final_patterns >= initial_patterns, "TOIN should have recorded new patterns"
assert len(all_ccr_refs) > 0, "Should have created CCR references"
# CCR store entries should exist (though count may vary due to TTL)
if final_store_size == 0 and initial_store_size == 0:
print(" Note: CCR store size shows 0 (entries may have different backend)")
else:
assert final_store_size > initial_store_size, "CCR store should have new entries"
print("\n✓ All assertions passed!")
return True
def test_with_real_llm():
"""Test with a real LLM call to verify end-to-end flow."""
print("\n" + "=" * 70)
print("Real LLM Integration Test")
print("=" * 70)
api_key = os.environ.get("OPENAI_API_KEY")
if not api_key:
print("Skipping real LLM test - OPENAI_API_KEY not set")
return
try:
from openai import OpenAI
client = OpenAI()
except ImportError:
print("Skipping real LLM test - openai package not installed")
return
# Create a conversation that will be compressed
messages = create_large_conversation(num_turns=20)
# Apply IntelligentContext compression
toin = get_toin()
config = IntelligentContextConfig(
enabled=True,
keep_system=True,
keep_last_turns=2,
)
manager = IntelligentContextManager(config=config, toin=toin)
tokenizer = Tokenizer(EstimatingTokenCounter())
tokens_before = tokenizer.count_messages(messages)
result = manager.apply(
messages,
tokenizer,
model_limit=tokens_before // 3, # Force significant compression
output_buffer=500,
)
print("\nCompression result:")
print(f" Messages: {len(messages)}{len(result.messages)}")
print(f" Tokens: {result.tokens_before}{result.tokens_after}")
# Convert to OpenAI format (filter out tool messages with None content)
openai_messages = []
for msg in result.messages:
if msg.get("role") == "tool":
continue # Skip tool messages for this test
if msg.get("content") is None:
continue # Skip messages with None content
openai_messages.append({"role": msg["role"], "content": msg["content"]})
# Add a question about the compressed context
openai_messages.append(
{
"role": "user",
"content": "Based on our conversation, what errors did we discuss? "
"If you see a message about compressed context, note the CCR reference.",
}
)
print(f"\nSending {len(openai_messages)} messages to OpenAI...")
try:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=openai_messages,
max_tokens=500,
)
print("\nLLM Response:")
print("-" * 40)
print(response.choices[0].message.content)
print("-" * 40)
print(f"\nTokens used: {response.usage.total_tokens}")
except Exception as e:
print(f"LLM call failed: {e}")
if __name__ == "__main__":
# Run the TOIN + CCR integration test
test_toin_ccr_integration()
# Run real LLM test if API key available
test_with_real_llm()

View file

@ -254,83 +254,21 @@ no longer resolves; use `[ml]` instead.
---
#### Transform 5: Rolling Window
#### Context Management: Live-Zone-Only Compression
**Problem:** Even after compression, you might exceed the model's context limit.
**Approach:** Headroom never drops messages from conversation history, and it does not do position-based or score-based context management. Context management is handled automatically inside the pipeline by compressing only the **live zone** — the newest content blocks (the latest user message and the latest tool result / tool output).
```python
# Model limit: 128K tokens
# Your messages: 150K tokens
# Need to drop 22K tokens
# Rolling Window drops OLDEST messages first:
# - Keeps system prompt (always)
# - Keeps last 2 turns (always)
# - Drops old tool calls + their responses as atomic units
```
# The cache hot zone is NEVER mutated:
# - System prompt
# - Tool definitions
# - Older conversation turns
#
# Only the newest content blocks are compressed (type-aware),
# and every compression is reversible via CCR.
```
**Safety rule:** If we drop a tool CALL, we MUST drop its RESPONSE too (or vice versa). Otherwise the model sees orphaned data.
---
#### Transform 6: Intelligent Context Manager (Advanced)
**Problem:** Rolling Window drops by position (oldest first), but position doesn't equal importance.
```python
# Scenario: Error at turn 3, verbose success at turn 10
# Rolling Window: Drops turn 3 error (oldest first)
# Intelligent Context: Keeps turn 3 error (high TOIN error score)
```
**The Solution:** Multi-factor importance scoring using TOIN-learned patterns:
```python
# Message scores (all learned, no hardcodes):
scores = {
"recency": 0.20, # Exponential decay from end
"semantic_similarity": 0.20, # Embedding similarity to recent context
"toin_importance": 0.25, # TOIN retrieval_rate (high = important)
"error_indicator": 0.15, # TOIN field_semantics.inferred_type
"forward_reference": 0.15, # Referenced by later messages
"token_density": 0.05, # Unique tokens / total tokens
}
# Drops lowest-scored messages first
# Preserves critical errors even if old
```
**Key principle:** No hardcoded patterns. Error detection uses TOIN's learned `field_semantics.inferred_type == "error_indicator"`, not keyword matching like "error" or "fail".
**TOIN + CCR Integration:**
IntelligentContext is a **message-level compressor** — just like SmartCrusher compresses items in an array, IntelligentContext "compresses" messages in a conversation. This means full CCR integration:
```python
# When messages are dropped:
# 1. Store dropped messages in CCR for potential retrieval
ccr_ref = store.store(
original=json.dumps(dropped_messages),
compressed="[60 messages dropped]",
tool_name="intelligent_context_drop",
)
# 2. Record drop to TOIN for cross-user learning
toin.record_compression(
tool_signature=message_signature, # Pattern of roles, tools, errors
original_count=len(dropped_messages),
compressed_count=1, # The marker
strategy="intelligent_context_drop",
)
# 3. Insert marker with CCR reference
marker = f"[Earlier context compressed: 60 messages dropped. Retrieve: {ccr_ref}]"
```
**The feedback loop:**
- If users retrieve dropped messages via CCR, TOIN learns those patterns are important
- Future drops of similar message patterns get higher importance scores
- The system gets smarter across all users, not just within one session
**Why this matters:** Leaving the hot zone untouched preserves provider prompt caching (e.g. Claude's cached-prefix read discount). Because only the newest blocks change, earlier turns stay byte-stable across requests and keep hitting the KV cache. Compression is type-aware (handled by ContentRouter and its compressors) and fully reversible — the LLM can call `headroom_retrieve` to restore any compressed block.
---
@ -1047,12 +985,11 @@ headroom/
│ ├── base.py # Transform protocol
│ ├── pipeline.py # Orchestrates all transforms
│ ├── cache_aligner.py # Date extraction for caching
│ ├── tool_crusher.py # Naive compression (disabled)
│ ├── smart_crusher.py # Statistical compression (default)
│ ├── rolling_window.py # Token limit enforcement (position-based)
├── intelligent_context.py # Semantic context management (score-based)
├── scoring.py # Message importance scoring
└── (legacy llmlingua_compressor.py removed — see [ml] extra for Kompress)
│ ├── code_compressor.py # AST-aware code compression (tree-sitter)
└── (legacy llmlingua_compressor.py removed — see [ml] extra for Kompress;
rolling_window.py / intelligent_context.py / scoring.py / tool_crusher.py
removed — context management is now live-zone-only)
├── cache/ # CCR Architecture - Caching & Storage
│ ├── compression_store.py # Phase 1: Store original content
@ -1117,8 +1054,7 @@ This means:
- Audit mode for testing before optimizing
### 4. Smart by Default
- SmartCrusher enabled (statistical analysis)
- ToolCrusher disabled (naive rules)
- SmartCrusher enabled (statistical, content-aware analysis — not naive rules)
- Conservative settings that preserve important data
---

View file

@ -30,7 +30,7 @@ Headroom includes an AST-aware CodeCompressor (tree-sitter, 8 languages) but it'
**Why this is the right default**: Code is almost always fetched because the user wants to work with it. Compressing function bodies would remove exactly what they need. LLMs like Claude are excellent at navigating large code files without compression.
**Where code savings come from**: The IntelligentContextManager drops old code messages that are no longer relevant (scoring-based), which is a better strategy than stripping function bodies from active code.
**Where code savings come from**: Headroom does not strip function bodies from active code or drop old code messages. Code savings come from compressing the newest content blocks (live-zone-only compression) when they are not protected, leaving the conversation history intact.
**Override**: Set `protect_analysis_context=False` in `ContentRouterConfig` for aggressive code compression. Requires `headroom-ai[code]` for tree-sitter.
@ -119,7 +119,7 @@ The Tool Output Intelligence Network (TOIN) learns compression patterns from usa
## Performance Characteristics
- **ContentRouter** accounts for 91-98% of pipeline cost — it does the actual compression work
- **CacheAligner** and **RollingWindow** are sub-millisecond
- **CacheAligner** is sub-millisecond
- Scaling is roughly **linear** with input size
- Full benchmark data: [LATENCY_BENCHMARKS.md](LATENCY_BENCHMARKS.md)

View file

@ -347,7 +347,7 @@ The integration operates at the model layer, not the agent layer. Some Agno feat
### Best Practices for Maximum Savings
1. **Tool-heavy agents see the biggest wins** — Tool results (JSON, logs, search results) compress 70-90%
2. **Long conversations benefit from RollingWindow** — Configure context limits to avoid hitting provider maximums
2. **Long conversations are handled automatically** — Headroom compresses the newest tool outputs and content blocks in place (live-zone-only compression) and never drops messages from history, so the cache hot zone stays intact. No context-limit configuration is required.
3. **Wrap at the model level, not agent level** — This ensures all LLM calls go through optimization
4. **Use hooks for observability** — Track token usage patterns to identify optimization opportunities

View file

@ -24,7 +24,6 @@ client = HeadroomClient(
| `store_url` | `str` | `None` | Storage URL for metrics |
| `smart_crusher_config` | `SmartCrusherConfig` | Default | Compression settings |
| `cache_aligner_config` | `CacheAlignerConfig` | Default | Cache alignment settings |
| `rolling_window_config` | `RollingWindowConfig` | Default | Context window settings |
### Methods
@ -96,64 +95,6 @@ config = CacheAlignerConfig(
)
```
### RollingWindowConfig
```python
from headroom import RollingWindowConfig
config = RollingWindowConfig(
max_tokens=100000,
preserve_system=True,
preserve_recent_turns=5,
drop_oldest_first=True,
)
```
### IntelligentContextConfig
```python
from headroom.config import IntelligentContextConfig, ScoringWeights
weights = ScoringWeights(
recency=0.20,
semantic_similarity=0.20,
toin_importance=0.25,
error_indicator=0.15,
forward_reference=0.15,
token_density=0.05,
)
config = IntelligentContextConfig(
enabled=True,
keep_system=True,
keep_last_turns=2,
output_buffer_tokens=4000,
use_importance_scoring=True,
scoring_weights=weights,
toin_integration=True,
recency_decay_rate=0.1,
compress_threshold=0.1,
)
```
### ScoringWeights
```python
from headroom.config import ScoringWeights
weights = ScoringWeights(
recency=0.20, # Exponential decay from end
semantic_similarity=0.20, # Embedding similarity to recent context
toin_importance=0.25, # TOIN retrieval_rate
error_indicator=0.15, # TOIN field_semantics error detection
forward_reference=0.15, # Messages referenced by later messages
token_density=0.05, # Unique/total token ratio
)
# Weights are auto-normalized to sum to 1.0
normalized = weights.normalized()
```
### RelevanceScorerConfig
```python
@ -335,64 +276,10 @@ aligner = CacheAligner()
result = aligner.align(messages)
```
### RollingWindow
```python
from headroom import RollingWindow
window = RollingWindow(config)
result = window.apply(messages, max_tokens=100000)
```
### IntelligentContextManager
```python
from headroom.transforms import IntelligentContextManager
from headroom.config import IntelligentContextConfig
from headroom.telemetry import get_toin
# With TOIN integration for learned patterns
toin = get_toin()
config = IntelligentContextConfig(
keep_system=True,
keep_last_turns=2,
use_importance_scoring=True,
)
manager = IntelligentContextManager(config, toin=toin)
result = manager.apply(messages, tokenizer, model_limit=128000)
# Access scoring details
print(result.transforms_applied) # ["intelligent_cap:3"]
print(result.tokens_before, result.tokens_after)
```
### MessageScorer
```python
from headroom.transforms import MessageScorer, MessageScore
from headroom.config import ScoringWeights
scorer = MessageScorer(
weights=ScoringWeights(),
toin=None, # Optional TOIN for learned patterns
embedding_provider=None, # Optional for semantic similarity
recency_decay_rate=0.1,
)
# Score messages
scores: list[MessageScore] = scorer.score_messages(
messages=messages,
protected_indices={0}, # System message
tool_unit_indices={2, 3}, # Tool call + response
)
for score in scores:
print(f"Message {score.message_index}: {score.total_score:.2f}")
print(f" Recency: {score.recency_score:.2f}")
print(f" TOIN: {score.toin_score:.2f}")
print(f" Protected: {score.is_protected}")
```
> **Context management** is handled automatically inside the pipeline
> (live-zone-only compression). The position-based `RollingWindow` and
> score-based `IntelligentContextManager` / `MessageScorer` APIs have been
> removed and are no longer part of Headroom.
### TransformPipeline
@ -402,7 +289,6 @@ from headroom import TransformPipeline
pipeline = TransformPipeline([
SmartCrusher(),
CacheAligner(),
RollingWindow(),
])
result = pipeline.transform(messages)

View file

@ -17,7 +17,6 @@ CCR eliminates this tradeoff.
|-----------|-------------------|-----------------|
| **SmartCrusher** | JSON arrays (tool outputs) | Stores original array, marker includes hash |
| **ContentRouter** | Code, logs, search results, text | Stores original content by strategy |
| **IntelligentContextManager** | Messages (conversation turns) | Stores dropped messages, marker includes hash |
## How CCR Works
@ -90,36 +89,31 @@ Turn 5: User asks "What about the auth middleware?"
→ LLM sees full file list, finds auth_middleware.py
```
## Message-Level CCR (IntelligentContext)
## CCR Stores Content Blocks, Not Dropped Messages
IntelligentContextManager is a **message-level compressor**. When it drops low-importance messages to fit the context budget, those messages are stored in CCR:
Headroom never drops whole messages from conversation history. CCR is purely about compressed **content blocks** — the newest tool outputs, tool results, and user content that the live-zone pipeline compresses. The original block is stored in the cache and is retrievable on demand:
```
┌─────────────────────────────────────────────────────────────────┐
│ LONG CONVERSATION (100 messages, 50K tokens) │
│ └─ IntelligentContext scores messages by importance │
│ └─ Drops 60 low-scoring messages │
│ └─ Dropped messages cached with hash=def456 │
│ └─ Marker inserted: "60 messages dropped, retrieve: def456" │
│ LATEST TOOL RESULT (500 files, 12K tokens) │
│ └─ ContentRouter / SmartCrusher compresses the block │
│ └─ Original cached with hash=def456 │
│ └─ Marker inserted: "500 items compressed, retrieve: def456" │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ LLM PROCESSING │
│ Option A: LLM solves task with remaining messages → Done
│ Option B: LLM needs earlier context
│ Option A: LLM solves task with the compressed block → Done
│ Option B: LLM needs the full content
│ → Calls headroom_retrieve(hash=def456) │
│ → Full conversation restored
│ → Full original block restored
└─────────────────────────────────────────────────────────────────┘
```
**The marker includes the CCR reference:**
```
[Earlier context compressed: 60 message(s) dropped by importance scoring.
Full content available via ccr_retrieve tool with reference 'def456'.]
```
The older conversation turns, system prompt, and tool definitions — the provider cache hot zone — are never mutated, so prompt caching keeps working. Compression happens only on the live zone (the newest content blocks) and is fully reversible via CCR.
**TOIN integration:** When users retrieve dropped messages, TOIN learns to score those message patterns higher next time, improving future drop decisions across all users.
**TOIN integration:** When users retrieve compressed content, TOIN learns to treat those patterns as higher value next time, improving future compression decisions across all users.
## Features

View file

@ -200,107 +200,21 @@ config = CacheAlignerConfig(
)
```
## Rolling Window Configuration
## Context Management
Control context window management:
Context management is handled automatically inside the pipeline
(live-zone-only compression) — there is nothing to configure. Headroom
**never** drops messages from the conversation history and does not do
position-based or score-based context management. It compresses only the
newest content blocks (the latest user message and the latest tool result /
tool output), type-aware and reversible via CCR. The cache hot zone — system
prompt, tools, and older turns — is never mutated, which preserves provider
prompt caching.
```python
from headroom.transforms import RollingWindowConfig
config = RollingWindowConfig(
# Minimum turns to always keep
min_keep_turns=3,
# Reserve tokens for output
output_buffer_tokens=4000,
# Drop oldest tool outputs first
prefer_drop_tool_outputs=True,
)
```
## Intelligent Context Manager Configuration
For semantic-aware context management with importance scoring:
```python
from headroom.config import IntelligentContextConfig, ScoringWeights
# Customize scoring weights (must sum to 1.0, or will be normalized)
weights = ScoringWeights(
recency=0.20, # Newer messages score higher
semantic_similarity=0.20, # Similarity to recent context
toin_importance=0.25, # TOIN-learned retrieval patterns
error_indicator=0.15, # TOIN-learned error field types
forward_reference=0.15, # Messages referenced by later messages
token_density=0.05, # Information density
)
config = IntelligentContextConfig(
# Enable/disable the manager
enabled=True,
# Protection settings
keep_system=True, # Never drop system messages
keep_last_turns=2, # Protect last N user turns
# Token budget
output_buffer_tokens=4000, # Reserve for model output
# Scoring settings
use_importance_scoring=True, # Use semantic scoring (vs position-only)
scoring_weights=weights, # Custom weights
toin_integration=True, # Use TOIN patterns if available
recency_decay_rate=0.1, # Exponential decay lambda
# Strategy thresholds
compress_threshold=0.1, # Try compression first if <10% over budget
)
```
### CCR Integration
When IntelligentContext drops messages, they're stored in CCR for potential retrieval:
```python
from headroom.telemetry import get_toin
# Pass TOIN for bidirectional integration
toin = get_toin()
manager = IntelligentContextManager(config=config, toin=toin)
# Dropped messages are:
# 1. Stored in CCR (so LLM can retrieve if needed)
# 2. Recorded to TOIN (so it learns which patterns matter)
# 3. Marked with CCR reference in the inserted message
```
The marker inserted when messages are dropped includes the CCR reference:
```
[Earlier context compressed: 14 message(s) dropped by importance scoring.
Full content available via ccr_retrieve tool with reference 'abc123def456'.]
```
### Scoring Weights
The `ScoringWeights` class controls how messages are scored:
| Weight | Default | Description |
|--------|---------|-------------|
| `recency` | 0.20 | Exponential decay from conversation end |
| `semantic_similarity` | 0.20 | Embedding cosine similarity to recent context |
| `toin_importance` | 0.25 | TOIN retrieval_rate (high retrieval = important) |
| `error_indicator` | 0.15 | TOIN field_semantics error detection |
| `forward_reference` | 0.15 | Count of later messages referencing this one |
| `token_density` | 0.05 | Unique tokens / total tokens |
Weights are automatically normalized to sum to 1.0:
```python
weights = ScoringWeights(recency=1.0, toin_importance=1.0)
normalized = weights.normalized()
# recency=0.5, toin_importance=0.5, others=0.0
```
> The earlier `RollingWindowConfig`, `IntelligentContextConfig`, and
> `ScoringWeights` configuration classes (and the position-/score-based
> context managers they configured) have been removed and are no longer part
> of Headroom.
## Environment Variables

View file

@ -236,24 +236,23 @@ headroom wrap openclaw
## How It Works
Headroom runs a three-stage pipeline on every request:
Headroom runs a two-stage pipeline on every request:
```mermaid
graph LR
A[Your Prompt] --> B[CacheAligner]
B --> C[ContentRouter]
C --> D[IntelligentContext]
D --> E[LLM Provider]
C --> E[LLM Provider]
C -->|JSON| F[SmartCrusher]
C -->|Code| G[CodeCompressor]
C -->|Text| H[Kompress]
C -->|Logs| I[LogCompressor]
F --> D
G --> D
H --> D
I --> D
F --> E
G --> E
H --> E
I --> E
```
**Stage 1: CacheAligner** — Stabilizes message prefixes so the provider's KV cache actually hits. Claude offers a 90% read discount on cached prefixes; CacheAligner makes that work.
@ -270,7 +269,7 @@ graph LR
| Git diffs | **DiffCompressor** | Preserves change hunks, drops unchanged context. |
| HTML | **HTMLExtractor** | Strips markup, extracts readable content. |
**Stage 3: IntelligentContext** — If the conversation still exceeds the model's context limit, scores each message by importance (recency, references, density) and drops the lowest-value ones.
Context management is handled automatically inside the pipeline (live-zone-only compression): Headroom compresses only the newest content blocks (the latest user message and tool results) and never drops messages from history. The system prompt, tool definitions, and older turns — the provider cache hot zone — are left untouched so prompt caching keeps working.
**Nothing is lost.** Compressed content goes into the CCR store (Compress-Cache-Retrieve). The LLM gets a `headroom_retrieve` tool and can fetch full originals when it needs more detail.

View file

@ -160,7 +160,7 @@ HeadroomStrandsModel.stream()
Provider API (Bedrock, etc.)
```
The model wrapper uses Headroom's full pipeline (CacheAligner → ContentRouter → IntelligentContext). The hook provider uses SmartCrusher directly for fast JSON compression of individual tool results.
The model wrapper uses Headroom's full pipeline (CacheAligner → ContentRouter). The hook provider uses SmartCrusher directly for fast JSON compression of individual tool results.
---

View file

@ -110,218 +110,19 @@ config = CacheAlignerConfig(
---
## RollingWindow
## Context management
Context management within token limits.
Context management is handled automatically inside the pipeline
(live-zone-only compression). Headroom **never** drops messages from the
conversation history and does not do position-based or score-based context
management. It compresses only the newest content blocks (the latest user
message and the latest tool result / tool output), type-aware and reversible
via CCR. The cache hot zone — system prompt, tools, and older turns — is never
mutated, which preserves provider prompt caching.
### The Problem
Long conversations exceed context limits. Naive truncation breaks tool calls:
```
[tool_call: search] # Kept
[tool_result: ...] # Dropped = orphaned call!
```
### The Solution
RollingWindow drops complete tool units, preserving pairs:
```python
from headroom import RollingWindow
window = RollingWindow(config)
result = window.apply(messages, max_tokens=100000)
# Guarantees:
# 1. Tool calls paired with results
# 2. System prompt preserved
# 3. Recent turns kept
# 4. Oldest tool outputs dropped first
```
### Configuration
```python
from headroom import RollingWindowConfig
config = RollingWindowConfig(
max_tokens=100000, # Target token limit
preserve_system=True, # Always keep system prompt
preserve_recent_turns=5, # Keep last 5 user/assistant turns
drop_oldest_first=True, # Remove oldest tool outputs
)
```
### Drop Priority
1. **Oldest tool outputs** - First to go
2. **Old assistant messages** - Summary preserved
3. **Old user messages** - Only if necessary
4. **Never dropped**: System prompt, recent turns, active tool pairs
> **Note:** For more intelligent context management based on semantic importance rather than just position, see [IntelligentContextManager](#intelligentcontextmanager) below.
---
## IntelligentContextManager
Semantic-aware context management with TOIN-learned importance scoring.
### The Problem
RollingWindow drops messages by position (oldest first), but position doesn't equal importance:
- An error message from turn 3 might be critical
- A verbose success response from turn 10 might be expendable
- Messages referenced by later turns should be preserved
### The Solution
IntelligentContextManager uses multi-factor importance scoring:
```python
from headroom.transforms import IntelligentContextManager, IntelligentContextConfig
manager = IntelligentContextManager(config)
result = manager.apply(messages, tokenizer, model_limit=128000)
# Guarantees:
# 1. System messages never dropped (configurable)
# 2. Last N turns always protected
# 3. Tool calls/responses dropped atomically
# 4. Drops by importance score, not just position
```
### How Scoring Works
Messages are scored on multiple factors (all learned, no hardcodes):
| Factor | Weight | Description |
|--------|--------|-------------|
| Recency | 20% | Exponential decay from conversation end |
| Semantic Similarity | 20% | Embedding similarity to recent context |
| TOIN Importance | 25% | Learned from retrieval patterns |
| Error Indicators | 15% | TOIN-learned error field detection |
| Forward References | 15% | Messages referenced by later messages |
| Token Density | 5% | Information density (unique/total tokens) |
**Key principle:** No hardcoded patterns. Error detection uses TOIN's `field_semantics.inferred_type == "error_indicator"`, not keyword matching.
### Configuration
```python
from headroom.transforms import IntelligentContextManager
from headroom.config import IntelligentContextConfig, ScoringWeights
# Custom scoring weights
weights = ScoringWeights(
recency=0.20,
semantic_similarity=0.20,
toin_importance=0.25,
error_indicator=0.15,
forward_reference=0.15,
token_density=0.05,
)
config = IntelligentContextConfig(
enabled=True,
keep_system=True, # Never drop system messages
keep_last_turns=2, # Protect last N user turns
output_buffer_tokens=4000, # Reserve for model output
use_importance_scoring=True, # Enable semantic scoring
scoring_weights=weights, # Custom weights
toin_integration=True, # Use TOIN patterns
recency_decay_rate=0.1, # Exponential decay lambda
compress_threshold=0.1, # Try compression first if <10% over
)
manager = IntelligentContextManager(config)
```
### Strategy Selection
Based on how much over budget you are:
| Overage | Strategy | Action |
|---------|----------|--------|
| Under budget | NONE | No action needed |
| < 10% over | COMPRESS_FIRST | Try deeper compression |
| >= 10% over | DROP_BY_SCORE | Drop lowest-scored messages |
### TOIN + CCR Integration
IntelligentContextManager is a **message-level compressor**. Just like SmartCrusher compresses items in a JSON array, IntelligentContext "compresses" messages in a conversation by dropping low-value ones.
**Bidirectional TOIN integration:**
1. **Scoring uses TOIN patterns**: Learned retrieval rates and field semantics inform importance scores
2. **Drops are recorded to TOIN**: When messages are dropped, TOIN learns the pattern
3. **CCR stores originals**: Dropped messages are stored in CCR for potential retrieval
4. **Retrievals feed back to TOIN**: If users retrieve dropped messages, TOIN learns to score those patterns higher
```python
from headroom.telemetry import get_toin
toin = get_toin()
manager = IntelligentContextManager(config, toin=toin)
# TOIN provides (for scoring):
# - retrieval_rate: How often this message pattern is retrieved (high = important)
# - field_semantics: Learned field types (error_indicator, identifier, etc.)
# - commonly_retrieved_fields: Fields that users frequently need
# TOIN receives (from drops):
# - Message pattern signatures (role counts, has_tools, has_errors)
# - Token counts (original vs marker size)
# - Retrieval feedback when users access CCR
```
**What this means:**
- When you drop a message pattern and users frequently retrieve it, TOIN learns to score it higher next time
- When you drop a pattern and no one retrieves it, that confirms it was safe to drop
- The feedback loop improves drop decisions across all users, not just in one session
### Example: Before vs After
**RollingWindow (position-based):**
```
Messages: [sys, user1, asst1, user2, asst2_error, user3, asst3, user4, asst4]
Over budget by 3 messages.
Drops: user1, asst1, user2 (oldest first)
Result: Loses context, keeps verbose asst3
```
**IntelligentContextManager (score-based):**
```
Messages scored:
- asst2_error: 0.85 (TOIN learned error indicator)
- asst1: 0.45 (old, low density)
- asst3: 0.40 (verbose, low unique tokens)
Drops: asst1, asst3, user1 (lowest scores)
Result: Preserves critical error message
```
### Backwards Compatibility
Convert from RollingWindowConfig:
```python
from headroom.config import IntelligentContextConfig, RollingWindowConfig
rolling_config = RollingWindowConfig(
max_tokens=100000,
preserve_system=True,
preserve_recent_turns=3,
)
# Convert to intelligent context config
intelligent_config = IntelligentContextConfig(
keep_system=rolling_config.preserve_system,
keep_last_turns=rolling_config.preserve_recent_turns,
)
```
> The earlier position-based `RollingWindow` and score-based
> `IntelligentContextManager` transforms have been removed and are no longer
> part of Headroom.
---
@ -528,12 +329,11 @@ This enables the feedback loop where compression decisions improve based on actu
Combine transforms for optimal results.
```python
from headroom import TransformPipeline, SmartCrusher, CacheAligner, RollingWindow
from headroom import TransformPipeline, SmartCrusher, CacheAligner
pipeline = TransformPipeline([
SmartCrusher(), # First: compress tool outputs
CacheAligner(), # Then: stabilize prefix
RollingWindow(), # Finally: fit in context
])
result = pipeline.transform(messages)
@ -551,13 +351,11 @@ The earlier hand-assembled `TransformPipeline([..., LLMLinguaCompressor(), ...])
| 1 | CacheAligner | Stabilize prefix for caching |
| 2 | SmartCrusher | Compress JSON tool outputs |
| 3 | Kompress (ML) | ML compression on remaining text (optional, `[ml]` extra) |
| 4 | RollingWindow | Enforce token limits (always last) |
**Why this order?**
- CacheAligner first to maximize prefix stability
- SmartCrusher handles JSON arrays efficiently
- Kompress compresses remaining long text
- RollingWindow truncates only if still over limit
---