2026-01-07 11:36:44 -08:00
|
|
|
# Headroom Examples
|
|
|
|
|
|
|
|
|
|
This directory contains examples demonstrating Headroom's capabilities.
|
|
|
|
|
|
|
|
|
|
## Quick Start Examples
|
|
|
|
|
|
|
|
|
|
### basic_usage.py
|
|
|
|
|
|
|
|
|
|
Basic integration with OpenAI client:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
export OPENAI_API_KEY='your-key'
|
|
|
|
|
python examples/basic_usage.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### anthropic_example.py
|
|
|
|
|
|
|
|
|
|
Integration with Anthropic Claude:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
export ANTHROPIC_API_KEY='your-key'
|
|
|
|
|
python examples/anthropic_example.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### streaming_example.py
|
|
|
|
|
|
|
|
|
|
Streaming responses with optimization:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
export OPENAI_API_KEY='your-key'
|
|
|
|
|
python examples/streaming_example.py
|
|
|
|
|
```
|
|
|
|
|
|
feat(transforms): tabular + spreadsheet (.xlsx/.xls) compression (#1128)
## Description
Adds a content-type-aware path for **tabular data** — CSV/TSV, markdown
tables, fixed-width text, and binary `.xlsx`/`.xls` spreadsheets — by
routing them through the existing, battle-tested `SmartCrusher` instead
of letting them fall through to `PLAIN_TEXT → Kompress`.
The pipeline already compressed tables losslessly when handed a JSON
array of records. This wires up the missing front door: detect tabular
text (and ingest binary spreadsheets), convert to JSON records, and
reuse `SmartCrusher.crush()`. No new compression algorithm.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **Detection** (`content_detector.py`): new `ContentType.TABULAR` +
`_try_detect_tabular()` for CSV/TSV, markdown tables, and fixed-width
columns. Ordered after search/log (which also look "delimited") and
before code, with a prose-rejection guard so it never steals
`file:line:content` search output, `key: value` logs, or sentences with
incidental commas. Rust backend returns `plain_text` for unknown types
and the router already falls back to the Python detector, so **no Rust
change**.
- **Bridge** (`tabular_ingest.py`): stdlib parsers + `to_records()` + a
`TabularCompressor` that parses → JSON records → `SmartCrusher`
(lossless `csv-schema` first; lossy row-drop with reversible
`<<ccr:HASH>>` markers stays SmartCrusher's built-in fallback). Only
adopts a result when it actually saves bytes.
- **Spreadsheets** (`spreadsheet_ingest.py`): `.xlsx`/`.xls` → per-sheet
CSV text at the SDK boundary. Optional deps (`pip install
headroom-ai[spreadsheet]`) fail loudly with an install hint, never
silently degrade.
- **Routing** (`content_router.py`): `CompressionStrategy.TABULAR`,
`enable_tabular_compressor` flag, lazy getter, apply branch, strategy
maps, Kompress fallback eligibility.
- **SDK** (`compress.py`): `compress_spreadsheet(path, ...)` helper (one
message per sheet).
- **Packaging** (`pyproject.toml`): new `[spreadsheet]` extra;
`openpyxl` added to `[dev]` so the xlsx path is exercised in CI.
- **Docs/demo**: `examples/tabular_compression_demo.py` + README entry.
### Design note: lossless-only
Compact, all-unique tables with no query yield ~0 savings — this is
correct, not a bug. SmartCrusher returns
`skip:unique_entities_no_signal` and won't drop unique rows without a
duplicate/relevance signal. Real wins come from verbose/redundant tables
and query-driven selection. A pressure-driven lossy row sampler was
considered and intentionally not added.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_transforms_tabular.py -q
collected 20 items
tests/test_transforms_tabular.py .................... [100%]
============================== 20 passed in 7.15s ==============================
$ ruff check headroom/transforms/tabular_ingest.py headroom/transforms/spreadsheet_ingest.py
All checks passed!
$ mypy headroom/transforms/tabular_ingest.py headroom/transforms/spreadsheet_ingest.py
Success: no issues found in 2 source files
```
`tests/test_transforms_tabular.py` (20 tests): detection true positives
+ no-misroute negatives (search/log/JSON/prose), parser units (incl.
fixed-width), the CSV→SmartCrusher bridge, router routing + disable
flag, and `.xlsx` ingestion (skipif openpyxl missing) + error paths.
`spreadsheet_ingest` 100% / `tabular_ingest` 90% line coverage.
## Real Behavior Proof
- **Environment:** local checkout of `feat/tabular-compression`, Python
3.x, `pip install -e ".[dev]"`.
- **Exact command / steps:** `python
examples/tabular_compression_demo.py` (no API key required).
- **Observed result:**
```text
=== Raw tabular text (ContentRouter, char-level) ===
compact unique CSV strat=tabular chars 1306 -> 1072 ( 17.9% saved)
redundant CSV strat=tabular chars 2661 -> 1350 ( 49.3% saved)
verbose markdown strat=tabular chars 2019 -> 1580 ( 21.7% saved)
=== Full pipeline (real tokenizer) ===
redundant CSV tokens 768 -> 394 ( 48.7% saved)
=== Binary spreadsheet (.xlsx) ===
2-sheet workbook tokens 1092 -> 683 ( 37.5% saved)
```
- **Not tested:** legacy `.xls` binary path (needs optional `xlrd` +
binary fixture; `# pragma: no cover`); base64-embedded `.xlsx` inside
multimodal blocks (out of scope, noted as a follow-up).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- CHANGELOG/version are intentionally untouched: this repo uses
**release-please**, which bumps the version and CHANGELOG via automated
`chore: release main` PRs, not per-feature PRs.
- The `.xls` path is `# pragma: no cover` (legacy, needs optional `xlrd`
+ a binary fixture).
- Follow-up (out of scope): base64-embedded `.xlsx` inside
tool-result/multimodal blocks; porting tabular parsers into the Rust
core for parity.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 09:30:20 -07:00
|
|
|
### tabular_compression_demo.py
|
|
|
|
|
|
|
|
|
|
Tabular + spreadsheet compression on generated sample data (no API key needed).
|
|
|
|
|
Shows where CSV/markdown tables and `.xlsx` workbooks compress and where compact,
|
|
|
|
|
all-unique data correctly passes through:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
python examples/tabular_compression_demo.py # run all scenarios
|
|
|
|
|
python examples/tabular_compression_demo.py --write DIR # also save the sample files
|
|
|
|
|
```
|
|
|
|
|
|
2026-01-07 11:36:44 -08:00
|
|
|
## Evaluation Examples
|
|
|
|
|
|
|
|
|
|
### smart_vs_naive_eval.py
|
|
|
|
|
|
|
|
|
|
Compare SmartCrusher against naive truncation:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
export OPENAI_API_KEY='your-key'
|
|
|
|
|
python examples/smart_vs_naive_eval.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### real_world_eval.py
|
|
|
|
|
|
|
|
|
|
Comprehensive evaluation with Anthropic models:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
export ANTHROPIC_API_KEY='your-key'
|
|
|
|
|
python examples/real_world_eval.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### real_world_openai_eval.py
|
|
|
|
|
|
|
|
|
|
Comprehensive evaluation with OpenAI models:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
export OPENAI_API_KEY='your-key'
|
|
|
|
|
python examples/real_world_openai_eval.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Demo Directories
|
|
|
|
|
|
|
|
|
|
### langchain_demo/
|
|
|
|
|
|
|
|
|
|
Full LangChain agent integration demo:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
# No API key needed for compression demo
|
|
|
|
|
PYTHONPATH=. python -m examples.langchain_demo.show_compression
|
|
|
|
|
|
|
|
|
|
# Full comparison (requires API key)
|
|
|
|
|
export OPENAI_API_KEY='your-key'
|
|
|
|
|
PYTHONPATH=. python -m examples.langchain_demo.run_comparison
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
See [langchain_demo/README.md](langchain_demo/README.md) for details.
|
|
|
|
|
|
|
|
|
|
### mcp_demo/
|
|
|
|
|
|
|
|
|
|
MCP (Model Context Protocol) integration demo:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
export OPENAI_API_KEY='your-key'
|
|
|
|
|
PYTHONPATH=. python -m examples.mcp_demo.run_agent_eval
|
|
|
|
|
```
|
|
|
|
|
|
feat: Add AWS Strands Agents SDK integration
## Description
Add Headroom integration with AWS Strands Agents SDK, enabling automatic
context optimization and tool output compression for Strands-based agents.
Fixes #14
## Type of Change
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update
## Changes Made
### Core Integration (`headroom/integrations/strands/`)
- **HeadroomHookProvider** - Implements Strands `HookProvider` interface for
automatic tool output compression via `AfterToolCallEvent`. Compresses
verbose tool outputs before they enter conversation context.
- **HeadroomStrandsModel** - Model wrapper that extends Strands `Model` base
class for message-level optimization. Implements all required abstract
methods: `stream()`, `get_config()`, `update_config()`, `structured_output()`.
- **Provider auto-detection** - Automatically detects appropriate Headroom
provider (Anthropic, OpenAI, Google) based on wrapped Strands model type.
- **`strands-agents` as optional dependency** - Install with
`pip install headroom-ai[strands]`
### Testing (`tests/integrations/test_strands/`)
- **Real integration tests (25 tests)** - Use actual AWS Bedrock API calls
with Claude 3 Haiku. Skip automatically when credentials unavailable.
- **Unit tests (57 tests)** - Mock-based tests for internal logic, edge cases,
and error handling. No credentials required.
### Demo (`examples/strands_bedrock_demo.py`)
- Interactive demo showcasing both integration patterns
- Visual before/after compression comparison with token savings
- 4 verbose tools (search, logs, database, metrics) demonstrating real savings
- Supports `--hook` and `--model` flags for individual demos
## Testing
All tests verified:
- [x] Unit tests pass (57 tests)
- [x] Integration tests pass (25 tests with real Bedrock API)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom/integrations/strands/`)
- [x] Formatting passes (`ruff format --check`)
- [x] Demo runs successfully with ~50% token savings
## Test Output
```
$ pytest tests/integrations/test_strands/ -v
=================== 82 passed in 90.09s ===================
$ ruff check headroom/integrations/strands/ --ignore E402
All checks passed!
$ mypy headroom/integrations/strands/ --ignore-missing-imports
Success: no issues found
```
## Demo Results
```
╭────────────────────────────────────────────────────────────╮
│ HeadroomHookProvider Results │
│────────────────────────────────────────────────────────────│
│ Tokens BEFORE compression: 51,961 │
│ Tokens AFTER compression: 25,658 │
│ Tokens SAVED: 26,303 (50.6%) │
╰────────────────────────────────────────────────────────────╯
```
2026-01-31 00:31:37 -08:00
|
|
|
### strands_bedrock_demo.py
|
|
|
|
|
|
|
|
|
|
AWS Strands Agents + Bedrock integration demo. Showcases two Headroom integration patterns:
|
|
|
|
|
|
|
|
|
|
1. **HeadroomHookProvider** - Compresses tool outputs in real-time
|
|
|
|
|
2. **HeadroomStrandsModel** - Optimizes entire conversation context
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
# Configure AWS credentials
|
|
|
|
|
export AWS_ACCESS_KEY_ID='your-access-key'
|
|
|
|
|
export AWS_SECRET_ACCESS_KEY='your-secret-key'
|
|
|
|
|
export AWS_DEFAULT_REGION='us-west-2' # Optional, defaults to us-west-2
|
|
|
|
|
|
|
|
|
|
# Or use AWS profile
|
|
|
|
|
export AWS_PROFILE='your-profile-name'
|
|
|
|
|
|
|
|
|
|
# Run the full demo (both integration patterns)
|
|
|
|
|
python examples/strands_bedrock_demo.py
|
|
|
|
|
|
|
|
|
|
# Run only the hook provider demo
|
|
|
|
|
python examples/strands_bedrock_demo.py --hook
|
|
|
|
|
|
|
|
|
|
# Run only the model wrapper demo
|
|
|
|
|
python examples/strands_bedrock_demo.py --model
|
|
|
|
|
|
|
|
|
|
# Specify a different AWS region
|
|
|
|
|
python examples/strands_bedrock_demo.py --region us-east-1
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The demo uses Claude 3 Haiku via Bedrock for cost efficiency. It creates agents with
|
|
|
|
|
4 tools that return verbose JSON output (search results, logs, database records, metrics)
|
|
|
|
|
and displays compression statistics with visual comparisons.
|
|
|
|
|
|
|
|
|
|
**Requirements:**
|
|
|
|
|
- AWS account with Bedrock enabled
|
|
|
|
|
- Claude 3 Haiku model access in your region
|
|
|
|
|
- `pip install strands-agents headroom-ai[strands]`
|
|
|
|
|
|
2026-01-07 11:36:44 -08:00
|
|
|
## Running Examples
|
|
|
|
|
|
|
|
|
|
All examples can be run from the repository root:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
# Install dependencies
|
|
|
|
|
pip install -e ".[dev]"
|
|
|
|
|
|
|
|
|
|
# Run any example
|
|
|
|
|
python examples/<example_name>.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Expected Results
|
|
|
|
|
|
|
|
|
|
| Example | Token Savings | Notes |
|
|
|
|
|
|---------|---------------|-------|
|
|
|
|
|
| basic_usage | 50-70% | Simple tool output compression |
|
|
|
|
|
| langchain_demo | 70-85% | Real agent with multiple tools |
|
|
|
|
|
| mcp_demo | 60-80% | MCP tool outputs |
|
feat: Add AWS Strands Agents SDK integration
## Description
Add Headroom integration with AWS Strands Agents SDK, enabling automatic
context optimization and tool output compression for Strands-based agents.
Fixes #14
## Type of Change
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update
## Changes Made
### Core Integration (`headroom/integrations/strands/`)
- **HeadroomHookProvider** - Implements Strands `HookProvider` interface for
automatic tool output compression via `AfterToolCallEvent`. Compresses
verbose tool outputs before they enter conversation context.
- **HeadroomStrandsModel** - Model wrapper that extends Strands `Model` base
class for message-level optimization. Implements all required abstract
methods: `stream()`, `get_config()`, `update_config()`, `structured_output()`.
- **Provider auto-detection** - Automatically detects appropriate Headroom
provider (Anthropic, OpenAI, Google) based on wrapped Strands model type.
- **`strands-agents` as optional dependency** - Install with
`pip install headroom-ai[strands]`
### Testing (`tests/integrations/test_strands/`)
- **Real integration tests (25 tests)** - Use actual AWS Bedrock API calls
with Claude 3 Haiku. Skip automatically when credentials unavailable.
- **Unit tests (57 tests)** - Mock-based tests for internal logic, edge cases,
and error handling. No credentials required.
### Demo (`examples/strands_bedrock_demo.py`)
- Interactive demo showcasing both integration patterns
- Visual before/after compression comparison with token savings
- 4 verbose tools (search, logs, database, metrics) demonstrating real savings
- Supports `--hook` and `--model` flags for individual demos
## Testing
All tests verified:
- [x] Unit tests pass (57 tests)
- [x] Integration tests pass (25 tests with real Bedrock API)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom/integrations/strands/`)
- [x] Formatting passes (`ruff format --check`)
- [x] Demo runs successfully with ~50% token savings
## Test Output
```
$ pytest tests/integrations/test_strands/ -v
=================== 82 passed in 90.09s ===================
$ ruff check headroom/integrations/strands/ --ignore E402
All checks passed!
$ mypy headroom/integrations/strands/ --ignore-missing-imports
Success: no issues found
```
## Demo Results
```
╭────────────────────────────────────────────────────────────╮
│ HeadroomHookProvider Results │
│────────────────────────────────────────────────────────────│
│ Tokens BEFORE compression: 51,961 │
│ Tokens AFTER compression: 25,658 │
│ Tokens SAVED: 26,303 (50.6%) │
╰────────────────────────────────────────────────────────────╯
```
2026-01-31 00:31:37 -08:00
|
|
|
| strands_bedrock_demo | 60-85% | Strands + Bedrock with verbose tools |
|
2026-01-07 11:36:44 -08:00
|
|
|
| real_world_eval | 50-90% | Varies by scenario |
|
|
|
|
|
|
|
|
|
|
## Troubleshooting
|
|
|
|
|
|
|
|
|
|
**ModuleNotFoundError: No module named 'headroom'**
|
|
|
|
|
|
|
|
|
|
Run from the repository root with PYTHONPATH:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
PYTHONPATH=. python examples/basic_usage.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Or install in development mode:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
pip install -e .
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
**API Key Errors**
|
|
|
|
|
|
|
|
|
|
Ensure your API keys are set:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
export OPENAI_API_KEY='sk-...'
|
|
|
|
|
export ANTHROPIC_API_KEY='sk-ant-...'
|
|
|
|
|
```
|
feat: Add AWS Strands Agents SDK integration
## Description
Add Headroom integration with AWS Strands Agents SDK, enabling automatic
context optimization and tool output compression for Strands-based agents.
Fixes #14
## Type of Change
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update
## Changes Made
### Core Integration (`headroom/integrations/strands/`)
- **HeadroomHookProvider** - Implements Strands `HookProvider` interface for
automatic tool output compression via `AfterToolCallEvent`. Compresses
verbose tool outputs before they enter conversation context.
- **HeadroomStrandsModel** - Model wrapper that extends Strands `Model` base
class for message-level optimization. Implements all required abstract
methods: `stream()`, `get_config()`, `update_config()`, `structured_output()`.
- **Provider auto-detection** - Automatically detects appropriate Headroom
provider (Anthropic, OpenAI, Google) based on wrapped Strands model type.
- **`strands-agents` as optional dependency** - Install with
`pip install headroom-ai[strands]`
### Testing (`tests/integrations/test_strands/`)
- **Real integration tests (25 tests)** - Use actual AWS Bedrock API calls
with Claude 3 Haiku. Skip automatically when credentials unavailable.
- **Unit tests (57 tests)** - Mock-based tests for internal logic, edge cases,
and error handling. No credentials required.
### Demo (`examples/strands_bedrock_demo.py`)
- Interactive demo showcasing both integration patterns
- Visual before/after compression comparison with token savings
- 4 verbose tools (search, logs, database, metrics) demonstrating real savings
- Supports `--hook` and `--model` flags for individual demos
## Testing
All tests verified:
- [x] Unit tests pass (57 tests)
- [x] Integration tests pass (25 tests with real Bedrock API)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom/integrations/strands/`)
- [x] Formatting passes (`ruff format --check`)
- [x] Demo runs successfully with ~50% token savings
## Test Output
```
$ pytest tests/integrations/test_strands/ -v
=================== 82 passed in 90.09s ===================
$ ruff check headroom/integrations/strands/ --ignore E402
All checks passed!
$ mypy headroom/integrations/strands/ --ignore-missing-imports
Success: no issues found
```
## Demo Results
```
╭────────────────────────────────────────────────────────────╮
│ HeadroomHookProvider Results │
│────────────────────────────────────────────────────────────│
│ Tokens BEFORE compression: 51,961 │
│ Tokens AFTER compression: 25,658 │
│ Tokens SAVED: 26,303 (50.6%) │
╰────────────────────────────────────────────────────────────╯
```
2026-01-31 00:31:37 -08:00
|
|
|
|
|
|
|
|
**AWS Credentials Errors (for Strands demo)**
|
|
|
|
|
|
|
|
|
|
Ensure AWS credentials are configured:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
# Option 1: Environment variables
|
|
|
|
|
export AWS_ACCESS_KEY_ID='your-access-key'
|
|
|
|
|
export AWS_SECRET_ACCESS_KEY='your-secret-key'
|
|
|
|
|
|
|
|
|
|
# Option 2: AWS profile
|
|
|
|
|
export AWS_PROFILE='your-profile-name'
|
|
|
|
|
|
|
|
|
|
# Option 3: AWS credentials file (~/.aws/credentials)
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Also ensure Bedrock and the Claude 3 Haiku model are enabled in your AWS account.
|