feat: add CrewAI and AutoGen tool compression integrations (#1384)

## Description

Add CrewAI and AutoGen tool compression integrations, following the same
patterns as the existing LangChain agent integration
(`HeadroomToolWrapper` / `wrap_tools_with_headroom`). Both delegate
compression to `compress_tool_result()` from the MCP integration, with
per-tool metrics tracking via `ToolCompressionMetrics` /
`ToolMetricsCollector`.

Closes #1379

## Type of Change

- [x] New feature (non-breaking change that adds functionality)

## Changes Made

- Add `headroom/integrations/crewai/` — `HeadroomToolWrapper` subclasses
CrewAI `BaseTool`, wraps `_run()` with compression
- Add `headroom/integrations/autogen/` — `HeadroomToolWrapper` wraps
AutoGen `FunctionTool` (sync and async) with compression
- Wire both into `headroom/integrations/__init__.py` with aliased
re-exports (avoids name collision with LangChain's
`HeadroomToolWrapper`)
- Add `[crewai]` and `[autogen]` optional dependency extras to
`pyproject.toml`
- Add 24 unit tests (12 per framework) under `tests/test_integrations/`
- Add `.mdx` doc pages for both frameworks under `docs/content/docs/`
- Update `CHANGELOG.md` with entries under `### Added`

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/integrations/crewai headroom/integrations/autogen tests/test_integrations/crewai tests/test_integrations/autogen
All checks passed!

$ pytest tests/test_integrations/autogen -v
12 passed

$ pytest tests/test_integrations/crewai -v
12 passed
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.11, crewai 1.14.7, autogen-agentchat
0.7.5
- Exact command / steps: Ran standalone adapter demos and benchmark
runner across 4 task types
- Observed result:

| Task | Tokens (raw) | Tokens (compressed) | Savings |
|------|-------------|-------------------|---------|
| Inventory JSON (80 items) | 5,044 | 1,532 | 69.6% |
| Server logs (150 lines) | 8,712 | 314 | 96.4% |
| Analytics query (100 rows) | 10,762 | 10,762 | 0% |
| API docs (20 endpoints) | 8,043 | 8,043 | 0% |

Compression results are identical across CrewAI and AutoGen — expected
since both route through the same `compress_tool_result()` pipeline.

- Not tested: Full end-to-end with a live LLM agent loop (demos test the
compression pipeline standalone). LangGraph not included — headroom
already has `headroom/integrations/langchain/langgraph.py`.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

- LangGraph integration is intentionally excluded — headroom already has
one at `headroom/integrations/langchain/langgraph.py`
- Re-exports in `__init__.py` are aliased (`CrewAIToolWrapper`,
`AutoGenToolWrapper`) to avoid collision with the existing LangChain
`HeadroomToolWrapper`
- Both integrations follow the exact same conventions as the existing
LangChain agents module: optional dep guard, `compress_tool_result()`
delegation, metrics with 1000-entry cap, Google-style docstrings
- `mypy` not checked due to Rust build dependency (`maturin`) that
requires Application Control policy changes on this machine

---------

Co-authored-by: Sneha27feb <sroy27.ai@gmail.com>
This commit is contained in:
Sneha Roy 2026-07-16 01:28:54 +05:30 committed by GitHub
parent 3dd9660d91
commit e8bff1cfe3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 1698 additions and 0 deletions

View file

@ -0,0 +1,150 @@
---
title: AutoGen
description: Automatic tool output compression for AutoGen agents with per-tool metrics tracking.
---
Headroom integrates with [AutoGen](https://github.com/microsoft/autogen) (`autogen-agentchat` >=0.7) to compress tool outputs before they enter the agent's model context. Tool-heavy agents that return large JSON arrays, database results, or verbose logs see 60-90% token reduction.
## Installation
```bash
pip install headroom-ai autogen-agentchat
```
## Quick start
Wrap tools in one line:
```python
from autogen_agentchat.agents import AssistantAgent
from autogen_core.tools import FunctionTool
from headroom.integrations.autogen import wrap_tools_with_headroom
def search_database(query: str) -> str:
"""Search the database and return results."""
return json.dumps({"results": [...], "total": 1000})
tool = FunctionTool(search_database, description="Search the database")
wrapped = wrap_tools_with_headroom([tool])
agent = AssistantAgent(
name="researcher",
model_client=model_client,
tools=wrapped,
)
```
## Per-tool metrics
Track compression stats across all tool invocations:
```python
from headroom.integrations.autogen import get_tool_metrics
metrics = get_tool_metrics()
print(metrics.get_summary())
# {
# 'total_invocations': 25,
# 'total_compressions': 18,
# 'total_chars_saved': 450000,
# 'average_compression_ratio': 0.35,
# 'by_tool': {
# 'search_database': {'invocations': 15, 'compressions': 12, 'chars_saved': 320000},
# }
# }
```
Reset between sessions:
```python
from headroom.integrations.autogen import reset_tool_metrics
reset_tool_metrics()
```
## Custom configuration
Control the compression threshold:
```python
wrapped = wrap_tools_with_headroom(
[search_tool, log_tool],
min_chars_to_compress=500, # Default: 1000
)
```
Use a dedicated metrics collector:
```python
from headroom.integrations.autogen import ToolMetricsCollector, wrap_tools_with_headroom
collector = ToolMetricsCollector()
wrapped = wrap_tools_with_headroom(
[search_tool],
metrics_collector=collector,
)
print(collector.get_summary())
```
## Wrapping individual tools
For finer control, wrap tools individually:
```python
from headroom.integrations.autogen import HeadroomToolWrapper
wrapper = HeadroomToolWrapper(
search_tool,
min_chars_to_compress=500,
)
# Get the wrapped FunctionTool
compressed_tool = wrapper.as_function_tool()
agent = AssistantAgent(
name="researcher",
model_client=model_client,
tools=[compressed_tool],
)
```
## Async support
AutoGen tools are natively async. The wrapper handles both sync and async
tool functions transparently:
```python
async def async_search(query: str) -> str:
"""Async database search."""
results = await db.search(query)
return json.dumps(results)
tool = FunctionTool(async_search, description="Async search")
wrapped = wrap_tools_with_headroom([tool])
# Compression works identically for async tools
```
## How it works
AutoGen routes tool execution through `FunctionTool`, which wraps a plain
Python function. The function's return value is stringified and becomes
`FunctionExecutionResult.content` — what the LLM reads on its next turn.
`HeadroomToolWrapper` creates a new `FunctionTool` with a wrapper function that:
1. Calls the original function
2. Checks if the stringified output exceeds `min_chars_to_compress`
3. If so, compresses via Headroom's `compress_tool_result()`
4. Records metrics and returns the compressed string
The wrapper preserves the original tool's name, description, and parameter
schema, so it works as a drop-in replacement.
### Why not `tool_call_summary_formatter`?
AutoGen's `AssistantAgent` accepts a `tool_call_summary_formatter` parameter,
which looks like a natural hook. However, it only controls the **final summary
message** emitted after the tool loop exits — it does not touch the raw
`FunctionExecutionResult` that gets added to `model_context` (what the LLM
actually reads). Wrapping the function is the only clean interception point.

View file

@ -0,0 +1,123 @@
---
title: CrewAI
description: Automatic tool output compression for CrewAI agents with per-tool metrics tracking.
---
Headroom integrates with [CrewAI](https://github.com/crewAIInc/crewAI) to compress tool outputs before they enter the agent's LLM context. Tool-heavy agents that return large JSON arrays, database results, or verbose logs see 60-90% token reduction.
## Installation
```bash
pip install headroom-ai crewai
```
## Quick start
Wrap tools in one line:
```python
from crewai import Agent, Crew, Task
from crewai.tools.base_tool import tool
from headroom.integrations.crewai import wrap_tools_with_headroom
@tool
def search_database(query: str) -> str:
"""Search the database and return results."""
return json.dumps({"results": [...], "total": 1000})
wrapped = wrap_tools_with_headroom([search_database])
agent = Agent(
role="Researcher",
goal="Answer questions using data",
backstory="You research things.",
tools=wrapped,
)
task = Task(description="Find all active users", agent=agent, expected_output="Summary")
crew = Crew(agents=[agent], tasks=[task])
crew.kickoff()
```
## Per-tool metrics
Track compression stats across all tool invocations:
```python
from headroom.integrations.crewai import get_tool_metrics
metrics = get_tool_metrics()
print(metrics.get_summary())
# {
# 'total_invocations': 25,
# 'total_compressions': 18,
# 'total_chars_saved': 450000,
# 'average_compression_ratio': 0.35,
# 'by_tool': {
# 'search_database': {'invocations': 15, 'compressions': 12, 'chars_saved': 320000},
# 'fetch_logs': {'invocations': 10, 'compressions': 6, 'chars_saved': 130000},
# }
# }
```
Reset between sessions:
```python
from headroom.integrations.crewai import reset_tool_metrics
reset_tool_metrics()
```
## Custom configuration
Control the compression threshold:
```python
wrapped = wrap_tools_with_headroom(
[search_database, fetch_logs],
min_chars_to_compress=500, # Default: 1000
)
```
Use a dedicated metrics collector instead of the global one:
```python
from headroom.integrations.crewai import ToolMetricsCollector, wrap_tools_with_headroom
collector = ToolMetricsCollector()
wrapped = wrap_tools_with_headroom(
[search_database],
metrics_collector=collector,
)
# After crew run
print(collector.get_summary())
```
## Wrapping individual tools
For finer control, wrap tools individually:
```python
from headroom.integrations.crewai import HeadroomToolWrapper
wrapper = HeadroomToolWrapper(
search_database,
min_chars_to_compress=500,
)
# Use wrapper directly — it's a BaseTool
agent = Agent(role="Researcher", tools=[wrapper], ...)
```
## How it works
CrewAI tools extend `BaseTool` with a `run()` → `_run()` execution flow.
`HeadroomToolWrapper` subclasses `BaseTool` and overrides `_run()` to:
1. Call the original tool's `run()` method
2. Check if the output exceeds `min_chars_to_compress`
3. If so, compress via Headroom's `compress_tool_result()`
4. Record metrics and return the compressed output
The wrapper preserves the original tool's name, description, and argument
schema, so it works as a drop-in replacement anywhere CrewAI expects a tool.