11ab5f83a1
## Summary - add a containerized differential network capture harness for Claude Code direct vs Claude Code routed through Headroom - capture both Headroom client-side traffic and Headroom upstream traffic with sanitized mitmproxy JSONL output - add `headroom capture network-diff` to compare captures and produce Markdown/JSON reports, including Anthropic tool-count/tool-byte deltas for deferred-tool investigations - add an on-demand GitHub Actions workflow for the harness; it only runs via `workflow_dispatch`, with live Claude Code/Anthropic capture gated on `ANTHROPIC_API_KEY` - document the workflow and ignore generated capture artifacts ## Validation - `C:\git\headroom\.venv\Scripts\python.exe -m pytest tests/test_network_diff_capture.py` - `ruff check headroom/capture headroom/cli/capture.py tests/test_network_diff_capture.py` - `ruff format --check headroom/capture headroom/cli/capture.py tests/test_network_diff_capture.py` - `C:\git\headroom\.venv\Scripts\python.exe -m mypy headroom/capture/network_diff.py headroom/cli/capture.py` - `docker compose -f docker/differential-network-capture/docker-compose.yml --profile run config` - `docker compose -f docker/differential-network-capture/docker-compose.yml --profile run build claude-direct` - `docker run --rm -e CLAUDE_COMMAND="claude --version" headroom-network-diff-claude-direct:latest` - parsed `.github/workflows/network-diff-capture.yml` with PyYAML and confirmed manual-only trigger Live Claude API capture was not run locally because `ANTHROPIC_API_KEY` is not set in this environment. The workflow can run it manually in GitHub Actions when that secret is present; otherwise it emits a visible skip warning and uploads a skipped artifact. ## Notes - Full pre-commit mypy still fails on unrelated Windows `fcntl` attributes in `headroom/subscription/tracker.py`; the feature commit skipped only that hook after narrow mypy passed for the new modules. - `tests/test_release_workflows.py` has two Windows-local failures because it shells out to a missing Unix/Rust command; unrelated workflow checks in that file passed before those failures. - Motivated by https://github.com/chopratejas/headroom/issues/746#issuecomment-4651276818 / Issue #746.
2.6 KiB
Differential Network Capture
Headroom includes a containerized harness for comparing Claude Code traffic sent directly to Anthropic with traffic sent through a Headroom proxy. The harness uses mitmproxy in two isolated lanes, writes sanitized JSONL captures, and then generates Markdown/JSON reports with request route, header, body size, body hash, and JSON payload differences.
Run The Harness
cd docker/differential-network-capture
mkdir -p captures
export ANTHROPIC_API_KEY=...
export CLAUDE_PROMPT="Summarize this repository in one sentence."
docker compose up --build mitm-direct mitm-headroom-upstream headroom-proxy mitm-headroom-client
docker compose --profile run run --rm claude-direct
docker compose --profile run run --rm claude-headroom
The primary captures are written to:
docker/differential-network-capture/captures/direct.jsonldocker/differential-network-capture/captures/headroom-client.jsonl
The Headroom lane also writes
docker/differential-network-capture/captures/headroom-upstream.jsonl, which is
the request Headroom forwards to Anthropic after proxy processing.
By default only api.anthropic.com is logged. Override
CAPTURE_INCLUDE_HOSTS with a comma-separated list to include other hosts.
Generate A Report
headroom capture network-diff \
--direct docker/differential-network-capture/captures/direct.jsonl \
--headroom docker/differential-network-capture/captures/headroom-client.jsonl \
--output docker/differential-network-capture/captures/report.md \
--json-output docker/differential-network-capture/captures/report.json
The report redacts sensitive header values and sensitive query values before
comparison. Request bodies are captured so structural payload differences can be
identified; keep the generated captures/ directory out of commits because it
may contain prompts, tool outputs, and repository context.
For Claude Code deferred-tool investigations, the paired exchange table includes
top-level Anthropic tools counts and serialized tool bytes. A jump from
tools=0->N in the Headroom client lane is evidence that Claude Code eagerly
materialized tool schemas before the request reached Headroom.
Custom Claude Invocation
Set CLAUDE_COMMAND to run the exact command under test in both lanes:
CLAUDE_COMMAND='claude -p "read README.md and summarize the proxy setup"' \
docker compose --profile run run --rm claude-direct
CLAUDE_COMMAND='claude -p "read README.md and summarize the proxy setup"' \
docker compose --profile run run --rm claude-headroom
Use CLAUDE_DIRECT_ARGS and CLAUDE_HEADROOM_ARGS when each lane needs
different flags.