Commit graph

3 commits

Author SHA1 Message Date
Focused Instability
914a60a2b0
feat(proxy): per-project savings breakdown on the dashboard (claude, codex, aider, copilot, cursor) (#803)
## Summary

Adds a **per-project savings breakdown** to the proxy dashboard,
covering **all wrap-supported agents: Claude Code, Codex, aider,
Copilot, and Cursor**. Spec and rationale in #802 (feature-request issue
— happy to adjust scope per maintainer feedback).

How it works — two attribution channels, by client capability:

**Header channel** (clients that can send custom headers):
- `headroom wrap claude` appends `X-Headroom-Project: <basename(cwd)>`
to `ANTHROPIC_CUSTOM_HEADERS` (a user-supplied x-headroom-project header
always wins; other user headers preserved).
- `headroom wrap codex` extends the injected
`[model_providers.headroom]` block with `env_http_headers = {
"X-Headroom-Project" = "HEADROOM_PROJECT" }` and sets `HEADROOM_PROJECT`
per launch — Codex only sends the header when the env var is set, so the
static config stays inert outside wrap.

**Base-URL prefix channel** (clients that cannot send custom headers):
- The proxy accepts `/p/<url-encoded-name>/...`; middleware strips the
prefix before routing (before Starlette caches the URL) and binds the
project. The explicit header wins over the prefix.
- `headroom wrap aider` points `OPENAI_API_BASE` / `ANTHROPIC_BASE_URL`
at the prefixed URL.
- `headroom wrap copilot` points `COPILOT_PROVIDER_BASE_URL` at the
prefixed URL (BYOK anthropic/openai provider types and the
GitHub-subscription path).
- `headroom wrap cursor` prints the prefixed Override Base URL in its
setup instructions, with a note explaining the attribution.

**Shared plumbing:**
- New `headroom/proxy/project_context.py`: header classification, `/p/`
prefix split/strip, URL-prefix builder, and a contextvar bound per
request (HTTP middleware + WS accept for the Codex responses bridge);
the outcome funnel resolves it (explicit `RequestOutcome.project` wins),
stamps `RequestLog.tags["project"]`, and forwards it through
`PrometheusMetrics.record_request` into the `SavingsTracker`.
- `SavingsTracker`: persisted state gains a `projects` map (requests,
tokens saved, savings USD, input tokens/cost, last activity). Schema v2
→ v3 with transparent forward migration (v2 files load cleanly,
`projects` starts empty). Names sanitized (printable-only, 128-char
cap); map capped at 50 projects, evicting the smallest bucket.
- `/stats` exposes `savings.per_project` (and
`projects`/`projects_limit` inside `persistent_savings`);
`/stats-history` exposes `projects`.
- Dashboard gains a "Per-Project Savings" table mirroring the per-model
table (Alpine `x-text` only — names are user-supplied, no HTML
injection).

**Behavior changes:** none for unattributed traffic — no header and no
prefix means no bucket, aggregate totals exactly as before
(regression-tested: legacy `/stats` and `/stats-history` shapes are
pinned by tests).

## Real behavior proof

**Setup tested on:** macOS (Darwin 25.5.0), Python 3.11.9, `pip install
-e ".[dev]"`, proxy on spare ports with isolated
`HEADROOM_SAVINGS_PATH`; real Claude Code CLI (subscription auth) and
real Codex CLI (ChatGPT auth) as clients.

**Header channel — exact steps:**

```
HEADROOM_SAVINGS_PATH=/tmp/headroom-proof-savings.json .venv/bin/python -m headroom.cli proxy --port 9123  # background
cd /tmp/proof-alpha && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK"
cd /tmp/proof-beta  && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK"
cd /tmp/proof-beta  && headroom wrap codex  --port 9123 --no-rtk --no-mcp --no-serena -- exec --skip-git-repo-check "Reply with exactly: OK"
```

**Observed** (copied live `/stats` output; all runs replied `OK` through
the proxy):

```json
{
 "proof-beta": {
  "requests": 3, "tokens_saved": 140, "compression_savings_usd": 0.0007,
  "total_input_tokens": 38581, "total_input_cost_usd": 0.360433,
  "last_activity_at": "2026-06-10T09:19:17Z", "savings_percent": 0.36
 },
 "proof-alpha": {
  "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0,
  "total_input_tokens": 9050, "total_input_cost_usd": 0.32245,
  "last_activity_at": "2026-06-10T09:18:09Z", "savings_percent": 0.0
 }
}
```

`proof-beta` aggregates one Claude Code turn + one Codex `exec` run
(Codex attribution flows through `env_http_headers`).

**Prefix channel — exact steps** (the same mechanism the
aider/copilot/cursor wraps emit, driven by a real client):

```
.venv/bin/python -m headroom.cli proxy --port 9124  # background, isolated savings path
ANTHROPIC_BASE_URL="http://127.0.0.1:9124/p/aider-style-project" claude -p "Reply with exactly: OK"
```

**Observed:**

```json
{
 "aider-style-project": {
  "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0,
  "total_input_tokens": 8571, "total_input_cost_usd": 1.018575,
  "last_activity_at": "2026-06-10T10:10:13Z", "savings_percent": 0.0
 }
}
```

`/stats-history` returned `schema_version: 3` with the same `projects`
keys; `GET /dashboard` HTML contains the new "Per-Project Savings"
table; persisted state survived a tracker reload; a hand-written v2
savings file loaded cleanly with an empty `projects` map.

**What I did NOT test live:** the actual aider/Copilot/Cursor binaries
end-to-end (their wraps emit exactly the prefixed URLs exercised above —
unit tests pin the emitted env/URLs); Codex subscription-mode routing
via the built-in `openai` provider (no provider headers there — such
traffic simply stays unattributed); multi-worker uvicorn; Windows.

## Tests

- `tests/test_proxy_project_savings.py` (17 tests): sanitization, header
classification, `/p/` prefix split + URL-builder round-trip, tracker
aggregation/persistence/migration/cardinality-cap/state-sanitization,
funnel→`/stats` end-to-end, middleware binding for header + prefix +
precedence, plus regression tests pinning legacy
`/stats`/`/stats-history` shape and unattributed-traffic totals.
- Wrap/provider tests extended: `test_cli/test_wrap_helpers.py`,
`test_cli/test_wrap_codex.py`, `test_provider_aider.py`,
`test_provider_cursor.py`, `test_provider_copilot_wrap.py` (prefixed
env/URLs, user-override wins, no duplicate header, TOML block contents,
block strip, setup-line note).
- Full `pytest` suite run locally; `ruff check` + `ruff format` clean on
all touched files.
- `CHANGELOG.md` updated.

## Dependencies

None added or bumped.

Closes #802 (pending maintainer 👍 per CONTRIBUTING — raised there first
with the full spec).

---------

Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
2026-06-10 21:04:45 -05:00
Garm
efd2ac1ca4 chore: renormalize line endings to LF
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but
74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings,
violating that contract. Every macOS/Linux clone reports these files as
"modified" on fresh checkout because git's diff engine sees the stored
bytes don't match the attribute contract, even though the working tree
and index match byte-for-byte.

Running `git add --renormalize .` rewrites each affected blob so the
stored form matches the attribute declaration. No semantic changes —
every affected file's diff is "N insertions, N deletions" with inserts
and deletes being the same lines modulo line endings.

Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub
blame skip this mechanical commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 15:33:30 +02:00
JerrettDavis
7831620eca test: expand provider slice coverage
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-22 21:58:03 -05:00