akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
2630 commits 410 branches 169 tags 142 MiB
Commit graph

4 commits

Author SHA1 Message Date
Abhay Singh
1b8c11ebfb
fix(proxy/openai): apply output shaping on /v1/chat/completions (#2328) 
## Description

Fixes #2302.

Output shaping (`HEADROOM_OUTPUT_SHAPER=1`) verbosity steering is wired
into the Anthropic `/v1/messages` handler and the OpenAI `/v1/responses`
handler, but never into `handle_openai_chat`. OpenAI-compatible clients
that route through `/v1/chat/completions` — GitHub Copilot CLI,
opencode, older SDKs — therefore got zero output savings, and `headroom
output-savings` reported:

```
No shaped requests recorded yet.
```

`handle_openai_chat` referenced verbosity only for cache-key
construction, never for actual shaping. The shared helpers
(`OutputShaperSettings`, `resolve_verbosity_level`, `assign_arm`,
`classify_turn`) existed but were not called from the chat path.

## Fix

Run the same shaping block the Anthropic handler already uses, at the
end of `handle_openai_chat` (after every other body mutation, before the
upstream forward, skipped under `x-headroom-bypass`):

- conversation-stable holdout via
`assign_arm(conversation_key_from_body(body), holdout)` —
`conversation_key_from_body` already reads `messages`, so it works
unchanged for a chat body;
- stratum labelling on the transforms channel so the outcome funnel
feeds the output-savings ledger from the chat path;
- for the treatment arm, verbosity steering via a new
`shape_openai_chat_request`.

The one genuinely new piece is a chat-specific steering injector.
Anthropic carries the system prompt in a top-level `system` field and
Responses in `instructions`; **chat/completions carries it as a `role:
"system"` message inside `messages`**, which neither existing injector
touches. `apply_openai_chat_verbosity_steering`:

- appends the byte-stable steering block to the tail of the last
`system`/`developer` message (idempotent via the
`<headroom_output_shaping>` sentinel, and it swaps cleanly when the
level changes);
- handles both string content and the content-part list form (`[{"type":
"text", ...}]`);
- inserts a `role: "system"` message at the front only when the request
has no system message.

Because a whole conversation is stably treatment or control and the
block text is fixed per level, a treatment conversation's steering is
byte-stable across turns, so the provider prefix cache is not thrashed.
Effort routing is intentionally not applied on this path —
`route_effort` writes Anthropic-shaped `output_config`/thinking config
with no portable chat/completions equivalent — so only the
token-reducing verbosity lever runs. Mutating `body` in place is enough
on this path; the outbound request serializes `body` fresh, so no
body-mutation tracker is needed.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/output_steering.py`: add
`apply_openai_chat_verbosity_steering` (inject the steering block into
the chat `messages` system prompt).
- `headroom/proxy/output_shaper.py`: add `shape_openai_chat_request`
(verbosity-only chat shaper) and export both new names.
- `headroom/proxy/handlers/openai.py`: run the holdout/stratum + shaping
block at the end of `handle_openai_chat`, mirroring the Anthropic
handler and respecting bypass.
- `tests/test_output_steering.py`: cover the injector (append,
idempotency, level swap, insert-when-absent, list content, level-0
no-op).
- `tests/test_output_shaper.py`: cover `shape_openai_chat_request`
(disabled no-op, applies steering, level override, stable second pass).
- `CHANGELOG.md`: Bug Fixes entry.

## Testing

- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ uvx ruff@0.15.17 check headroom/proxy/output_steering.py headroom/proxy/output_shaper.py headroom/proxy/handlers/openai.py tests/test_output_steering.py tests/test_output_shaper.py
All checks passed!
$ uvx ruff@0.15.17 format --check <same files>
5 files already formatted
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/output_steering.py headroom/proxy/output_shaper.py
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOMs this box (ML-stack import), so I
reproduced the injector with a dependency-free script and left the full
pytest to CI.
- Exact command / steps: replicated
`apply_openai_chat_verbosity_steering` (and the
`steering_text`/`replace_or_append_steering_block` primitives it uses)
and exercised: an existing string system message, an existing
content-part list, no system message, re-apply at the same level, and a
level swap.
- Observed result: the steering block is appended to the system message
while user turns and message order are untouched; re-applying at the
same level is a no-op; a level change replaces the block (exactly one
remains); a request with no system message gets one inserted at the
front; level 0 is a no-op. The added unit tests assert the same through
`shape_openai_chat_request`.
- Not tested: a live Copilot CLI `/v1/chat/completions` round trip; the
added tests drive the pure shaper/injector directly, matching the
existing `test_output_shaper.py` / `test_output_steering.py` patterns.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

The "unit tests pass locally" box is unchecked because a local pytest
run imports the ML stack and OOMs this box; the added tests are pure (no
ML imports) and run under the normal CI pytest job, and the injector
behavior is corroborated by the standalone proof above. Effort routing
on chat/completions is deliberately out of scope here (no portable
equivalent to the Anthropic effort levers); this PR restores the
verbosity-steering savings the issue reports as missing, and effort
routing for chat can follow separately if wanted.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
 2026-07-17 16:16:29 -07:00
Rudimar Ronsoni
4bd3ddfaa5
fix(opencode): use local MCP config (#1383) 
## Description

Fixes OpenCode Headroom MCP configuration across wrap, MCP
install/status/uninstall, and persistent install docs/CLI.

OpenCode was being configured to use a remote HTTP MCP endpoint at
`/mcp`, but the Headroom proxy does not expose MCP there. The correct
OpenCode configuration is a local stdio MCP server that runs `headroom
mcp serve`.

Closes #1380

## Type of Change

- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [x] Documentation update
- [x] Tests

## Changes Made

- Changed OpenCode MCP registration to emit `type: "local"` with
`command: ["headroom", "mcp", "serve"]`.
- Changed OpenCode MCP environment serialization from `env` to
OpenCode's `environment` key, while still reading legacy `env` entries.
- Removed generated remote `/mcp` entries from OpenCode wrap/runtime
config.
- Made `wrap opencode --no-mcp` skip persistent `mcp.headroom`
injection.
- Kept provider-only OpenCode config injection from writing MCP; MCP
persistence is owned by the registrar path.
- Made `headroom mcp status` and `headroom mcp uninstall` use the
registrar lifecycle so OpenCode is covered.
- Added `opencode` to persistent install `--target` choices.
- Clarified OpenCode persistent install docs to use `--scope provider`
for direct `opencode.json` edits.
- Added regression coverage for registrar serialization, wrap behavior,
runtime config, provider-scope install, MCP CLI lifecycle, and install
target parsing.

## Testing

- [x] `rtk .venv/bin/python -m pytest tests/test_mcp_registry
tests/test_cli/test_mcp.py tests/test_cli/test_wrap_opencode.py
tests/test_providers_opencode_config.py
tests/test_providers_opencode_install.py tests/test_install -q`
- [x] Result after absorbing #1381 overlap: `263 passed, 1 skipped`
- [x] Targeted Ruff check passed for the changed Python/test files.
- [x] Targeted Ruff format check passed for the changed Python/test
files.
- [x] Isolated HOME smoke tests with real `opencode mcp list --pure`.

## Real Behavior Proof

- `headroom mcp install --agent opencode --proxy-url
http://127.0.0.1:9000 --force` against an isolated HOME wrote a valid
local OpenCode MCP entry with `environment.HEADROOM_PROXY_URL`.
- `opencode mcp list --pure` against that isolated HOME connected to
`headroom mcp serve`.
- `headroom wrap opencode --prepare-only --no-rtk --no-serena --port
9001` wrote local MCP plus provider config.
- `headroom wrap opencode --prepare-only --no-rtk --no-serena --no-mcp
--port 9002` wrote provider config without `mcp.headroom`.
- Generated runtime `OPENCODE_CONFIG_CONTENT` was accepted by `opencode
mcp list --pure`; `include_mcp=False` reported no MCP servers.
- `headroom mcp status` detected the isolated OpenCode config and read
the custom proxy URL.
- `headroom mcp uninstall` removed `mcp.headroom` from the isolated
OpenCode config while leaving provider config intact.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review
 2026-07-06 06:22:15 -07:00
Andrew McFague
0b0133b7fd
Wire OpenAI Responses output shaping (#1438) 
## Description

Wire output shaping for OpenAI Responses traffic across HTTP
`/v1/responses` and Codex WebSocket `response.create` frames. The change
adds provider-specific shaping for `instructions`, `reasoning.effort`,
and `text.verbosity` while keeping Anthropic request mutation separate.

Review follow-up: merged byte-faithful `/v1/responses` forwarding from
#1557 and marks shaped HTTP Responses payloads as `body_mutated=True`,
so retry forwarding sends the shaped body instead of the original raw
bytes.

## Type of Change

- [ ] Bug fix (non-breaking change fixes an issue)
- [x] New feature (non-breaking change adds functionality)
- [ ] Breaking change (fix or feature would cause existing functionality
change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added OpenAI Responses output shaping for `instructions`,
`reasoning.effort`, and `text.verbosity`.
- Wired shaping into `/v1/responses` HTTP and Codex WebSocket
`response.create` paths.
- Preserved `x-headroom-bypass` and `HEADROOM_OUTPUT_HOLDOUT` behavior.
- Added output-shaper transform labels for verbosity, text verbosity,
reasoning effort, holdout control, and strata.
- Updated output-savings conversation keys for Responses payloads and WS
`response.create` envelopes.
- Counted WS frame payload tokens when assigning output-savings strata.
- Merged byte-faithful `/v1/responses` forwarding from #1557 and kept
shaped HTTP bodies on the mutated-forwarding path.
- Added tests for classification, shaping, holdout, bypass, labels, WS
strata, and byte-faithful forwarding compatibility.
- Updated `CHANGELOG.md` for OpenAI Responses output-shaping support.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run --extra dev python -m pytest tests/test_openai_codex_ws_lifecycle.py tests/test_openai_responses_output_shaper.py tests/test_codex_responses_passthrough_bytes.py tests/test_output_shaper.py tests/test_output_savings.py -q
110 passed, 1 warning in 1.49s

$ uv run --extra dev ruff check headroom/proxy/handlers/openai.py tests/test_openai_responses_output_shaper.py tests/test_codex_responses_passthrough_bytes.py tests/test_openai_codex_ws_lifecycle.py tests/test_output_shaper.py tests/test_output_savings.py
All checks passed!

$ git diff --check
No whitespace errors.
```

## Real Behavior Proof

- Environment: local macOS checkout, branch
`output-shaper-openai-responses`.
- Exact command / steps: ran targeted pytest, ruff, and diff checks
listed above.
- Observed result: targeted tests passed with an existing FastAPI
TestClient deprecation warning; ruff passed; diff check passed.
- Not tested: full repository test suite, live OpenAI traffic, browser
dashboard rendering, full `mypy headroom`.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows project's style guidelines
- [x] I performed self-review of my code
- [x] I commented my code, particularly in hard-to-understand areas
- [x] I made corresponding changes to documentation
- [x] My changes generate no new warnings
- [x] I added tests that prove my fix is effective or that my feature
works
- [x] New and existing unit tests pass locally with my changes
- [x] I updated `CHANGELOG.md` if applicable

## Screenshots

N/A

## Additional Notes

- Non-applicable Type Change items are left unchecked.
- The pytest warning comes from `fastapi.testclient` importing Starlette
TestClient and was not introduced by this change.
- `CHANGELOG.md` includes entries for OpenAI Responses output-shaping
support and byte-faithful `/v1/responses` forwarding compatibility.

---------

Co-authored-by: obchain <riteshnikhoriya94@gmail.com>
 2026-07-05 13:59:21 -07:00
Tejas Chopra
a99dc61424
feat: output-token reduction — verbosity shaper, per-user learning, counterfactual savings (#965) 
## Description

Adds the first levers that reduce the tokens the model **writes back**
(output), complementing Headroom's existing input compression. Output
costs 5× input on Opus-class models and is full of waste (ceremony,
restated code, deep "thinking" on routine steps). Two phases in one
self-contained PR off `main`: the request-side output shaper, then
per-user verbosity learning plus an honest counterfactual savings
estimator and dashboard surfacing.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **Output shaper** (`output_shaper.py`, opt-in
`HEADROOM_OUTPUT_SHAPER=1`): cache-safe verbosity steering appended to
the system-prompt tail (5 levels); effort routing that lowers
`output_config.effort` on mechanical tool-result continuations; legacy
`thinking.budget_tokens` clamp. Never injects effort where absent, never
toggles `thinking.type`.
- **`headroom learn --verbosity`**: mines Claude Code transcripts for
behavioral signals (interrupts, length-adaptive fast-skips, echo ratio),
recommends a verbosity level (heuristic + optional `--llm-judge`), and
seeds the savings baseline.
- **Counterfactual estimator** (`output_savings.py`): per-stratum
synthetic-control (estimated) + A/B holdout (measured) with a propagated
95% CI; conversation-stable arm assignment for A/B validity and
prefix-cache safety.
- **AIMD verbosity controller** (`verbosity_controller.py`):
additive-increase / fast-back-off state machine; live signal emission
gated off by default.
- **Wiring + surfaces**: shaper resolves the learned level; recording
rides the existing `transforms_applied` channel through the outcome
funnel (no `RequestOutcome` changes); `headroom output-savings` CLI;
dashboard "Output Tokens Saved" card.
- **Docs**: simple-words user guide + design doc with the counterfactual
methodology.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_output_savings.py tests/test_output_savings_cli.py \
        tests/test_verbosity_learn.py tests/test_verbosity_controller.py \
        tests/test_output_shaper.py -q
94 passed in 0.54s

$ pytest tests/test_request_outcome.py tests/test_handler_outcome_tag_invariant.py \
        tests/test_proxy_dashboard_stats_cache.py -q
44 passed

$ ruff format --check .
831 files already formatted

$ mypy headroom --ignore-missing-imports
Success: no issues found in 361 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12 (`.venv`), `anthropic` 0.76, live API
model `claude-opus-4-8`.
- Exact command / steps: `HEADROOM_OUTPUT_SHAPER=1`; `headroom learn
--verbosity --apply` (seeds level + baseline); `python
scripts/eval_output_shaper.py A` (live before/after); simulate holdout
traffic then `headroom output-savings`.
- Observed result: code-review ask — baseline 1,750 output tokens → L2
1,354 (−22.7%) → L3 599 (−65.8%), same bugs found. `learn --verbosity`
on 24 real sessions → 11% interrupt / 26% fast-skip → L3 (high
confidence). Measured A/B path → 31.7% reduction (95% CI 27.7%–35.7%).
94 new tests + 44 existing outcome/dashboard tests green; ruff + mypy
clean.
- Not tested: live streaming-path recording exercised only via unit
tests (the `transforms_applied` funnel is shared across paths); runtime
AIMD signal emission is gated off by default and not exercised live.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

Output savings are counterfactual (we never observe what the model
*would* have written), so the estimator separates **estimated** (vs a
learned baseline) from **measured** (A/B holdout via
`HEADROOM_OUTPUT_HOLDOUT`) and always reports a confidence band — never
a single made-up number. CHANGELOG left unchecked (release-please
manages it). Runtime AIMD self-tuning is intentionally a TODO
(controller built/tested; live signal emission gated behind
`HEADROOM_VERBOSITY_AUTOTUNE`).
 2026-06-16 21:06:43 -07:00