akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
headroom/docs
History
Yasser Sheikh 0dc2e1cb3f
feat(bedrock): cross-region + Converse compression; bundle proxy binary in images (#999) 
## Description

The native Bedrock path (Phase D) compresses + signs
Anthropic-on-Bedrock requests, but
two real-world cases slipped through, and the native binary that powers
it was never
shipped. This PR closes those gaps as a focused set of give-backs.

Aligns with the Rust migration plan (see below).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **Cross-region inference-profile detection** via a new
`bedrock::vendor` module
(`canonical_vendor()`), following the design proposed in #953: strip a
known geo prefix
(`eu.`/`us.`/`apac.`/`global.`) then match the canonical vendor.
Geo-prefixed Anthropic
profiles (`eu.anthropic.…`) now get live-zone compression instead of
being silently
  skipped; geo-prefixed non-Anthropic vendors stay correctly excluded.
- **Converse-body compression (two parts)**:
1. `run_anthropic_compression` no longer bails to passthrough when the
body lacks an
InvokeModel `anthropic_version` envelope; envelope re-emit stays gated
on successful parse.
2. The **live-zone dispatcher now recognizes Bedrock Converse content
blocks**. Converse
blocks carry no `type` discriminator (the variant is the key: `{"text":
…}` vs
Anthropic's `{"type":"text","text":…}`), so real Converse user-message
text was still
passing through uncompressed. A typeless block whose `text` is a JSON
string now routes
through the same surgical text path. Anthropic blocks always carry
`type`, so the
Anthropic path is byte-for-byte unchanged; non-text Converse blocks
(`{"image":…}`,
     `{"toolUse":…}`) stay unrecognized and no-op.
- **Correct `/converse` upstream routing**: the non-streaming handler
resolved the upstream
action from a hard-coded `"invoke"`, so `/converse` requests were
forwarded to Bedrock's
`/invoke` endpoint. It now resolves the action from the inbound path
(`extract_invoke_action`),
mirroring the streaming handler's `extract_streaming_action`. SigV4
signs the same URL it
  forwards, so the signature stays consistent.
- **`aws-config` `sso` feature**: SSO profiles now resolve through the
default credential
chain for SigV4 — the credential chain in `docs/bedrock.md` already
promised SSO; this
  makes the code match.
- **Ship the `headroom-proxy` binary in published images**
(`Dockerfile`): built in the
builder stage (`--locked`, with the cargo registry cache mounted at
`CARGO_HOME`) and
  copied into both the debian and distroless runtime images.
- **Docs** (`docs/bedrock.md`): document cross-region inference profiles
and a "Running the
proxy" section. AWS credentials mount at `/home/nonroot/.aws` (the
default nonroot image
home) where the SDK looks for `~/.aws`, with a note on the root-image
alternative.

## Related issues

- Closes #976 — ship the `headroom-proxy` binary in published images
(this PR implements
  the exact fix proposed there).
- Addresses the **cross-region inference-profile** half of #953 via its
proposed
`canonical_vendor()` design. Non-Anthropic vendor compression parity
(Nova/GLM/MiniMax/
Kimi) is the natural follow-up — `bedrock::vendor` is the shared
resolver it can build on.
- Extends the native Bedrock InvokeModel compression requested in #734
(the Bedrock slice of
  #510) to cross-region profiles and Converse bodies.
- Partially enables #181 (native, Python-free packaging): the native
binary now ships in the
  images, though full Python-free distribution remains out of scope.

## Alignment with the Rust migration plan

Per `docs/spec/022-rust-migration.md`, the migration is **proxy-first**:
`headroom-proxy` is
the deployable Rust artifact, native routes replace Python passthroughs
one at a time
(Stage 4 = provider expansion, Bedrock included), and the binary is
meant to be "built,
tested, and **released together with the Python package**." Two ways
this PR advances that:

- The binary-in-images change makes the codebase do what the spec
already states (ship the
artifact) — closing the gap that forced downstreams to build from
source.
- Hardening the native Bedrock route (cross-region, Converse routing +
body compression) is
exactly the Stage-4 provider-expansion work, keeping the native path at
parity with real
  traffic so it can be the default rather than a passthrough.

## Testing

- [x] Unit tests pass (`cargo test -p headroom-core -p headroom-proxy` —
full suites, 0 failures)
- [x] Linting passes (`cargo clippy -p headroom-core -p headroom-proxy
--all-targets -- -D warnings`)
- [x] Formatting passes (`cargo fmt -- --check`)
- [x] New tests added — `bedrock::vendor` (foundation +
inference-profile matching),
`extract_invoke_action` + converse upstream URL, and live-zone Converse
text-block routing
(`block_has_string_text_field`, converse-vs-anthropic dispatch
equivalence).
- [x] Manual testing performed

### Test Output

```text
$ cargo test -p headroom-core -p headroom-proxy   # all suites: ok, 0 failed
$ cargo clippy -p headroom-core -p headroom-proxy --all-targets -- -D warnings   # Finished, no warnings
$ cargo fmt -- --check                            # clean
# image validation (local, proxy/code extras):
$ docker build --target runtime ...      # debian: /usr/local/bin/headroom-proxy, --help OK
$ docker build --target runtime-slim ... # distroless: binary links + --help OK
```

## Real Behavior Proof

- Environment: native Bedrock proxy against `bedrock-runtime.eu-west-2`,
SSO profile, model
  `eu.anthropic.claude-haiku-4-5-20251001-v1:0`.
- Exact command / steps: POST a large multi-turn Converse body to
`/model/eu.anthropic.claude-haiku-4-5-20251001-v1:0/converse`;
separately build the
`runtime` + `runtime-slim` targets and run
`/usr/local/bin/headroom-proxy --help`.
- Observed result: before — `bedrock_compression_skipped` (geo-prefixed
id not recognized),
forwarded uncompressed to the wrong `/invoke` upstream; after —
geo-prefixed id recognized,
`/converse` forwarded to the `/converse` upstream, live-zone dispatcher
compresses the
Converse user-message text, measurable token savings. Images contain a
runnable
  `headroom-proxy` in both variants.
- Not tested: non-Anthropic vendor compression parity (#953 follow-up);
Converse
`toolResult` nested-text compression (follow-up — only top-level
Converse text blocks
  compress today).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

- An earlier revision flipped the EventStream `Accept` default
(`*/*`/absent → passthrough);
**dropped** — `*/*` is what most clients (incl. reqwest and the proxy's
own metrics tests)
send while expecting SSE, so forcing passthrough breaks the standard SSE
path.
- The binary build adds the native-proxy compile to the image build;
happy to gate it behind
  a build arg if maintainers prefer it opt-in.
- Addressed a Copilot review round: corrected the `/converse` upstream
routing, the stale
`run_anthropic_compression` comment, the Dockerfile cargo cache mount +
`--locked`, and the
  nonroot AWS-credentials docs example.
2026-06-16 09:45:24 -05:00
..
app docs: improve discoverability for AI agents and search crawlers 2026-05-13 17:36:06 -07:00
components tokens saved grid 2026-04-12 13:42:20 +06:00
content/docs fix(proxy): read RTK gain stats globally by default (#957) 2026-06-13 21:21:38 -07:00
overrides fix: repair release and docs pipelines 2026-04-16 12:53:51 -05:00
screenshots Merge pull request #147 from JerrettDavis/feat/anthropic-usage-insights 2026-04-12 10:54:22 -07:00
spec docs: align MCP proxy docs with implemented API (#614) 2026-06-10 21:14:23 -05:00
.gitignore new docs UI + ts doc coverage 2026-04-12 13:15:58 +06:00
auth-modes.md fix: PR-F1 classify_auth_mode helper (Phase F kickoff) 2026-05-03 17:20:14 -07:00
bedrock.md feat(bedrock): cross-region + Converse compression; bundle proxy binary in images (#999) 2026-06-16 09:45:24 -05:00
bun.lock fix(docs): update bun.lock to next 16.2.6 for GHSA-h64f-5h5j-jqjh (CVE-2026-44577) 2026-06-02 04:25:21 +00:00
next.config.mjs new docs UI + ts doc coverage 2026-04-12 13:15:58 +06:00
observability.md fix(observability): G3 remediation — bound cardinality + wire dead metrics 2026-05-24 10:41:56 -07:00
package-lock.json ci: bump esbuild from 0.27.7 to 0.28.1 in /docs in the npm_and_yarn group across 1 directory (#936) 2026-06-12 23:16:29 -05:00
package.json ci: bump esbuild from 0.27.7 to 0.28.1 in /docs in the npm_and_yarn group across 1 directory (#936) 2026-06-12 23:16:29 -05:00
postcss.config.mjs new docs UI + ts doc coverage 2026-04-12 13:15:58 +06:00
proxy.ts new docs UI + ts doc coverage 2026-04-12 13:15:58 +06:00
README.md new docs UI + ts doc coverage 2026-04-12 13:15:58 +06:00
rtk-architecture.md fix(observability): wire Phase G PR-G3 RTK + proxy metrics (H-blocker) 2026-05-22 13:18:42 -07:00
source.config.ts new docs UI + ts doc coverage 2026-04-12 13:15:58 +06:00
tsconfig.json new docs UI + ts doc coverage 2026-04-12 13:15:58 +06:00

README.md

docs

This is a Next.js application generated with Create Fumadocs.

Run development server:

npm run dev
# or
pnpm dev
# or
yarn dev

Open http://localhost:3000 with your browser to see the result.

Explore

In the project, you can see:

  • lib/source.ts: Code for content source adapter, loader() provides the interface to access your content.
  • lib/layout.shared.tsx: Shared options for layouts, optional but preferred to keep.
Route Description
app/(home) The route group for your landing page and other pages.
app/docs The documentation layout and pages.
app/api/search/route.ts The Route Handler for search.

Fumadocs MDX

A source.config.ts config file has been included, you can customise different options like frontmatter schema.

Read the Introduction for further details.

Learn More

To learn more about Next.js and Fumadocs, take a look at the following resources: