mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
90ef66213d
Phase D close. Adds the operator-facing observability surface that
PRs D1 (native invoke) and D2 (streaming EventStream) deferred, and
wires the Phase F PR-F1 auth-mode classifier into the Bedrock route
so downstream cache/compression policy gates have something to read.
Changes
-------
* New `bedrock::auth_mode_layer` middleware. Classifies every
inbound Bedrock request via F1's `classify`, coerces the result
to `AuthMode::OAuth` per the Bedrock policy matrix (SigV4 IAM is
OAuth-equivalent), and stores the resolved value in
`request.extensions()` so PR-F2/F3 can read it without
re-classifying. Mismatches are logged at WARN with
`event=bedrock_auth_mode_unexpected` — no silent coercion.
* New `observability` module with three Prometheus families:
- `bedrock_invoke_count_total{model, region, auth_mode}` (counter)
- `bedrock_invoke_latency_seconds{model, region}` (histogram)
- `bedrock_eventstream_message_count_total{model, region, event_type}`
(counter)
Registered lazily via `OnceLock` so per-request work is just
`inc_with_label_values` / `observe`. Latency observed via an
RAII `LatencyGuard` so every error path is instrumented; a
future regression that adds a new return path can't drop the
observation.
* New `GET /metrics` endpoint serves the registry in Prometheus
text format. Mounted unconditionally — no feature flag gate — so
scrape works regardless of which provider routes are mounted.
* Bedrock invoke + invoke-streaming handlers now extract
`Extension<AuthMode>`, log it in their entry breadcrumbs
(`event=bedrock_invoke_received`, `event=bedrock_invoke_streaming_received`),
and pass `model`/`region` into `translate_stream` so per-message
metrics carry the right labels.
* Operator docs at `docs/bedrock.md`: AWS credential chain,
region/endpoint config, supported model IDs (`anthropic.*`
literal-match — no regexes), compression behaviour, sample
PromQL queries, structured-log correlation, rollback path.
Tests added (6, all green)
--------------------------
Auth-mode (`integration_bedrock_authmode.rs`):
1. `bedrock_classified_as_oauth` — empty headers → OAuth in
extensions.
2. `oauth_policy_passthrough_prefer` — body byte-equal upstream;
no auto cache_control / prompt_cache_key injected.
Metrics (`integration_bedrock_metrics.rs`):
3. `metrics_increment_per_invoke` — 3 invokes → counter=3 with
correct labels.
4. `metrics_observe_latency` — 1 invoke → histogram count=1,
sum>0.
5. `eventstream_metrics_per_message_type` — 5 chunks → counter=5
with `event_type=chunk`.
6. `metrics_endpoint_serves_scrape` — `/metrics` returns 200,
`text/plain`, all three metric families' HELP/TYPE lines
present.
Each metrics test owns a unique (model, region) tuple so the
global `prometheus` registry — shared across parallel tests in
the same binary — gives each test isolated label rows. Without
isolation, parallel tests cross-contaminate counters.
Constraints honoured
--------------------
* No silent fallbacks — auth-mode coercion is logged at WARN.
* No hardcodes — region from `--bedrock-region`, model from axum
path parameter.
* No regexes — vendor prefix is literal `anthropic.`.
* Comprehensive structured logs — every metric increment paired
with `tracing::debug!` carrying the same labels for incident
correlation.
* Performant — `OnceLock`-cached descriptors, RAII guard, total
D3 overhead well under 1us per request.
* Cardinality bounded — labels driven by config + bounded enums,
never by user-controlled bytes.
Live cloud validation deferred
------------------------------
The wiremock-backed integration tests are the canonical correctness
gate for D3. A real Bedrock smoke test requires `bedrock:InvokeModel`
permissions in the developer's AWS account and is documented in
`docs/bedrock.md` — both D1 and D2 hit sandbox permission issues
trying this path; D3 follows the same convention.
Stacked on
----------
PR #364 (D1 native invoke), PR #365 (D2 streaming EventStream),
PR #366 (F1 classifier helper). Merge those first; this PR will be
rebased onto main once they land.
|
||
|---|---|---|
| .. | ||
| app | ||
| components | ||
| content/docs | ||
| overrides | ||
| screenshots | ||
| spec | ||
| .gitignore | ||
| auth-modes.md | ||
| bedrock.md | ||
| bun.lock | ||
| next.config.mjs | ||
| package-lock.json | ||
| package.json | ||
| postcss.config.mjs | ||
| proxy.ts | ||
| README.md | ||
| source.config.ts | ||
| tsconfig.json | ||
docs
This is a Next.js application generated with Create Fumadocs.
Run development server:
npm run dev
# or
pnpm dev
# or
yarn dev
Open http://localhost:3000 with your browser to see the result.
Explore
In the project, you can see:
lib/source.ts: Code for content source adapter,loader()provides the interface to access your content.lib/layout.shared.tsx: Shared options for layouts, optional but preferred to keep.
| Route | Description |
|---|---|
app/(home) |
The route group for your landing page and other pages. |
app/docs |
The documentation layout and pages. |
app/api/search/route.ts |
The Route Handler for search. |
Fumadocs MDX
A source.config.ts config file has been included, you can customise different options like frontmatter schema.
Read the Introduction for further details.
Learn More
To learn more about Next.js and Fumadocs, take a look at the following resources:
- Next.js Documentation - learn about Next.js features and API.
- Learn Next.js - an interactive Next.js tutorial.
- Fumadocs - learn about Fumadocs