feat(rust): scaffold workspace + parity harness (phase-0)
Bootstrap the Rust port of Headroom. Additive only — no existing Python
code modified. Ships the workshop, not the widgets.
Layout
Cargo.toml (workspace) + rust-toolchain.toml
crates/headroom-core — transform library, stub only
crates/headroom-proxy — axum binary, /healthz only
crates/headroom-py — PyO3 cdylib, exposes headroom._core.hello()
crates/headroom-parity — Rust-vs-Python oracle harness + parity-run CLI
Tooling
Makefile: test, test-parity, bench, build-proxy, build-wheel, fmt, lint
.github/workflows/rust.yml: test, wheels (linux/mac), audit, parity-nightly
deny.toml for cargo-deny
Parity corpus
tests/parity/recorder.py + scripts/record_fixtures.py
125 recorded fixtures across 5 leaf transforms (ccr, tokenizer,
log_compressor, diff_compressor, cache_aligner)
Docs
RUST_DEV.md — developer setup and workspace reference
docs/spec/022-rust-migration.md — migration plan and stage breakdown
.gitignore: whitelist scripts/record_fixtures.py; ignore target/
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 13:39:48 -07:00
|
|
|
|
[workspace]
|
|
|
|
|
|
resolver = "2"
|
|
|
|
|
|
members = [
|
|
|
|
|
|
"crates/headroom-core",
|
|
|
|
|
|
"crates/headroom-proxy",
|
|
|
|
|
|
"crates/headroom-py",
|
|
|
|
|
|
"crates/headroom-parity",
|
|
|
|
|
|
]
|
|
|
|
|
|
# headroom-py is a Python extension module — it must be built via maturin, not
|
|
|
|
|
|
# plain cargo (the "extension-module" feature tells pyo3 not to link libpython,
|
|
|
|
|
|
# which is required for `import` to work). `cargo build --workspace` without
|
|
|
|
|
|
# explicit members skips it; `cargo test --workspace` still runs its tests
|
|
|
|
|
|
# because pyo3 can dynamically link here for the cdylib used by tests.
|
|
|
|
|
|
default-members = [
|
|
|
|
|
|
"crates/headroom-core",
|
|
|
|
|
|
"crates/headroom-proxy",
|
|
|
|
|
|
"crates/headroom-parity",
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
|
|
[workspace.package]
|
|
|
|
|
|
edition = "2021"
|
feat(rust): tokenizer crate with tiktoken-rs + HuggingFace + estimator
Stage 2 of the Rust port: a `headroom_core::tokenizer` module mirroring the
Python `headroom.tokenizers` surface, with three backends behind a single
`Tokenizer` trait.
Backends, in dispatch order:
1. HuggingFace (`HfTokenizer`) — pure-Rust `tokenizers` crate loading any
public `tokenizer.json`. Covers the gap between OpenAI (tiktoken) and the
Anthropic/Gemini estimator: Cohere `command-*`, Llama-3.x, Mistral, Qwen,
BERT, T5, etc. Construct from bytes or a file path; register against a
model-name prefix via `register_hf` for automatic dispatch. No `hf-hub`
auto-download yet — keeps networking, auth, and `~/.cache/huggingface` out
of core. Longest-prefix wins; lookups are RwLock-protected.
2. Tiktoken (`TiktokenCounter`) — `tiktoken-rs` 0.11 BPE for OpenAI / o-series
families. Byte-identical to Python `tiktoken` for ordinary text. Lazy
shared `Arc<CoreBPE>` per encoding (o200k_base, cl100k_base, p50k_base,
r50k_base).
3. Estimation (`EstimatingCounter`) — `chars / cpt` last-resort fallback.
Matches Python's `max(1, int(len(text) / cpt + 0.5))` round-half-up
formula (a self-review caught and fixed an earlier `ceil`-based version
that diverged in the middle of the range, e.g. 5 chars at 4.0 cpt).
Tests: 43 unit tests + 5 proptests; parity 40/40 byte-equal.
Bench: criterion baseline on small/medium/large inputs.
Workspace MSRV bumped 1.78 → 1.80 for `LazyLock`/`OnceLock`.
No proxy wiring. Library-only; production behavior unchanged.
2026-04-25 14:22:09 -07:00
|
|
|
|
rust-version = "1.80"
|
feat(rust): scaffold workspace + parity harness (phase-0)
Bootstrap the Rust port of Headroom. Additive only — no existing Python
code modified. Ships the workshop, not the widgets.
Layout
Cargo.toml (workspace) + rust-toolchain.toml
crates/headroom-core — transform library, stub only
crates/headroom-proxy — axum binary, /healthz only
crates/headroom-py — PyO3 cdylib, exposes headroom._core.hello()
crates/headroom-parity — Rust-vs-Python oracle harness + parity-run CLI
Tooling
Makefile: test, test-parity, bench, build-proxy, build-wheel, fmt, lint
.github/workflows/rust.yml: test, wheels (linux/mac), audit, parity-nightly
deny.toml for cargo-deny
Parity corpus
tests/parity/recorder.py + scripts/record_fixtures.py
125 recorded fixtures across 5 leaf transforms (ccr, tokenizer,
log_compressor, diff_compressor, cache_aligner)
Docs
RUST_DEV.md — developer setup and workspace reference
docs/spec/022-rust-migration.md — migration plan and stage breakdown
.gitignore: whitelist scripts/record_fixtures.py; ignore target/
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 13:39:48 -07:00
|
|
|
|
license = "Apache-2.0"
|
|
|
|
|
|
repository = "https://github.com/chopratejas/headroom"
|
|
|
|
|
|
authors = ["Headroom Maintainers"]
|
|
|
|
|
|
|
|
|
|
|
|
[workspace.dependencies]
|
|
|
|
|
|
serde = { version = "1", features = ["derive"] }
|
fix(rust): smart_crusher scaffold review findings — hash truncation, int parse, python-repr matcher
Code review (`/code-review` on commit `d219bee`) caught one critical
bug, two important parity gaps, and a few quality nits. Fixed all of
them; all 135 unit tests pass; diff_compressor parity harness
unaffected (27/27 still matched).
# Critical fix — `hash_field_name` truncation length
Rust truncated SHA-256 to **16** hex chars; Python uses **8** (per
`smart_crusher.py:177`: `hashlib.sha256(...).hexdigest()[:8]`). 16-char
hashes would never collide with TOIN's 8-char `preserve_fields`,
silently disabling the entire `use_feedback_hints` cache lookup path.
Fix: `hex[..8]` instead of `hex[..16]`. Three pinning tests re-verified
against actual Python reference output. Doc comment now warns
explicitly that the length must match Python or TOIN lookups silently miss.
# Important fix — `python_int_parse` mirrors Python's `int()` semantics
`statistics.rs::detect_sequential_pattern` previously called
`s.parse::<i64>()`. Python's `int()` differs in three ways that affect
realistic payloads:
- strips ASCII whitespace (Rust's `parse` rejects)
- accepts leading `+` (Rust accepts; same)
- accepts PEP 515 underscores like `"3_000"` (Rust rejects)
A field with `[" 1 ", " 2 ", " 3 ", "4", "5"]` would parse all five
in Python (sequential = True) but only one in Rust (`nums.len() < 5`
→ False). Silent parity break.
Fix: new private `python_int_parse` helper that strips whitespace,
handles underscore separators, and rejects edge cases Python rejects.
Six new tests pin the behavior.
# Important fix — `python_repr` for `item_matches_anchors`
Python compares anchors via `anchor in str(item).lower()`. We were
using `serde_json::to_string(&item).to_lowercase()`, which differs in
three ways that affect substring matching:
- quote chars (`'` vs `"`)
- bool/null literals (`True`/`False`/`None` vs `true`/`false`/`null`)
- spacing (`key: value, ...` vs `key:value,...`)
Anchor `"none"` would match Python form but not JSON. Inverse for
`"null"`. Real divergence.
Fix: new private `python_repr` walks `serde_json::Value` and emits
Python-equivalent form. Plus enable `serde_json/preserve_order` at
workspace level so `Value::Object` preserves JSON parse order
(matching Python `dict` since 3.7).
# Suggestion fixes
- Classifier comment for `[True, False, 1] -> MIXED_ARRAY` now walks
both Python and Rust paths step by step.
- `ArrayAnalysis::field_stats` doc notes the BTreeMap vs Python-dict
order nuance for the analyzer port to resolve.
- Added regression tests for "all unparseable strings", "single int
among strings", fractional-step sequential, and the email-typo
pattern.
# Build / test
- `cargo build -p headroom-core` clean.
- `cargo clippy -p headroom-core -- -D warnings` clean.
- 135 unit tests in `headroom-core`, all passing (was 55).
- `cargo run -p headroom-parity run` — diff_compressor 27/27 still matched.
2026-04-26 17:01:46 -07:00
|
|
|
|
# `preserve_order` makes `serde_json::Value::Object` use IndexMap so JSON
|
|
|
|
|
|
# parse order is preserved through Value→string→Value round-trips. The
|
|
|
|
|
|
# smart_crusher port relies on this to match Python's `str(dict)` output,
|
|
|
|
|
|
# which preserves insertion order; otherwise BTreeMap's sorted-key default
|
|
|
|
|
|
# would diverge from Python on every multi-key object.
|
fix(rust): A4 — honor cache_control markers; serde_json arbitrary_precision + raw_value
PR-A4 of the Realignment Phase A lockdown
(REALIGNMENT/03-phase-A-lockdown.md). Eliminates P0-3 (Rust proxy
ignores customer cache_control markers) and P0-5 (numeric precision
lost via serde_json::Value round-trip) at the library level; Phase B
PR-B2 wires the helper into the live-zone block dispatcher.
Cargo.toml — add `arbitrary_precision` and `raw_value` to
`serde_json` workspace features. `arbitrary_precision` keeps `1.0`
from collapsing to `1` and preserves >2^53 integers; `raw_value`
exposes `&RawValue` so PR-B2 can forward unmodified `messages[*]`
entries as exact byte copies.
crates/headroom-core/src/cache_control.rs (new) — `compute_frozen_count`
walks `messages[i].content[*].cache_control` via serde_json
accessors only (no regex) and returns the smallest N such that
`messages[i]` is frozen for every i < N. Markers in `system` or
`tools[*]` log at debug! but never bump the floor (those fields are
unconditionally cache-hot per invariant I2). TTL ordering violations
(5m before 1h, guide §2.19) emit `tracing::warn!` but the function
computes the correct count regardless — the customer's request, not
ours to reject.
crates/headroom-core/src/lib.rs — re-export `compute_frozen_count` at
crate root so the proxy crate has a stable import path.
crates/headroom-proxy/src/compression/anthropic.rs — add
`resolve_frozen_count` thin wrapper that consults the
`cache_control_auto_frozen` config flag. When `disabled`, returns 0
regardless of body content (operator opt-out for benchmarking).
crates/headroom-proxy/src/config.rs — add `CacheControlAutoFrozen`
enum and the matching CLI flag `--cache-control-auto-frozen` /
env var `HEADROOM_PROXY_CACHE_CONTROL_AUTO_FROZEN`. Default is
`enabled`. Documented in the doc comments.
Tests
- crates/headroom-core/src/cache_control.rs (inline): 11 unit tests
covering marker detection, system/tools negative cases, ordering
state machine, defensive (missing fields, non-array messages,
non-object content blocks).
- crates/headroom-core/tests/cache_control.rs: 11 unit + 3 property
tests (monotonic non-decrease as markers are added; system/tools
markers don't change count; empty messages → 0).
- crates/headroom-proxy/tests/integration_cache_control.rs: 8 tests
exercising the proxy wrapper (configurability gate; tracing
capture for the 5m-before-1h warn path).
Acceptance gates: `cargo build --workspace`, `cargo test --workspace`
(33 new tests green), `cargo clippy --workspace -- -D warnings`,
`cargo fmt --all --check` all clean. No new `regex::` imports;
`git grep -n 'regex::' crates/{headroom-core/src/cache_control.rs,
headroom-core/tests/cache_control.rs, headroom-proxy/tests/
integration_cache_control.rs}` empty.
Honors the realignment build constraints: configurable (CLI + env),
no hardcodes (TTL strings live as const), no regex (serde_json
accessor walk), no fallbacks (one impl), structured logging
(debug!/warn! with field/index/ttl/rule context), tests
comprehensive (unit + property + integration + tracing capture).
2026-05-02 08:22:10 -07:00
|
|
|
|
#
|
|
|
|
|
|
# `arbitrary_precision` keeps the literal numeric token from the source
|
|
|
|
|
|
# JSON intact: `Value::Number` becomes a wrapper around the original
|
|
|
|
|
|
# digit string, so `1.0` does NOT collapse to `1`, and `12345678901234567`
|
|
|
|
|
|
# does NOT lose precision through f64. Required by Realignment invariant
|
|
|
|
|
|
# I1 (byte-faithful passthrough on unmutated bytes; see REALIGNMENT/02-
|
|
|
|
|
|
# architecture.md §2.2) and PR-A4 (see REALIGNMENT/03-phase-A-lockdown.md).
|
|
|
|
|
|
#
|
|
|
|
|
|
# `raw_value` exposes `serde_json::value::RawValue`, the unparsed JSON
|
|
|
|
|
|
# fragment type. Phase B PR-B2 uses this to forward unmodified
|
|
|
|
|
|
# `messages[*]` entries as exact byte copies — the parser captures the
|
|
|
|
|
|
# original byte slice, so byte-for-byte round-trips work even with
|
|
|
|
|
|
# whitespace, key order, or escape preferences the producer chose.
|
|
|
|
|
|
# Enabled here in Phase A so PR-B2 can land as a pure consumer change.
|
|
|
|
|
|
serde_json = { version = "1", features = ["preserve_order", "arbitrary_precision", "raw_value"] }
|
feat(rust): scaffold workspace + parity harness (phase-0)
Bootstrap the Rust port of Headroom. Additive only — no existing Python
code modified. Ships the workshop, not the widgets.
Layout
Cargo.toml (workspace) + rust-toolchain.toml
crates/headroom-core — transform library, stub only
crates/headroom-proxy — axum binary, /healthz only
crates/headroom-py — PyO3 cdylib, exposes headroom._core.hello()
crates/headroom-parity — Rust-vs-Python oracle harness + parity-run CLI
Tooling
Makefile: test, test-parity, bench, build-proxy, build-wheel, fmt, lint
.github/workflows/rust.yml: test, wheels (linux/mac), audit, parity-nightly
deny.toml for cargo-deny
Parity corpus
tests/parity/recorder.py + scripts/record_fixtures.py
125 recorded fixtures across 5 leaf transforms (ccr, tokenizer,
log_compressor, diff_compressor, cache_aligner)
Docs
RUST_DEV.md — developer setup and workspace reference
docs/spec/022-rust-migration.md — migration plan and stage breakdown
.gitignore: whitelist scripts/record_fixtures.py; ignore target/
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 13:39:48 -07:00
|
|
|
|
bytes = "1"
|
|
|
|
|
|
thiserror = "1"
|
Pin ORT dylib on Windows; init Python logging (#1010)
## Description
On Windows, headroom's Rust core resolves `onnxruntime.dll` at runtime
via `ort-load-dynamic`. Without an explicit `ORT_DYLIB_PATH`, the bare
DLL search can land on `C:\Windows\System32\onnxruntime.dll`, the
Windows ML OS component, and `Session::new()` can deadlock instead of
returning an error. Since a hang is not an `Err`, the tiered fallback
cannot engage until the proxy-level timeout fires.
This PR pins `ORT_DYLIB_PATH` to the pip-installed `onnxruntime` DLL at
import time, and wires Rust `tracing` events into Python logging so the
proxy log surfaces these failures when they occur.
Closes #928
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Added `headroom/_ort.py` with a Windows-only, idempotent
`ensure_ort_dylib_pinned()` resolver that respects an existing
`ORT_DYLIB_PATH`.
- Call the pin from `headroom/__init__.py` before importing `_core`
consumers.
- Log the effective ORT dylib path from the content router startup path
on Windows.
- Enable Rust tracing-to-log compatibility and initialize `pyo3-log` in
the `_core` module.
- Add timeout diagnostics in the Magika detector with the effective
`ORT_DYLIB_PATH`.
- Document `ORT_DYLIB_PATH` and `HEADROOM_MAGIKA_INIT_TIMEOUT_SECS`.
- Add unit coverage for the resolver behavior.
## Testing
- [x] Unit tests pass (`python -m pytest
tests/test_transforms/test_ort_dylib.py -q`)
- [x] Linting passes (`ruff check headroom/_ort.py headroom/__init__.py
headroom/transforms/content_router.py
tests/test_transforms/test_ort_dylib.py`)
- [x] Formatting passes (`ruff format --check headroom/_ort.py
headroom/__init__.py headroom/transforms/content_router.py
tests/test_transforms/test_ort_dylib.py`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_transforms/test_ort_dylib.py -q
7 passed in 0.19s
$ ruff check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py
All checks passed!
$ ruff format --check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py
4 files already formatted
$ cargo check -p headroom-py
cargo: The term 'cargo' is not recognized as a name of a cmdlet, function, script file, or executable program.
```
## Real Behavior Proof
- Environment: Windows 11 24H2, Python 3.13, RTX 4080
- Exact command / steps: `python -c "import headroom; from
headroom._core import detect_content_type as d;
print(d(open('headroom/compress.py').read()).content_type)"`
- Observed result: `source_code` in 301ms, clean exit, `Magika: ENABLED`
in proxy log
- Not tested: macOS/Linux manual runtime behavior; `_ort.py` is a no-op
outside Windows, and CI covers cross-platform build/test behavior.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable (N/A: repo uses
release-please)
## Additional Notes
The branch was rebased onto current `main` and the commit subject was
updated to satisfy commitlint. Local Rust verification could not be run
on this Windows machine because `cargo` is not installed; GitHub CI
should be treated as the Rust build verification for the `pyo3-log`
dependency and workspace lockfile changes.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 14:46:24 +02:00
|
|
|
|
# `log` compat: when no tracing subscriber is active (the case inside the
|
|
|
|
|
|
# headroom-py cdylib), events are re-emitted as `log` records so pyo3-log
|
|
|
|
|
|
# can forward them to Python's logging. No effect on binaries that install
|
|
|
|
|
|
# a real subscriber.
|
|
|
|
|
|
tracing = { version = "0.1", features = ["log"] }
|
feat(rust): scaffold workspace + parity harness (phase-0)
Bootstrap the Rust port of Headroom. Additive only — no existing Python
code modified. Ships the workshop, not the widgets.
Layout
Cargo.toml (workspace) + rust-toolchain.toml
crates/headroom-core — transform library, stub only
crates/headroom-proxy — axum binary, /healthz only
crates/headroom-py — PyO3 cdylib, exposes headroom._core.hello()
crates/headroom-parity — Rust-vs-Python oracle harness + parity-run CLI
Tooling
Makefile: test, test-parity, bench, build-proxy, build-wheel, fmt, lint
.github/workflows/rust.yml: test, wheels (linux/mac), audit, parity-nightly
deny.toml for cargo-deny
Parity corpus
tests/parity/recorder.py + scripts/record_fixtures.py
125 recorded fixtures across 5 leaf transforms (ccr, tokenizer,
log_compressor, diff_compressor, cache_aligner)
Docs
RUST_DEV.md — developer setup and workspace reference
docs/spec/022-rust-migration.md — migration plan and stage breakdown
.gitignore: whitelist scripts/record_fixtures.py; ignore target/
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 13:39:48 -07:00
|
|
|
|
anyhow = "1"
|
|
|
|
|
|
clap = { version = "4", features = ["derive"] }
|
|
|
|
|
|
tokio = { version = "1", features = ["macros", "rt-multi-thread", "signal"] }
|
|
|
|
|
|
axum = "0.7"
|
|
|
|
|
|
tower = "0.5"
|
|
|
|
|
|
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
|
ci: bump pyo3 from 0.22.6 to 0.24.1 in the cargo group across 1 directory (#270)
Bumps the cargo group with 1 update in the / directory:
[pyo3](https://github.com/pyo3/pyo3).
Updates `pyo3` from 0.22.6 to 0.24.1
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pyo3/pyo3/releases">pyo3's
releases</a>.</em></p>
<blockquote>
<h2>PyO3 0.24.1</h2>
<p>This release is a security fix for the
<code>PyString::from_object</code> method, which passed
<code>&str</code> data to the Python C API without checking for a
terminating nul byte. All historical PyO3 versions are affected, and we
recommend you upgrade if you are using
<code>PyString::from_object</code>. Thank you to <a
href="https://github.com/vthib"><code>@vthib</code></a> for the report
and <a href="https://github.com/Dr-Emann"><code>@Dr-Emann</code></a>
for the fix. A RUSTSEC advisory will be published shortly.</p>
<p>Aside from the security fix, this release contains a number of other
non-breaking additions:</p>
<ul>
<li>An <code>abi3-py313</code> feature to support compiling with the
Python 3.13 stable ABI.</li>
<li><code>PyAnyMethods::getattr_opt</code> to get optional attributes
without paying the cost of a Python exception when the attribute in
question does not exist.</li>
<li>Constructor for <code>PyInt::new</code>.</li>
<li><code>with_critical_section2</code> for locking two objects at the
same time on the free-threaded build.</li>
<li>Fix for a PyO3 0.24.0 regression with
<code>Option<&str></code> and
<code>Option<&T></code> (where <code>T: PyClass</code>)
function arguments no longer being permitted</li>
</ul>
<p>There are also a few other small bug fixes for edge cases, mostly
related to compile errors from PyO3's macro code.</p>
<p>Thank you to the following contributors for the improvements:</p>
<p><a
href="https://github.com/bschoenmaeckers"><code>@bschoenmaeckers</code></a>
<a href="https://github.com/davidhewitt"><code>@davidhewitt</code></a>
<a href="https://github.com/Dr-Emann"><code>@Dr-Emann</code></a>
<a href="https://github.com/emmagordon"><code>@emmagordon</code></a>
<a href="https://github.com/epontan"><code>@epontan</code></a>
<a href="https://github.com/Icxolu"><code>@Icxolu</code></a>
<a
href="https://github.com/IvanIsCoding"><code>@IvanIsCoding</code></a>
<a href="https://github.com/jelmer"><code>@jelmer</code></a>
<a href="https://github.com/jonaspleyer"><code>@jonaspleyer</code></a>
<a href="https://github.com/ngoldbaum"><code>@ngoldbaum</code></a>
<a
href="https://github.com/Owen-CH-Leung"><code>@Owen-CH-Leung</code></a>
<a href="https://github.com/Tpt"><code>@Tpt</code></a>
<a
href="https://github.com/Trolldemorted"><code>@Trolldemorted</code></a>
<a href="https://github.com/XuehaiPan"><code>@XuehaiPan</code></a></p>
<h2>PyO3 0.24.0</h2>
<p>This release is an incremental improvement of refinements and
optimizations following the new APIs established in PyO3's last few
releases.</p>
<p>Support for <code>jiff</code> datetime conversions have been added,
and also UUID conversions.</p>
<p>The <code>FromPyObject</code> derive macro has gained new
<code>#[pyo3(default = ...)]</code> and <code>#[pyo3(rename_all =
...)]</code> options, and the <code>IntoPyObject</code> derive macro has
gained a new <code>#[pyo3(into_py_with = ...)]</code> option.</p>
<p>PyO3 will now pass positional arguments to Python functions using the
"vectorcall" protocol in many cases, which should be an
optimization over the previous behaviour (of creating a Python tuple of
positional arguments).</p>
<p>Many methods on iterators of Python collections have been
optimized.</p>
<p>There are also many other incremental improvements, bug fixes and
smaller features.</p>
<p>Thank you to everyone who contributed code, documentation, design
ideas, bug reports, and feedback. The following contributors' commits
are included in this release:</p>
<p><a href="https://github.com/0x676e67"><code>@0x676e67</code></a>
<a href="https://github.com/alex"><code>@alex</code></a>
<a href="https://github.com/arielb1"><code>@arielb1</code></a>
<a
href="https://github.com/bschoenmaeckers"><code>@bschoenmaeckers</code></a>
<a
href="https://github.com/davidhewitt"><code>@davidhewitt</code></a></p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md">pyo3's
changelog</a>.</em></p>
<blockquote>
<h2>[0.24.1] - 2025-03-31</h2>
<h3>Added</h3>
<ul>
<li>Add <code>abi3-py313</code> feature. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4969">#4969</a></li>
<li>Add <code>PyAnyMethods::getattr_opt</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4978">#4978</a></li>
<li>Add <code>PyInt::new</code> constructor for all supported number
types (i32, u32, i64, u64, isize, usize). <a
href="https://redirect.github.com/PyO3/pyo3/pull/4984">#4984</a></li>
<li>Add <code>pyo3::sync::with_critical_section2</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4992">#4992</a></li>
<li>Implement <code>PyCallArgs</code> for <code>Borrowed<'_, 'py,
PyTuple></code>, <code>&Bound<'py, PyTuple></code>, and
<code>&Py<PyTuple></code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/5013">#5013</a></li>
</ul>
<h3>Fixed</h3>
<ul>
<li>Fix <code>is_type_of</code> for native types not using same
specialized check as <code>is_type_of_bound</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4981">#4981</a></li>
<li>Fix <code>Probe</code> class naming issue with
<code>#[pymethods]</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4988">#4988</a></li>
<li>Fix compile failure with required <code>#[pyfunction]</code>
arguments taking <code>Option<&str></code> and
<code>Option<&T></code> (for <code>#[pyclass]</code> types).
<a href="https://redirect.github.com/PyO3/pyo3/pull/5002">#5002</a></li>
<li>Fix <code>PyString::from_object</code> causing of bounds reads with
<code>encoding</code> and <code>errors</code> parameters which are not
nul-terminated. <a
href="https://redirect.github.com/PyO3/pyo3/pull/5008">#5008</a></li>
<li>Fix compile error when additional options follow after
<code>crate</code> for <code>#[pyfunction]</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/5015">#5015</a></li>
</ul>
<h2>[0.24.0] - 2025-03-09</h2>
<h3>Packaging</h3>
<ul>
<li>Add supported CPython/PyPy versions to cargo package metadata. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4756">#4756</a></li>
<li>Bump <code>target-lexicon</code> dependency to 0.13. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4822">#4822</a></li>
<li>Add optional <code>jiff</code> dependency to add conversions for
<code>jiff</code> datetime types. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4823">#4823</a></li>
<li>Add optional <code>uuid</code> dependency to add conversions for
<code>uuid::Uuid</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4864">#4864</a></li>
<li>Bump minimum supported <code>inventory</code> version to 0.3.5. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4954">#4954</a></li>
</ul>
<h3>Added</h3>
<ul>
<li>Add <code>PyIterator::send</code> method to allow sending values
into a python generator. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4746">#4746</a></li>
<li>Add <code>PyCallArgs</code> trait for passing arguments into the
Python calling protocol. This enabled using a faster calling convention
for certain types, improving performance. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4768">#4768</a></li>
<li>Add <code>#[pyo3(default = ...']</code> option for
<code>#[derive(FromPyObject)]</code> to set a default value for
extracted fields of named structs. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4829">#4829</a></li>
<li>Add <code>#[pyo3(into_py_with = ...)]</code> option for
<code>#[derive(IntoPyObject, IntoPyObjectRef)]</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4850">#4850</a></li>
<li>Add FFI definitions <code>PyThreadState_GetFrame</code> and
<code>PyFrame_GetBack</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4866">#4866</a></li>
<li>Optimize <code>last</code> for <code>BoundListIterator</code>,
<code>BoundTupleIterator</code> and <code>BorrowedTupleIterator</code>.
<a href="https://redirect.github.com/PyO3/pyo3/pull/4878">#4878</a></li>
<li>Optimize <code>Iterator::count()</code> for <code>PyDict</code>,
<code>PyList</code>, <code>PyTuple</code> & <code>PySet</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4878">#4878</a></li>
<li>Optimize <code>nth</code>, <code>nth_back</code>,
<code>advance_by</code> and <code>advance_back_by</code> for
<code>BoundTupleIterator</code> <a
href="https://redirect.github.com/PyO3/pyo3/pull/4897">#4897</a></li>
<li>Add support for <code>types.GenericAlias</code> as
<code>pyo3::types::PyGenericAlias</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4917">#4917</a></li>
<li>Add <code>MutextExt</code> trait to help avoid deadlocks with the
GIL while locking a <code>std::sync::Mutex</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4934">#4934</a></li>
<li>Add <code>#[pyo3(rename_all = "...")]</code> option for
<code>#[derive(FromPyObject)]</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4941">#4941</a></li>
</ul>
<h3>Changed</h3>
<ul>
<li>Optimize <code>nth</code>, <code>nth_back</code>,
<code>advance_by</code> and <code>advance_back_by</code> for
<code>BoundListIterator</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4810">#4810</a></li>
<li>Use <code>DerefToPyAny</code> in blanket implementations of
<code>From<Py<T>></code> and <code>From<Bound<'py,
T>></code> for <code>PyObject</code>. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4593">#4593</a></li>
<li>Map
<code>io::ErrorKind::IsADirectory</code>/<code>NotADirectory</code> to
the corresponding Python exception on Rust 1.83+. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4747">#4747</a></li>
<li><code>PyAnyMethods::call</code> and friends now require
<code>PyCallArgs</code> for their positional arguments. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4768">#4768</a></li>
<li>Expose FFI definitions for <code>PyObject_Vectorcall(Method)</code>
on the stable abi on 3.12+. <a
href="https://redirect.github.com/PyO3/pyo3/pull/4853">#4853</a></li>
<li><code>#[pyo3(from_py_with = ...)]</code> now take a path rather than
a string literal <a
href="https://redirect.github.com/PyO3/pyo3/pull/4860">#4860</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/PyO3/pyo3/commit/a213b368bd5bf859c2acb655bfed029e17c3b447"><code>a213b36</code></a>
release: 0.24.1 (<a
href="https://redirect.github.com/pyo3/pyo3/issues/5021">#5021</a>)</li>
<li><a
href="https://github.com/PyO3/pyo3/commit/d85a02d9b11f7c057e3627a0393d5d9b876dbc0a"><code>d85a02d</code></a>
split <code>PyFunctionArgument</code> to specialize <code>Option</code>
(<a
href="https://redirect.github.com/pyo3/pyo3/issues/5002">#5002</a>)</li>
<li><a
href="https://github.com/PyO3/pyo3/commit/c37a50a7a33e145f6bb87f40cb89cf85f9e5fac7"><code>c37a50a</code></a>
Add example of more complex exceptions (<a
href="https://redirect.github.com/pyo3/pyo3/issues/5014">#5014</a>)</li>
<li><a
href="https://github.com/PyO3/pyo3/commit/dcacb9bbbc8c130238bd88480fc53074e445b4fc"><code>dcacb9b</code></a>
Simplify PyFunctionArgument impl on &Bound<T> (<a
href="https://redirect.github.com/pyo3/pyo3/issues/5018">#5018</a>)</li>
<li><a
href="https://github.com/PyO3/pyo3/commit/03c31c5c7affdd8805957b5944bd8ca05d1bdec8"><code>03c31c5</code></a>
fix <code>#[pyfunction]</code> option parsing (<a
href="https://redirect.github.com/pyo3/pyo3/issues/5015">#5015</a>)</li>
<li><a
href="https://github.com/PyO3/pyo3/commit/0f49eb14b0358a8fe85c5930db84c5c404f97dd7"><code>0f49eb1</code></a>
docs: Remove examples with outdated PyO3 and unmaintained projects (<a
href="https://redirect.github.com/pyo3/pyo3/issues/4952">#4952</a>)</li>
<li><a
href="https://github.com/PyO3/pyo3/commit/1b00b0d27f1b49d4b4237bc616d99016b06c1bd8"><code>1b00b0d</code></a>
implement <code>PyCallArgs</code> for borrowed types (<a
href="https://redirect.github.com/pyo3/pyo3/issues/5013">#5013</a>)</li>
<li><a
href="https://github.com/PyO3/pyo3/commit/5caaa371dce8fe8a93c64d7a465c3c2c80ce6e2f"><code>5caaa37</code></a>
fix: convert to cstrings in PyString::from_object (<a
href="https://redirect.github.com/pyo3/pyo3/issues/5008">#5008</a>)</li>
<li><a
href="https://github.com/PyO3/pyo3/commit/4aca459fd30441fa006c3eb388c812047f5465ce"><code>4aca459</code></a>
docs: guide - add link to tables and traits (<a
href="https://redirect.github.com/pyo3/pyo3/issues/5001">#5001</a>)</li>
<li><a
href="https://github.com/PyO3/pyo3/commit/0452c0ee5299a1af42f9d966ba3d136a79edb15d"><code>0452c0e</code></a>
replace quansight-labs/setup-python with actions/setup-python (<a
href="https://redirect.github.com/pyo3/pyo3/issues/5007">#5007</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/pyo3/pyo3/compare/v0.22.6...v0.24.1">compare
view</a></li>
</ul>
</details>
<br />
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-06-10 23:01:33 -05:00
|
|
|
|
pyo3 = { version = "0.24", features = ["abi3-py310"] }
|
Pin ORT dylib on Windows; init Python logging (#1010)
## Description
On Windows, headroom's Rust core resolves `onnxruntime.dll` at runtime
via `ort-load-dynamic`. Without an explicit `ORT_DYLIB_PATH`, the bare
DLL search can land on `C:\Windows\System32\onnxruntime.dll`, the
Windows ML OS component, and `Session::new()` can deadlock instead of
returning an error. Since a hang is not an `Err`, the tiered fallback
cannot engage until the proxy-level timeout fires.
This PR pins `ORT_DYLIB_PATH` to the pip-installed `onnxruntime` DLL at
import time, and wires Rust `tracing` events into Python logging so the
proxy log surfaces these failures when they occur.
Closes #928
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Added `headroom/_ort.py` with a Windows-only, idempotent
`ensure_ort_dylib_pinned()` resolver that respects an existing
`ORT_DYLIB_PATH`.
- Call the pin from `headroom/__init__.py` before importing `_core`
consumers.
- Log the effective ORT dylib path from the content router startup path
on Windows.
- Enable Rust tracing-to-log compatibility and initialize `pyo3-log` in
the `_core` module.
- Add timeout diagnostics in the Magika detector with the effective
`ORT_DYLIB_PATH`.
- Document `ORT_DYLIB_PATH` and `HEADROOM_MAGIKA_INIT_TIMEOUT_SECS`.
- Add unit coverage for the resolver behavior.
## Testing
- [x] Unit tests pass (`python -m pytest
tests/test_transforms/test_ort_dylib.py -q`)
- [x] Linting passes (`ruff check headroom/_ort.py headroom/__init__.py
headroom/transforms/content_router.py
tests/test_transforms/test_ort_dylib.py`)
- [x] Formatting passes (`ruff format --check headroom/_ort.py
headroom/__init__.py headroom/transforms/content_router.py
tests/test_transforms/test_ort_dylib.py`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_transforms/test_ort_dylib.py -q
7 passed in 0.19s
$ ruff check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py
All checks passed!
$ ruff format --check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py
4 files already formatted
$ cargo check -p headroom-py
cargo: The term 'cargo' is not recognized as a name of a cmdlet, function, script file, or executable program.
```
## Real Behavior Proof
- Environment: Windows 11 24H2, Python 3.13, RTX 4080
- Exact command / steps: `python -c "import headroom; from
headroom._core import detect_content_type as d;
print(d(open('headroom/compress.py').read()).content_type)"`
- Observed result: `source_code` in 301ms, clean exit, `Magika: ENABLED`
in proxy log
- Not tested: macOS/Linux manual runtime behavior; `_ort.py` is a no-op
outside Windows, and CI covers cross-platform build/test behavior.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable (N/A: repo uses
release-please)
## Additional Notes
The branch was rebased onto current `main` and the commit subject was
updated to satisfy commitlint. Local Rust verification could not be run
on this Windows machine because `cargo` is not installed; GitHub CI
should be treated as the Rust build verification for the `pyo3-log`
dependency and workspace lockfile changes.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 14:46:24 +02:00
|
|
|
|
# Forwards Rust `log` records (incl. tracing events via the `log` compat
|
|
|
|
|
|
# feature above) into Python's `logging` inside the _core extension module.
|
|
|
|
|
|
pyo3-log = "0.12"
|
fix(proxy): PR-D1 native Bedrock InvokeModel route + SigV4
Phase D PR-D1 lands the first native Rust path for AWS Bedrock,
replacing the lossy LiteLLM Python converter for Anthropic-on-Bedrock
non-streaming requests. Eliminates part of P4-37 and P4-39.
What landed
-----------
- New crates/headroom-proxy/src/bedrock/ module:
- envelope.rs: parses the {"anthropic_version": "...", ...}
Bedrock body shape; re-emits with anthropic_version preserved
as the first key (relies on serde_json preserve_order).
- sigv4.rs: AWS SigV4 signer wrapping the aws-sigv4 crate.
Forces PayloadChecksumKind::XAmzSha256 so x-amz-content-sha256
is in the canonical request, hashed over the post-compression
body bytes (the bytes that actually hit Bedrock). No silent
fallback: signing failures return 5xx with
event=bedrock_sigv4_failed.
- invoke.rs: POST handler for /model/{model_id}/invoke
(and /converse - same wire shape for anthropic.claude-*).
Detects Anthropic vendor via literal starts_with("anthropic.")
(no regex per project rule), routes Anthropic-shape bodies
through the existing compress_anthropic_request live-zone
dispatcher, then signs and forwards to the configured Bedrock
endpoint.
- Modified:
- proxy.rs: routes /model/:model_id/invoke and
/model/:model_id/converse when enable_bedrock_native is on
(default). Adds bedrock_credentials: Option<Arc<Credentials>>
to AppState.
- config.rs: new flags --bedrock-region (default us-east-1,
env HEADROOM_PROXY_BEDROCK_REGION), --bedrock-endpoint
(operator override for FIPS/VPC/test setups),
--enable-bedrock-native (default true), --aws-profile.
- main.rs: resolves AWS credentials at startup via
aws_config::defaults(BehaviorVersion::latest()). Failure logs
event=bedrock_credentials_unavailable at WARN; the handler
refuses to forward unsigned (event=bedrock_credentials_missing).
- Cargo.toml: workspace deps aws-sigv4, aws-config,
aws-credential-types, aws-smithy-runtime-api.
Tests
-----
8 integration tests under
crates/headroom-proxy/tests/integration_bedrock_invoke.rs:
1. native_envelope_round_trip_byte_equal
2. sigv4_signed_correctly_after_compression - confirms
authorization is SigV4-shape and x-amz-content-sha256
matches sha256(body received by upstream).
3. thinking_block_preserved_through_bedrock
4. redacted_thinking_preserved
5. document_block_preserved
6. tool_result_array_with_image_preserved
7. stop_sequence_null_only_when_present - pins that the proxy
does NOT inject stop_sequence: null (P4-37 hardcode).
8. tool_use_input_byte_equal_preserves_key_order
All eight pass. Full workspace test run is green; clippy + fmt
clean. make ci-precheck (rust + python + commitlint) passes
locally.
Build constraints honoured
--------------------------
- No silent fallbacks: missing creds / signing failures return
5xx with structured event=... log; no path ever forwards
unsigned.
- No hardcodes: region, endpoint, profile, enable-flag all
configurable via CLI + env.
- No regexes: vendor detection is str::starts_with.
- Comprehensive structured logs: event=bedrock_invoke_received,
bedrock_envelope_parsed, bedrock_compression_skipped,
bedrock_credentials_missing, sigv4_signed,
bedrock_invoke_forwarded, etc.
- Performant: body buffered once, passed by &[u8] to signer
(zero-copy), Bytes::clone only for ownership transfer to
reqwest. Sign exactly once per request.
- Elegant: 4 small focused modules mirror handlers/ + sse/.
- Tests use realistic Anthropic block content (real thinking,
redacted_thinking, document, base64 image fixtures).
Streaming (PR-D2) and observability (PR-D3) follow.
2026-05-03 16:22:32 -07:00
|
|
|
|
# Phase D PR-D1: AWS SigV4 signing for native Bedrock InvokeModel route.
|
|
|
|
|
|
# `aws-sigv4` provides the canonical-request + signing-key implementation;
|
|
|
|
|
|
# `aws-config` resolves credentials from the standard provider chain
|
|
|
|
|
|
# (env vars, profiles, IMDS, ECS task role, etc); `aws-credential-types`
|
|
|
|
|
|
# exposes `Credentials` so the signer accepts whatever the chain returned.
|
|
|
|
|
|
aws-sigv4 = { version = "1", default-features = false, features = ["sign-http", "http1"] }
|
feat(bedrock): cross-region + Converse compression; bundle proxy binary in images (#999)
## Description
The native Bedrock path (Phase D) compresses + signs
Anthropic-on-Bedrock requests, but
two real-world cases slipped through, and the native binary that powers
it was never
shipped. This PR closes those gaps as a focused set of give-backs.
Aligns with the Rust migration plan (see below).
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **Cross-region inference-profile detection** via a new
`bedrock::vendor` module
(`canonical_vendor()`), following the design proposed in #953: strip a
known geo prefix
(`eu.`/`us.`/`apac.`/`global.`) then match the canonical vendor.
Geo-prefixed Anthropic
profiles (`eu.anthropic.…`) now get live-zone compression instead of
being silently
skipped; geo-prefixed non-Anthropic vendors stay correctly excluded.
- **Converse-body compression (two parts)**:
1. `run_anthropic_compression` no longer bails to passthrough when the
body lacks an
InvokeModel `anthropic_version` envelope; envelope re-emit stays gated
on successful parse.
2. The **live-zone dispatcher now recognizes Bedrock Converse content
blocks**. Converse
blocks carry no `type` discriminator (the variant is the key: `{"text":
…}` vs
Anthropic's `{"type":"text","text":…}`), so real Converse user-message
text was still
passing through uncompressed. A typeless block whose `text` is a JSON
string now routes
through the same surgical text path. Anthropic blocks always carry
`type`, so the
Anthropic path is byte-for-byte unchanged; non-text Converse blocks
(`{"image":…}`,
`{"toolUse":…}`) stay unrecognized and no-op.
- **Correct `/converse` upstream routing**: the non-streaming handler
resolved the upstream
action from a hard-coded `"invoke"`, so `/converse` requests were
forwarded to Bedrock's
`/invoke` endpoint. It now resolves the action from the inbound path
(`extract_invoke_action`),
mirroring the streaming handler's `extract_streaming_action`. SigV4
signs the same URL it
forwards, so the signature stays consistent.
- **`aws-config` `sso` feature**: SSO profiles now resolve through the
default credential
chain for SigV4 — the credential chain in `docs/bedrock.md` already
promised SSO; this
makes the code match.
- **Ship the `headroom-proxy` binary in published images**
(`Dockerfile`): built in the
builder stage (`--locked`, with the cargo registry cache mounted at
`CARGO_HOME`) and
copied into both the debian and distroless runtime images.
- **Docs** (`docs/bedrock.md`): document cross-region inference profiles
and a "Running the
proxy" section. AWS credentials mount at `/home/nonroot/.aws` (the
default nonroot image
home) where the SDK looks for `~/.aws`, with a note on the root-image
alternative.
## Related issues
- Closes #976 — ship the `headroom-proxy` binary in published images
(this PR implements
the exact fix proposed there).
- Addresses the **cross-region inference-profile** half of #953 via its
proposed
`canonical_vendor()` design. Non-Anthropic vendor compression parity
(Nova/GLM/MiniMax/
Kimi) is the natural follow-up — `bedrock::vendor` is the shared
resolver it can build on.
- Extends the native Bedrock InvokeModel compression requested in #734
(the Bedrock slice of
#510) to cross-region profiles and Converse bodies.
- Partially enables #181 (native, Python-free packaging): the native
binary now ships in the
images, though full Python-free distribution remains out of scope.
## Alignment with the Rust migration plan
Per `docs/spec/022-rust-migration.md`, the migration is **proxy-first**:
`headroom-proxy` is
the deployable Rust artifact, native routes replace Python passthroughs
one at a time
(Stage 4 = provider expansion, Bedrock included), and the binary is
meant to be "built,
tested, and **released together with the Python package**." Two ways
this PR advances that:
- The binary-in-images change makes the codebase do what the spec
already states (ship the
artifact) — closing the gap that forced downstreams to build from
source.
- Hardening the native Bedrock route (cross-region, Converse routing +
body compression) is
exactly the Stage-4 provider-expansion work, keeping the native path at
parity with real
traffic so it can be the default rather than a passthrough.
## Testing
- [x] Unit tests pass (`cargo test -p headroom-core -p headroom-proxy` —
full suites, 0 failures)
- [x] Linting passes (`cargo clippy -p headroom-core -p headroom-proxy
--all-targets -- -D warnings`)
- [x] Formatting passes (`cargo fmt -- --check`)
- [x] New tests added — `bedrock::vendor` (foundation +
inference-profile matching),
`extract_invoke_action` + converse upstream URL, and live-zone Converse
text-block routing
(`block_has_string_text_field`, converse-vs-anthropic dispatch
equivalence).
- [x] Manual testing performed
### Test Output
```text
$ cargo test -p headroom-core -p headroom-proxy # all suites: ok, 0 failed
$ cargo clippy -p headroom-core -p headroom-proxy --all-targets -- -D warnings # Finished, no warnings
$ cargo fmt -- --check # clean
# image validation (local, proxy/code extras):
$ docker build --target runtime ... # debian: /usr/local/bin/headroom-proxy, --help OK
$ docker build --target runtime-slim ... # distroless: binary links + --help OK
```
## Real Behavior Proof
- Environment: native Bedrock proxy against `bedrock-runtime.eu-west-2`,
SSO profile, model
`eu.anthropic.claude-haiku-4-5-20251001-v1:0`.
- Exact command / steps: POST a large multi-turn Converse body to
`/model/eu.anthropic.claude-haiku-4-5-20251001-v1:0/converse`;
separately build the
`runtime` + `runtime-slim` targets and run
`/usr/local/bin/headroom-proxy --help`.
- Observed result: before — `bedrock_compression_skipped` (geo-prefixed
id not recognized),
forwarded uncompressed to the wrong `/invoke` upstream; after —
geo-prefixed id recognized,
`/converse` forwarded to the `/converse` upstream, live-zone dispatcher
compresses the
Converse user-message text, measurable token savings. Images contain a
runnable
`headroom-proxy` in both variants.
- Not tested: non-Anthropic vendor compression parity (#953 follow-up);
Converse
`toolResult` nested-text compression (follow-up — only top-level
Converse text blocks
compress today).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- An earlier revision flipped the EventStream `Accept` default
(`*/*`/absent → passthrough);
**dropped** — `*/*` is what most clients (incl. reqwest and the proxy's
own metrics tests)
send while expecting SSE, so forcing passthrough breaks the standard SSE
path.
- The binary build adds the native-proxy compile to the image build;
happy to gate it behind
a build arg if maintainers prefer it opt-in.
- Addressed a Copilot review round: corrected the `/converse` upstream
routing, the stale
`run_anthropic_compression` comment, the Dockerfile cargo cache mount +
`--locked`, and the
nonroot AWS-credentials docs example.
2026-06-16 16:45:24 +02:00
|
|
|
|
aws-config = { version = "1", default-features = false, features = ["behavior-version-latest", "rustls", "rt-tokio", "sso"] }
|
fix(proxy): PR-D1 native Bedrock InvokeModel route + SigV4
Phase D PR-D1 lands the first native Rust path for AWS Bedrock,
replacing the lossy LiteLLM Python converter for Anthropic-on-Bedrock
non-streaming requests. Eliminates part of P4-37 and P4-39.
What landed
-----------
- New crates/headroom-proxy/src/bedrock/ module:
- envelope.rs: parses the {"anthropic_version": "...", ...}
Bedrock body shape; re-emits with anthropic_version preserved
as the first key (relies on serde_json preserve_order).
- sigv4.rs: AWS SigV4 signer wrapping the aws-sigv4 crate.
Forces PayloadChecksumKind::XAmzSha256 so x-amz-content-sha256
is in the canonical request, hashed over the post-compression
body bytes (the bytes that actually hit Bedrock). No silent
fallback: signing failures return 5xx with
event=bedrock_sigv4_failed.
- invoke.rs: POST handler for /model/{model_id}/invoke
(and /converse - same wire shape for anthropic.claude-*).
Detects Anthropic vendor via literal starts_with("anthropic.")
(no regex per project rule), routes Anthropic-shape bodies
through the existing compress_anthropic_request live-zone
dispatcher, then signs and forwards to the configured Bedrock
endpoint.
- Modified:
- proxy.rs: routes /model/:model_id/invoke and
/model/:model_id/converse when enable_bedrock_native is on
(default). Adds bedrock_credentials: Option<Arc<Credentials>>
to AppState.
- config.rs: new flags --bedrock-region (default us-east-1,
env HEADROOM_PROXY_BEDROCK_REGION), --bedrock-endpoint
(operator override for FIPS/VPC/test setups),
--enable-bedrock-native (default true), --aws-profile.
- main.rs: resolves AWS credentials at startup via
aws_config::defaults(BehaviorVersion::latest()). Failure logs
event=bedrock_credentials_unavailable at WARN; the handler
refuses to forward unsigned (event=bedrock_credentials_missing).
- Cargo.toml: workspace deps aws-sigv4, aws-config,
aws-credential-types, aws-smithy-runtime-api.
Tests
-----
8 integration tests under
crates/headroom-proxy/tests/integration_bedrock_invoke.rs:
1. native_envelope_round_trip_byte_equal
2. sigv4_signed_correctly_after_compression - confirms
authorization is SigV4-shape and x-amz-content-sha256
matches sha256(body received by upstream).
3. thinking_block_preserved_through_bedrock
4. redacted_thinking_preserved
5. document_block_preserved
6. tool_result_array_with_image_preserved
7. stop_sequence_null_only_when_present - pins that the proxy
does NOT inject stop_sequence: null (P4-37 hardcode).
8. tool_use_input_byte_equal_preserves_key_order
All eight pass. Full workspace test run is green; clippy + fmt
clean. make ci-precheck (rust + python + commitlint) passes
locally.
Build constraints honoured
--------------------------
- No silent fallbacks: missing creds / signing failures return
5xx with structured event=... log; no path ever forwards
unsigned.
- No hardcodes: region, endpoint, profile, enable-flag all
configurable via CLI + env.
- No regexes: vendor detection is str::starts_with.
- Comprehensive structured logs: event=bedrock_invoke_received,
bedrock_envelope_parsed, bedrock_compression_skipped,
bedrock_credentials_missing, sigv4_signed,
bedrock_invoke_forwarded, etc.
- Performant: body buffered once, passed by &[u8] to signer
(zero-copy), Bytes::clone only for ownership transfer to
reqwest. Sign exactly once per request.
- Elegant: 4 small focused modules mirror handlers/ + sse/.
- Tests use realistic Anthropic block content (real thinking,
redacted_thinking, document, base64 image fixtures).
Streaming (PR-D2) and observability (PR-D3) follow.
2026-05-03 16:22:32 -07:00
|
|
|
|
aws-credential-types = { version = "1", default-features = false }
|
|
|
|
|
|
# `Identity` lives in aws-smithy-runtime-api; the SigV4 builder
|
|
|
|
|
|
# accepts `&Identity`. Pinning the version explicitly avoids a
|
|
|
|
|
|
# silent semver bump from the transitive dep tree.
|
|
|
|
|
|
aws-smithy-runtime-api = { version = "1", default-features = false, features = ["client"] }
|
fix(proxy): PR-D4 native Vertex publisher path + ADC bearer auth
Adds a Rust-native Vertex AI publisher route ahead of the LiteLLM
Python converter (which dropped `thinking`, `redacted_thinking`,
`document`, `image`, `server_tool_use`, `mcp_tool_use` block kinds —
the P4-37 / P4-38 bug). After this PR the Vertex `:rawPredict` and
`:streamRawPredict` calls survive byte-equal upstream and benefit
from the live-zone Anthropic dispatcher (PR-B-series) running over
the body — same behaviour as `/v1/messages`.
New module `crates/headroom-proxy/src/vertex/`:
- `mod.rs` — single dispatch handler at the
`/v1beta1/.../models/:model_action` route. Splits the trailing
`:<verb>` segment with `str::rsplit_once(':')` (no regex) and
flips an `attach_sse_tee` flag to dispatch to the streaming or
non-streaming arm. Both verbs share one axum route shape because
matchit can't distinguish two patterns that overlap on a
parameter.
- `envelope.rs` — `VertexEnvelope` parser. Confirms
`anthropic_version` present + `model` field absent (the two
fingerprints of the Vertex envelope vs `/v1/messages`).
- `adc.rs` — `TokenSource` trait + `GcpAdcTokenSource` (production,
`gcp_auth` 0.12) + `StaticTokenSource` (tests). Caches tokens
with a 60s refresh-ahead-of-expiry window. Emits structured
`event = "vertex_adc_token_refreshed"` per refresh.
- `raw_predict.rs` — POST handler + shared `forward_vertex_request`.
Buffers body, parses envelope, runs live-zone Anthropic
compression, fetches ADC bearer, attaches
`Authorization: Bearer <token>` (overwrites client-supplied
Authorization header), forwards. SSE telemetry tee for the
streaming verb reuses PR-C1's `AnthropicStreamState` directly
(Vertex streams plain SSE, unlike Bedrock's binary EventStream).
- `stream_raw_predict.rs` — module-level docs + alias to the
shared dispatcher (the streaming-vs-non-streaming difference is
one boolean flag inside the shared forwarder).
Modifications:
- `proxy.rs::build_app` — registers the single Vertex route.
- `proxy.rs::AppState` — new `vertex_token_source: Arc<dyn TokenSource>`
field. Production constructs `GcpAdcTokenSource` lazily (no GCP
call until first `bearer()`); tests inject `StaticTokenSource`
via the new `AppState::with_token_source` helper.
- `config.rs` — adds `--vertex-region` / `HEADROOM_PROXY_VERTEX_REGION`
(default `us-central1`, observability tag only — the upstream URL
is `--upstream`) and `--vertex-adc-scope` /
`HEADROOM_PROXY_VERTEX_ADC_SCOPE` (default `cloud-platform`).
- `Cargo.toml` (workspace + proxy) — adds `gcp_auth = "0.12"` and
`async-trait = "0.1"`.
- `tests/common/mod.rs` — `start_proxy_with_state` accepts both
config + state customizers; `install_static_token_source` helper
for tests.
`crates/headroom-proxy/tests/integration_vertex_raw_predict.rs` —
all five tests pass:
1. `native_envelope_round_trip_byte_equal` — Vertex-shape body
(with `anthropic_version`, no `model`) round-trips SHA-256
byte-equal upstream.
2. `adc_bearer_token_signed_correctly` — `Authorization: Bearer
<static-test-token>` reaches upstream verbatim and OVERWRITES a
client-supplied Authorization header.
3. `thinking_block_preserved` — request with `thinking` (incl.
signature) + `redacted_thinking` (incl. opaque `data`) blocks
round-trips byte-equal even with `LiveZone` compression mode
enabled. This is the P4-37 / P4-38 teeth.
4. `stream_raw_predict_sse_handled` — `:streamRawPredict` proxies
an Anthropic SSE response (full `message_start` →
`content_block_delta` → `message_stop` sequence) back to the
client without corruption; SSE content-type preserved end-to-end;
bearer attached.
5. (bonus, no-silent-fallback contract)
`adc_failure_returns_5xx_no_silent_forward` — when the token
source returns `Err`, the proxy returns 5xx and never reaches
upstream. Verifies the `event = "vertex_adc_fetch_failed"`
error path.
Workspace: `cargo test --workspace` green; `cargo clippy --workspace
-- -D warnings` clean; `make ci-precheck` passes.
- No silent fallbacks: ADC failure → structured 5xx, never an
unauthenticated forward.
- No hardcodes: every knob (region, ADC scope, upstream URL) is
CLI-flag + env-var configurable.
- No regexes: axum path parameters + `str::rsplit_once` only.
- Comprehensive structured logs: `event` field on every decision
point — `vertex_envelope_parsed`, `vertex_envelope_invalid`,
`vertex_compression_skipped`, `vertex_compression_applied`,
`vertex_adc_token_refreshed`, `vertex_adc_fetch_failed`,
`vertex_streaming_pipeline_active`, `vertex_sse_stream_closed`,
`vertex_forwarded`, `vertex_unknown_verb`, etc.
- Performant: no body clone; ADC token cached + refreshed
ahead-of-expiry, not fetched per request.
- Comprehensive tests: realistic Anthropic block content
(signature payload, redacted_thinking opaque blob) in
`thinking_block_preserved`.
The local `gcloud auth application-default print-access-token`
returns no credentials, so manual validation against a real Vertex
endpoint is not possible in this PR. Follow-up: the user runs
`gcloud auth application-default login` once and exercises a live
Vertex request — should be a no-code-change check.
PR-D1 (Bedrock native) is running concurrently and will land its
own envelope module at `crates/headroom-proxy/src/bedrock/envelope.rs`.
The two envelope modules are intentionally siblings (not a shared
trait) — the shapes differ (Bedrock has a different
`anthropic_version` value, no `model` field, AWS SigV4 instead of
GCP ADC), and a premature shared abstraction would obscure the
provider-specific contracts. Whichever PR merges second rebases
without conflict.
Retires P4-38 (and the Vertex parts of P4-39); marketplace BYOC
pitch (per project memory) gets one more native provider.
2026-05-03 16:22:07 -07:00
|
|
|
|
# PR-D4: Vertex publisher path uses GCP Application Default Credentials
|
|
|
|
|
|
# (ADC) → bearer token for the `Authorization: Bearer <token>` header.
|
|
|
|
|
|
# `gcp_auth` resolves the chain (gcloud user creds, GCE/GKE metadata
|
|
|
|
|
|
# server, service-account JSON, workload-identity federation) without
|
|
|
|
|
|
# us baking provider-specific knowledge in. The token source is wrapped
|
|
|
|
|
|
# in a `TokenSource` trait so tests inject a static-token mock.
|
|
|
|
|
|
gcp_auth = "0.12"
|
2026-05-14 19:31:23 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# ── Release profile — wheel size optimization ───────────────────────
|
|
|
|
|
|
#
|
|
|
|
|
|
# PyPI imposes a 10 GB cumulative storage limit per project. We hit it
|
|
|
|
|
|
# at version 0.21.36 (191 versions × ~213 MB/release = 10.00 GB
|
|
|
|
|
|
# exactly). Recent wheels were ~16-18 MB each, of which ~6.4 MB was
|
|
|
|
|
|
# pure debug metadata (`.strtab` + `.symtab` ELF sections; uncovered
|
|
|
|
|
|
# by post-mortem inspection of an actual production wheel).
|
|
|
|
|
|
#
|
|
|
|
|
|
# This profile shrinks each Linux wheel from ~18 MB → ~10-11 MB by:
|
|
|
|
|
|
# * Stripping symbol/string tables (~6.4 MB direct savings)
|
|
|
|
|
|
# * Link-time optimization across crate boundaries (~5-10% .text
|
|
|
|
|
|
# savings via dead-code elim across the workspace)
|
|
|
|
|
|
# * Single codegen unit (better inlining + dead-code elim, at the
|
|
|
|
|
|
# cost of slightly slower release builds)
|
|
|
|
|
|
#
|
|
|
|
|
|
# We deliberately do NOT set ``panic = "abort"``. The proxy is a
|
|
|
|
|
|
# long-lived async process — a single misbehaving request triggering
|
|
|
|
|
|
# panic-abort would terminate the whole proxy and disconnect every
|
|
|
|
|
|
# concurrent client. Accept the smaller savings; keep unwind behaviour.
|
|
|
|
|
|
#
|
|
|
|
|
|
# Estimated impact: 213 MB/release → ~130 MB/release. Buys ~30+ more
|
|
|
|
|
|
# release slots within the 10 GB ceiling at the current release
|
|
|
|
|
|
# cadence. Per-PyPI-version savings AND faster downloads for end
|
|
|
|
|
|
# users. Tradeoff: release builds take ~30-50% longer due to
|
|
|
|
|
|
# `codegen-units = 1` + LTO; acceptable for the size win.
|
|
|
|
|
|
[profile.release]
|
|
|
|
|
|
strip = "symbols"
|
|
|
|
|
|
lto = "thin"
|
|
|
|
|
|
codegen-units = 1
|
2026-06-04 11:38:30 -07:00
|
|
|
|
|
|
|
|
|
|
# Fast-to-compile profile for CI test wheels. The shipped wheel uses
|
|
|
|
|
|
# `release` (lto + codegen-units=1) for runtime/size; CI only needs a working
|
|
|
|
|
|
# extension, so trade runtime perf for ~parallel, lto-free compilation. Used
|
|
|
|
|
|
# via `maturin build --profile ci`. Does NOT affect `--release` builds.
|
|
|
|
|
|
[profile.ci]
|
|
|
|
|
|
inherits = "release"
|
|
|
|
|
|
lto = false
|
|
|
|
|
|
codegen-units = 256
|
|
|
|
|
|
opt-level = 1
|
|
|
|
|
|
strip = "none"
|
|
|
|
|
|
debug = false
|
|
|
|
|
|
incremental = false
|