headroom/Cargo.toml

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

71 lines
3.3 KiB
TOML
Raw Normal View History

[workspace]
resolver = "2"
members = [
"crates/headroom-core",
"crates/headroom-proxy",
"crates/headroom-py",
"crates/headroom-parity",
]
# headroom-py is a Python extension module — it must be built via maturin, not
# plain cargo (the "extension-module" feature tells pyo3 not to link libpython,
# which is required for `import` to work). `cargo build --workspace` without
# explicit members skips it; `cargo test --workspace` still runs its tests
# because pyo3 can dynamically link here for the cdylib used by tests.
default-members = [
"crates/headroom-core",
"crates/headroom-proxy",
"crates/headroom-parity",
]
[workspace.package]
edition = "2021"
feat(rust): tokenizer crate with tiktoken-rs + HuggingFace + estimator Stage 2 of the Rust port: a `headroom_core::tokenizer` module mirroring the Python `headroom.tokenizers` surface, with three backends behind a single `Tokenizer` trait. Backends, in dispatch order: 1. HuggingFace (`HfTokenizer`) — pure-Rust `tokenizers` crate loading any public `tokenizer.json`. Covers the gap between OpenAI (tiktoken) and the Anthropic/Gemini estimator: Cohere `command-*`, Llama-3.x, Mistral, Qwen, BERT, T5, etc. Construct from bytes or a file path; register against a model-name prefix via `register_hf` for automatic dispatch. No `hf-hub` auto-download yet — keeps networking, auth, and `~/.cache/huggingface` out of core. Longest-prefix wins; lookups are RwLock-protected. 2. Tiktoken (`TiktokenCounter`) — `tiktoken-rs` 0.11 BPE for OpenAI / o-series families. Byte-identical to Python `tiktoken` for ordinary text. Lazy shared `Arc<CoreBPE>` per encoding (o200k_base, cl100k_base, p50k_base, r50k_base). 3. Estimation (`EstimatingCounter`) — `chars / cpt` last-resort fallback. Matches Python's `max(1, int(len(text) / cpt + 0.5))` round-half-up formula (a self-review caught and fixed an earlier `ceil`-based version that diverged in the middle of the range, e.g. 5 chars at 4.0 cpt). Tests: 43 unit tests + 5 proptests; parity 40/40 byte-equal. Bench: criterion baseline on small/medium/large inputs. Workspace MSRV bumped 1.78 → 1.80 for `LazyLock`/`OnceLock`. No proxy wiring. Library-only; production behavior unchanged.
2026-04-25 14:22:09 -07:00
rust-version = "1.80"
license = "Apache-2.0"
repository = "https://github.com/chopratejas/headroom"
authors = ["Headroom Maintainers"]
[workspace.dependencies]
serde = { version = "1", features = ["derive"] }
fix(rust): smart_crusher scaffold review findings — hash truncation, int parse, python-repr matcher Code review (`/code-review` on commit `d219bee`) caught one critical bug, two important parity gaps, and a few quality nits. Fixed all of them; all 135 unit tests pass; diff_compressor parity harness unaffected (27/27 still matched). # Critical fix — `hash_field_name` truncation length Rust truncated SHA-256 to **16** hex chars; Python uses **8** (per `smart_crusher.py:177`: `hashlib.sha256(...).hexdigest()[:8]`). 16-char hashes would never collide with TOIN's 8-char `preserve_fields`, silently disabling the entire `use_feedback_hints` cache lookup path. Fix: `hex[..8]` instead of `hex[..16]`. Three pinning tests re-verified against actual Python reference output. Doc comment now warns explicitly that the length must match Python or TOIN lookups silently miss. # Important fix — `python_int_parse` mirrors Python's `int()` semantics `statistics.rs::detect_sequential_pattern` previously called `s.parse::<i64>()`. Python's `int()` differs in three ways that affect realistic payloads: - strips ASCII whitespace (Rust's `parse` rejects) - accepts leading `+` (Rust accepts; same) - accepts PEP 515 underscores like `"3_000"` (Rust rejects) A field with `[" 1 ", " 2 ", " 3 ", "4", "5"]` would parse all five in Python (sequential = True) but only one in Rust (`nums.len() < 5` → False). Silent parity break. Fix: new private `python_int_parse` helper that strips whitespace, handles underscore separators, and rejects edge cases Python rejects. Six new tests pin the behavior. # Important fix — `python_repr` for `item_matches_anchors` Python compares anchors via `anchor in str(item).lower()`. We were using `serde_json::to_string(&item).to_lowercase()`, which differs in three ways that affect substring matching: - quote chars (`'` vs `"`) - bool/null literals (`True`/`False`/`None` vs `true`/`false`/`null`) - spacing (`key: value, ...` vs `key:value,...`) Anchor `"none"` would match Python form but not JSON. Inverse for `"null"`. Real divergence. Fix: new private `python_repr` walks `serde_json::Value` and emits Python-equivalent form. Plus enable `serde_json/preserve_order` at workspace level so `Value::Object` preserves JSON parse order (matching Python `dict` since 3.7). # Suggestion fixes - Classifier comment for `[True, False, 1] -> MIXED_ARRAY` now walks both Python and Rust paths step by step. - `ArrayAnalysis::field_stats` doc notes the BTreeMap vs Python-dict order nuance for the analyzer port to resolve. - Added regression tests for "all unparseable strings", "single int among strings", fractional-step sequential, and the email-typo pattern. # Build / test - `cargo build -p headroom-core` clean. - `cargo clippy -p headroom-core -- -D warnings` clean. - 135 unit tests in `headroom-core`, all passing (was 55). - `cargo run -p headroom-parity run` — diff_compressor 27/27 still matched.
2026-04-26 17:01:46 -07:00
# `preserve_order` makes `serde_json::Value::Object` use IndexMap so JSON
# parse order is preserved through Value→string→Value round-trips. The
# smart_crusher port relies on this to match Python's `str(dict)` output,
# which preserves insertion order; otherwise BTreeMap's sorted-key default
# would diverge from Python on every multi-key object.
fix(rust): A4 — honor cache_control markers; serde_json arbitrary_precision + raw_value PR-A4 of the Realignment Phase A lockdown (REALIGNMENT/03-phase-A-lockdown.md). Eliminates P0-3 (Rust proxy ignores customer cache_control markers) and P0-5 (numeric precision lost via serde_json::Value round-trip) at the library level; Phase B PR-B2 wires the helper into the live-zone block dispatcher. Cargo.toml — add `arbitrary_precision` and `raw_value` to `serde_json` workspace features. `arbitrary_precision` keeps `1.0` from collapsing to `1` and preserves >2^53 integers; `raw_value` exposes `&RawValue` so PR-B2 can forward unmodified `messages[*]` entries as exact byte copies. crates/headroom-core/src/cache_control.rs (new) — `compute_frozen_count` walks `messages[i].content[*].cache_control` via serde_json accessors only (no regex) and returns the smallest N such that `messages[i]` is frozen for every i < N. Markers in `system` or `tools[*]` log at debug! but never bump the floor (those fields are unconditionally cache-hot per invariant I2). TTL ordering violations (5m before 1h, guide §2.19) emit `tracing::warn!` but the function computes the correct count regardless — the customer's request, not ours to reject. crates/headroom-core/src/lib.rs — re-export `compute_frozen_count` at crate root so the proxy crate has a stable import path. crates/headroom-proxy/src/compression/anthropic.rs — add `resolve_frozen_count` thin wrapper that consults the `cache_control_auto_frozen` config flag. When `disabled`, returns 0 regardless of body content (operator opt-out for benchmarking). crates/headroom-proxy/src/config.rs — add `CacheControlAutoFrozen` enum and the matching CLI flag `--cache-control-auto-frozen` / env var `HEADROOM_PROXY_CACHE_CONTROL_AUTO_FROZEN`. Default is `enabled`. Documented in the doc comments. Tests - crates/headroom-core/src/cache_control.rs (inline): 11 unit tests covering marker detection, system/tools negative cases, ordering state machine, defensive (missing fields, non-array messages, non-object content blocks). - crates/headroom-core/tests/cache_control.rs: 11 unit + 3 property tests (monotonic non-decrease as markers are added; system/tools markers don't change count; empty messages → 0). - crates/headroom-proxy/tests/integration_cache_control.rs: 8 tests exercising the proxy wrapper (configurability gate; tracing capture for the 5m-before-1h warn path). Acceptance gates: `cargo build --workspace`, `cargo test --workspace` (33 new tests green), `cargo clippy --workspace -- -D warnings`, `cargo fmt --all --check` all clean. No new `regex::` imports; `git grep -n 'regex::' crates/{headroom-core/src/cache_control.rs, headroom-core/tests/cache_control.rs, headroom-proxy/tests/ integration_cache_control.rs}` empty. Honors the realignment build constraints: configurable (CLI + env), no hardcodes (TTL strings live as const), no regex (serde_json accessor walk), no fallbacks (one impl), structured logging (debug!/warn! with field/index/ttl/rule context), tests comprehensive (unit + property + integration + tracing capture).
2026-05-02 08:22:10 -07:00
#
# `arbitrary_precision` keeps the literal numeric token from the source
# JSON intact: `Value::Number` becomes a wrapper around the original
# digit string, so `1.0` does NOT collapse to `1`, and `12345678901234567`
# does NOT lose precision through f64. Required by Realignment invariant
# I1 (byte-faithful passthrough on unmutated bytes; see REALIGNMENT/02-
# architecture.md §2.2) and PR-A4 (see REALIGNMENT/03-phase-A-lockdown.md).
#
# `raw_value` exposes `serde_json::value::RawValue`, the unparsed JSON
# fragment type. Phase B PR-B2 uses this to forward unmodified
# `messages[*]` entries as exact byte copies — the parser captures the
# original byte slice, so byte-for-byte round-trips work even with
# whitespace, key order, or escape preferences the producer chose.
# Enabled here in Phase A so PR-B2 can land as a pure consumer change.
serde_json = { version = "1", features = ["preserve_order", "arbitrary_precision", "raw_value"] }
bytes = "1"
thiserror = "1"
tracing = "0.1"
anyhow = "1"
clap = { version = "4", features = ["derive"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread", "signal"] }
axum = "0.7"
tower = "0.5"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
pyo3 = "0.22"
fix(proxy): PR-D1 native Bedrock InvokeModel route + SigV4 Phase D PR-D1 lands the first native Rust path for AWS Bedrock, replacing the lossy LiteLLM Python converter for Anthropic-on-Bedrock non-streaming requests. Eliminates part of P4-37 and P4-39. What landed ----------- - New crates/headroom-proxy/src/bedrock/ module: - envelope.rs: parses the {"anthropic_version": "...", ...} Bedrock body shape; re-emits with anthropic_version preserved as the first key (relies on serde_json preserve_order). - sigv4.rs: AWS SigV4 signer wrapping the aws-sigv4 crate. Forces PayloadChecksumKind::XAmzSha256 so x-amz-content-sha256 is in the canonical request, hashed over the post-compression body bytes (the bytes that actually hit Bedrock). No silent fallback: signing failures return 5xx with event=bedrock_sigv4_failed. - invoke.rs: POST handler for /model/{model_id}/invoke (and /converse - same wire shape for anthropic.claude-*). Detects Anthropic vendor via literal starts_with("anthropic.") (no regex per project rule), routes Anthropic-shape bodies through the existing compress_anthropic_request live-zone dispatcher, then signs and forwards to the configured Bedrock endpoint. - Modified: - proxy.rs: routes /model/:model_id/invoke and /model/:model_id/converse when enable_bedrock_native is on (default). Adds bedrock_credentials: Option<Arc<Credentials>> to AppState. - config.rs: new flags --bedrock-region (default us-east-1, env HEADROOM_PROXY_BEDROCK_REGION), --bedrock-endpoint (operator override for FIPS/VPC/test setups), --enable-bedrock-native (default true), --aws-profile. - main.rs: resolves AWS credentials at startup via aws_config::defaults(BehaviorVersion::latest()). Failure logs event=bedrock_credentials_unavailable at WARN; the handler refuses to forward unsigned (event=bedrock_credentials_missing). - Cargo.toml: workspace deps aws-sigv4, aws-config, aws-credential-types, aws-smithy-runtime-api. Tests ----- 8 integration tests under crates/headroom-proxy/tests/integration_bedrock_invoke.rs: 1. native_envelope_round_trip_byte_equal 2. sigv4_signed_correctly_after_compression - confirms authorization is SigV4-shape and x-amz-content-sha256 matches sha256(body received by upstream). 3. thinking_block_preserved_through_bedrock 4. redacted_thinking_preserved 5. document_block_preserved 6. tool_result_array_with_image_preserved 7. stop_sequence_null_only_when_present - pins that the proxy does NOT inject stop_sequence: null (P4-37 hardcode). 8. tool_use_input_byte_equal_preserves_key_order All eight pass. Full workspace test run is green; clippy + fmt clean. make ci-precheck (rust + python + commitlint) passes locally. Build constraints honoured -------------------------- - No silent fallbacks: missing creds / signing failures return 5xx with structured event=... log; no path ever forwards unsigned. - No hardcodes: region, endpoint, profile, enable-flag all configurable via CLI + env. - No regexes: vendor detection is str::starts_with. - Comprehensive structured logs: event=bedrock_invoke_received, bedrock_envelope_parsed, bedrock_compression_skipped, bedrock_credentials_missing, sigv4_signed, bedrock_invoke_forwarded, etc. - Performant: body buffered once, passed by &[u8] to signer (zero-copy), Bytes::clone only for ownership transfer to reqwest. Sign exactly once per request. - Elegant: 4 small focused modules mirror handlers/ + sse/. - Tests use realistic Anthropic block content (real thinking, redacted_thinking, document, base64 image fixtures). Streaming (PR-D2) and observability (PR-D3) follow.
2026-05-03 16:22:32 -07:00
# Phase D PR-D1: AWS SigV4 signing for native Bedrock InvokeModel route.
# `aws-sigv4` provides the canonical-request + signing-key implementation;
# `aws-config` resolves credentials from the standard provider chain
# (env vars, profiles, IMDS, ECS task role, etc); `aws-credential-types`
# exposes `Credentials` so the signer accepts whatever the chain returned.
aws-sigv4 = { version = "1", default-features = false, features = ["sign-http", "http1"] }
aws-config = { version = "1", default-features = false, features = ["behavior-version-latest", "rustls", "rt-tokio"] }
aws-credential-types = { version = "1", default-features = false }
# `Identity` lives in aws-smithy-runtime-api; the SigV4 builder
# accepts `&Identity`. Pinning the version explicitly avoids a
# silent semver bump from the transitive dep tree.
aws-smithy-runtime-api = { version = "1", default-features = false, features = ["client"] }