mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
cdba2eccdd
## Description `TextCrusher` and the BM25 relevance path can run without the ONNX-backed ML stack, but `headroom-core` previously compiled `ort`, `fastembed`, and `magika` unconditionally. This made lexical-only downstream consumers carry the ONNX Runtime dependency even when they never used embedding relevance or Magika detection. This PR makes those ML crates optional behind a new default-on `ml` Cargo feature. Default builds keep the existing ML-backed behavior. Consumers that only need lexical compression can opt out with `default-features = false`; in that mode the ML modules are compiled out and the relevance path falls back to BM25. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `crates/headroom-core/Cargo.toml`: marks `ort`, `fastembed`, and `magika` optional; adds default-on `ml = ["dep:ort", "dep:fastembed", "dep:magika"]`. - `crates/headroom-core/src/lib.rs`: gates the shared ONNX CPU helper behind `ml`. - `crates/headroom-core/src/relevance/embedding.rs`: gates the fastembed implementation behind `ml` and provides a no-ml stub with the same scorer surface so `HybridScorer` naturally falls back to BM25. - `crates/headroom-core/src/transforms/detection.rs`: gates the Magika tier behind `ml`; no-ml builds start at the existing unidiff/plain-text fallback tiers. - `crates/headroom-core/src/transforms/mod.rs`: gates the Magika module and re-exports behind `ml`. ## Testing - [x] Default build compiles (`cargo build -p headroom-core`) - [x] Lexical-only build compiles (`cargo build -p headroom-core --no-default-features`) - [x] Default tests pass (`cargo test -p headroom-core`) - [x] Lexical-only tests pass (`cargo test -p headroom-core --no-default-features`) - [x] Dependency tree checked for no-ml build (`cargo tree -p headroom-core --no-default-features` contains no `fastembed`, `magika`, or `ort` packages) - [ ] Manual testing performed ## Real Behavior Proof - Environment: Windows 11 review worktree, Rust/Cargo workspace. - Exact command / steps: - `cargo build -p headroom-core` - `cargo build -p headroom-core --no-default-features` - `cargo test -p headroom-core` - `cargo test -p headroom-core --no-default-features` - `cargo tree -p headroom-core --no-default-features` - Observed result: both feature configurations build and test cleanly. The no-default dependency tree does not include `fastembed`, `magika`, or `ort`, while the default build still compiles the ML path. - Not tested: model-backed `RUN_FASTEMBED_TESTS=1` cases that require downloading the embedding model; those remain env-gated as before. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The no-ml build intentionally degrades embedding relevance to the existing unavailable-model behavior, so `HybridScorer` takes its BM25 fallback path. Magika detection is skipped when `ml` is disabled; detection then proceeds through unidiff and plain-text fallback tiers. --------- Co-authored-by: Matthew Jackson <mattjackson86@gmail.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|---|---|---|
| .. | ||
| headroom-core | ||
| headroom-parity | ||
| headroom-proxy | ||
| headroom-py | ||
| headroom-simulators | ||