feat(rust): smart_crusher SmartAnalyzer — field stats, change-points, crushability

Port Python's `SmartAnalyzer` class (smart_crusher.py:960-1489) to Rust.
All eight methods land here:

- analyze_array — top-level orchestrator: builds field stats, detects
  pattern, runs crushability, picks strategy, estimates reduction.
- analyze_field — per-field stats: type, count, uniqueness, plus
  type-specific (numeric: min/max/mean/variance/change_points; string:
  avg_length, top-5 frequencies).
- detect_change_points — sliding-window mean-shift detector for numeric
  fields; mirrors Python's deduped greedy walk with `> threshold` test.
- detect_pattern — classifies as time_series / logs / search_results /
  generic via structural signals (no field-name heuristics).
- detect_temporal_field — ISO 8601 prefix checks (handcoded byte-level
  to avoid a regex per call) + Unix epoch range checks.
- analyze_crushability — six-case decision tree with explicit signal
  list (id/score/structural-outlier/keyword-error/anomaly/change-points).
- select_strategy — strategy picker honoring `min_items_to_analyze`,
  `crushable=False` skip, and pattern-specific selections.
- estimate_reduction — coarse base+constant-ratio heuristic, capped 0.95.

Supporting pieces:

- stats_math.rs: mean / sample_variance / sample_stdev with n-1
  denominator (matches Python's `statistics` module).
- python_repr helper: str(v) parity for None/True/False/numbers/strings,
  used by _analyze_field's uniqueness count.
- top_n_by_count helper: Counter.most_common(n) parity with
  first-occurrence tie-break (mirrors Python dict insertion order).

Field iteration order: BTreeMap gives ASCII-sorted iteration. Python's
set-based for-key-in-all_keys is non-deterministic; Python source
needs a sorted(all_keys) for parity, scheduled for commit 7
(fixture regeneration alongside bug fixes).

Tests: 24 new unit tests covering empty/non-dict guards, numeric/string/
constant analyze_field paths, change-point detection on three-segment
step functions, log/generic pattern detection, ISO datetime + Unix
epoch temporal detection, six crushability cases, all strategy
selection branches, and reduction edge cases.

Net: 205 unit tests passing, clippy clean, parity harness still 4/4.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

crates/headroom-core/src/transforms/smart_crusher/mod.rs

@@ -32,6 +32,7 @@
 //! Each fix has a fixture entry in the parity harness and a corresponding
 //! test in `tests/test_transforms/test_smart_crusher_bugs.py`.
 
+mod analyzer;
 mod anchors;
 mod classifier;
 mod config;
@@ -40,8 +41,10 @@ mod field_detect;
 mod hashing;
 mod outliers;
 mod statistics;
+mod stats_math;
 mod types;
 
+pub use analyzer::SmartAnalyzer;
 pub use anchors::{extract_query_anchors, item_matches_anchors};
 pub use classifier::{classify_array, ArrayType};
 pub use config::SmartCrusherConfig;
@@ -52,6 +55,7 @@ pub use outliers::{
     detect_error_items_for_preservation, detect_rare_status_values, detect_structural_outliers,
 };
 pub use statistics::{calculate_string_entropy, detect_sequential_pattern, is_uuid_format};
+pub use stats_math::{mean, sample_stdev, sample_variance};
 pub use types::{
     ArrayAnalysis, CompressionPlan, CompressionStrategy, CrushResult, CrushabilityAnalysis,
     FieldStats,

crates/headroom-core/src/transforms/smart_crusher/stats_math.rs

@@ -0,0 +1,91 @@
+//! Numeric statistics helpers — port of Python's `statistics` module
+//! semantics used by `SmartAnalyzer`.
+//!
+//! Python's `statistics` module uses **sample** variance/stdev (n-1
+//! denominator), not population (n denominator). Mismatching the
+//! denominator silently shifts every variance-based decision (change
+//! points, anomaly thresholds, crushability cases). These helpers
+//! mirror Python's defaults.
+
+/// Arithmetic mean. Returns `None` on empty input — Python's
+/// `statistics.mean([])` raises `StatisticsError`; we model that as
+/// "no value to return", and callers must handle it.
+pub fn mean(values: &[f64]) -> Option<f64> {
+    if values.is_empty() {
+        return None;
+    }
+    let sum: f64 = values.iter().sum();
+    Some(sum / values.len() as f64)
+}
+
+/// Sample variance with `n-1` denominator (Python `statistics.variance`).
+/// Requires at least 2 values; returns `None` for fewer (mirrors
+/// Python which raises `StatisticsError` for n < 2).
+pub fn sample_variance(values: &[f64]) -> Option<f64> {
+    if values.len() < 2 {
+        return None;
+    }
+    let m = mean(values)?;
+    let sum_sq_diff: f64 = values.iter().map(|v| (v - m).powi(2)).sum();
+    Some(sum_sq_diff / (values.len() - 1) as f64)
+}
+
+/// Sample standard deviation — sqrt of `sample_variance`. Same n>=2
+/// requirement as the variance helper.
+pub fn sample_stdev(values: &[f64]) -> Option<f64> {
+    sample_variance(values).map(f64::sqrt)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    fn approx_eq(a: f64, b: f64) -> bool {
+        (a - b).abs() < 1e-9
+    }
+
+    #[test]
+    fn mean_empty_is_none() {
+        assert_eq!(mean(&[]), None);
+    }
+
+    #[test]
+    fn mean_single() {
+        assert!(approx_eq(mean(&[5.0]).unwrap(), 5.0));
+    }
+
+    #[test]
+    fn mean_basic() {
+        // Python: statistics.mean([1, 2, 3, 4, 5]) == 3.0
+        assert!(approx_eq(mean(&[1.0, 2.0, 3.0, 4.0, 5.0]).unwrap(), 3.0));
+    }
+
+    #[test]
+    fn sample_variance_too_few_values_is_none() {
+        // Python: statistics.variance([5]) raises; we return None.
+        assert_eq!(sample_variance(&[]), None);
+        assert_eq!(sample_variance(&[5.0]), None);
+    }
+
+    #[test]
+    fn sample_variance_uses_n_minus_1_denominator() {
+        // Python: statistics.variance([1, 2, 3, 4, 5]) == 2.5
+        // (Population variance with n in denominator would give 2.0.)
+        let v = sample_variance(&[1.0, 2.0, 3.0, 4.0, 5.0]).unwrap();
+        assert!(approx_eq(v, 2.5), "got {v}, expected 2.5");
+    }
+
+    #[test]
+    fn sample_stdev_basic() {
+        // Python: statistics.stdev([1, 2, 3, 4, 5]) == sqrt(2.5)
+        let s = sample_stdev(&[1.0, 2.0, 3.0, 4.0, 5.0]).unwrap();
+        assert!(approx_eq(s, 2.5_f64.sqrt()), "got {s}");
+    }
+
+    #[test]
+    fn sample_variance_constant_values_is_zero() {
+        // All-identical values: variance = 0.
+        let v = sample_variance(&[7.0, 7.0, 7.0]).unwrap();
+        assert!(approx_eq(v, 0.0));
+    }
+}