feat(text-crusher): fold full-width ASCII to half-width in CJK token keys (#2259)

## Description

Real CJK content routinely mixes full-width and half-width forms (`ＡＰＩ`
vs `API`, `０` vs `0`, the ideographic space `　` vs a normal space).
After #1504's ICU tokenization, these width variants produced
*different* token keys, so `ＡＰＩ` and `API` didn't dedup or match as the
same term on the CJK relevance and near-duplicate paths.

This folds full-width ASCII (U+FF01–U+FF5E, via the U+FEE0 offset) and
the ideographic space (U+3000 → space) to their half-width forms **when
building the internal token key** inside `tokens_icu`. Only the token
key is normalized — the kept output stays byte-verbatim, so
TextCrusher's extractive / byte-faithful contract is preserved.
CJK-gated (`tokens_icu` is the CJK path); the ASCII path is untouched.

## Type of Change

- [x] Bug fix / enhancement (non-breaking)

## Changes Made

- `crates/headroom-core/src/transforms/text_crusher/crusher.rs`: a
`width_fold(c)` helper applied when building token keys in `tokens_icu`.
- Rust unit tests: full-width ASCII folds to half-width in token keys;
CJK segments split on full-width terminators.

## Testing

- [x] Unit tests pass (`cargo test`)
- [x] Linting passes (`cargo clippy` / `cargo fmt`)
- [x] New tests added

### Test Output

```text
$ cargo test -p headroom-core --lib text_crusher
test result: ok. 13 passed; 0 failed
$ cargo clippy / fmt   # clean
```

## Real Behavior Proof

- Environment: macOS (Darwin), Rust via cargo, branch
`feat/text-crusher-fullwidth-fold` off `main` (rebased after #1504
merged).
- Exact command / steps: `cargo test -p headroom-core --lib
text_crusher`.
- Observed result: `fullwidth_ascii_folds_to_halfwidth` confirms `ＡＰＩ`
and `API` now produce the same token key (so they dedup/relevance-match
as one term); `cjk_splits_on_full_width_terminators` confirms full-width
`！`/`？` terminate segments. All 13 text_crusher tests pass; the kept
output is byte-verbatim (only the internal key is folded).
- Not tested: no Python side — TextCrusher is Rust-only, and output
stays byte-verbatim, so no parity fixtures change.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation — N/A
- [x] My changes generate no new warnings
- [x] I have added tests that prove my change is effective
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md — happy to add an entry if
preferred.

## Additional Notes

- Follow-up to #1504 (CJK-aware TextCrusher); it only touches the
CJK-gated `tokens_icu` path, so English tokenization is unchanged.

crates/headroom-core/src/transforms/text_crusher/crusher.rs

@@ -362,7 +362,19 @@ fn tokens_ascii(text: &str) -> Vec<String> {
     out
 }
 
-/// CJK path: ICU WordSegmenter (dictionary) word units; alnum-bearing, lowercased.
+/// Fold full-width ASCII variants (Ａ-Ｚ, ０-９, full-width punctuation) to their
+/// half-width form, and the ideographic space to a normal space. Real CJK text
+/// mixes these with normal ASCII; folding makes a token match regardless of
+/// width. Only the internal token KEY is folded -- the kept output stays verbatim.
+fn width_fold(c: char) -> char {
+    match c as u32 {
+        0xFF01..=0xFF5E => char::from_u32(c as u32 - 0xFEE0).unwrap_or(c),
+        0x3000 => ' ',
+        _ => c,
+    }
+}
+
+/// CJK path: ICU WordSegmenter (dictionary) word units; width-folded, lowercased.
 fn tokens_icu(text: &str) -> Vec<String> {
     let seg = *WORD_SEGMENTER;
     let mut out = Vec::new();
@@ -371,7 +383,7 @@ fn tokens_icu(text: &str) -> Vec<String> {
         if b > prev {
             let w = text[prev..b].trim();
             if !w.is_empty() && w.chars().any(|c| c.is_alphanumeric()) {
-                out.push(w.to_lowercase());
+                out.push(w.chars().map(width_fold).collect::<String>().to_lowercase());
             }
             prev = b;
         }
@@ -562,6 +574,22 @@ mod tests {
         );
     }
 
+    #[test]
+    fn fullwidth_ascii_folds_to_halfwidth() {
+        // full-width "ＡＰＩ" inside CJK must fold to the same token as "api",
+        // so dedup/relevance match across width variants.
+        let toks = tokens("认证ＡＰＩ密钥");
+        assert!(
+            toks.iter().any(|t| t == "api"),
+            "full-width ASCII should fold to 'api': {toks:?}"
+        );
+        // full-width digits too
+        assert!(
+            tokens("端口８０８０").iter().any(|t| t == "8080"),
+            "full-width digits should fold"
+        );
+    }
+
     #[test]
     fn cjk_relevance_keeps_query_match() {
         let needle = "认证令牌的缓存策略采用最近最少使用淘汰算法来管理过期。";