fix(packaging): move hnswlib to optional [vector] extra so [all] needs no C++ toolchain (#1499)

## Description

`pip install "headroom-ai[all]"` aborts on any machine without a C++
toolchain.
`[all]` pulls `[memory]`, which was the only extra carrying
`hnswlib>=0.8.0`. hnswlib
compiles from source where no wheel matches the target, and that build
failure rolls
back the **entire** `[all]` install.

hnswlib is already fully optional at runtime: `MemoryConfig` defaults to
`VectorBackend.AUTO` → **sqlite-vec** (pure Python, no compiler), and
only falls back
to HNSW. So `[memory]` does not need hnswlib to function. This moves
hnswlib into a
dedicated optional `[vector]` extra, exactly like `[pytorch-mps]` is
already kept out
of `[all]`.

Closes #1368

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `pyproject.toml`:
  - Removed `hnswlib>=0.8.0` from `[memory]` (keeps `sqlite-vec` +
`sentence-transformers`; the default sqlite-vec backend still works).
- Added `vector = ["hnswlib>=0.8.0"]` for users who opt into the HNSW
backend.
- `[all]` still references `[memory]` (now hnswlib-free) and does
**not** add
    `[vector]`, so it resolves with no compiler.
- `[dev]` keeps `hnswlib`, so CI still installs and exercises the HNSW
backend tests.
- Docs: documented the new `[vector]` extra in `installation.mdx` and
the README, and
noted it is excluded from `[all]`; fixed the `[memory]` row that claimed
to bundle
  hnswlib.

No application code changed.

## Testing

- [x] Linting passes (`ruff check .`)
- [x] Manual testing performed (TOML resolution check — see proof)
- [ ] Unit tests pass (`pytest`) — no app code changed; existing
memory/HNSW tests are
unaffected (the HNSW backend dependency moved extras but `[dev]`/CI
still install it).

### Test Output

```text
$ python - <<'PY'  # resolve [all] transitively and check hnswlib placement
memory has hnswlib: False
vector has hnswlib: True
dev has hnswlib:    True
[all] resolved has hnswlib: False
[all] has sqlite-vec: True
[all] has sentence-transformers: True
PY

$ ruff check headroom/ tests/
All checks passed!
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.11; `tomllib` + a small
transitive-extra
  resolver over the edited `pyproject.toml`.
- Exact command / steps: parse `pyproject.toml`, expand
`headroom-ai[...]`
self-references in `[all]` recursively, then check which extras carry
`hnswlib`.
- Observed result: the resolved `[all]` set contains no hnswlib while
`[vector]` and `[dev]` do. Full output:
  ```text
  memory has hnswlib: False
  vector has hnswlib: True
  dev has hnswlib:    True
  [all] resolved has hnswlib: False
  [all] has sqlite-vec: True
  [all] has sentence-transformers: True
  ```
`[all]` now resolves with **no** hnswlib (so no compiler needed), while
the HNSW
  backend stays installable via `[vector]` and still tested via `[dev]`.
- Not tested: a real `pip install` on a compiler-less host (the failure
is a build-time
rollback that the resolver check captures deterministically); the
native-wrapper e2e
jobs that this `pyproject.toml` change triggers run `wrap` e2e, not the
memory HNSW
  path, so dropping hnswlib from `[all]` does not affect them.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Additional Notes

- Editing `pyproject.toml` trips the `e2e` path filter, so the
Windows/macOS/Docker
native-wrapper jobs also run on this PR. They install + run the `wrap`
e2e flow (not
  the memory HNSW backend), so the extras change is safe for them.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

README.md

@@ -101,7 +101,7 @@ headroom perf
 headroom dashboard                      # live savings dashboard (proxy must be running)
 ```
 
-Granular extras: `[proxy]`, `[mcp]`, `[ml]`, `[code]`, `[memory]`, `[relevance]`, `[image]`, `[agno]`, `[langchain]`, `[evals]`, `[pytorch-mps]` (Apple-GPU memory-embedder offload — set `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps`). Requires **Python 3.10+**.
+Granular extras: `[proxy]`, `[mcp]`, `[ml]`, `[code]`, `[memory]`, `[vector]` (optional HNSW backend — needs a C++ toolchain, not in `[all]`), `[relevance]`, `[image]`, `[agno]`, `[langchain]`, `[evals]`, `[pytorch-mps]` (Apple-GPU memory-embedder offload — set `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps`). Requires **Python 3.10+**.
 
 ## Proof
 
@@ -325,7 +325,7 @@ npm install headroom-ai                 # TypeScript / Node
 docker pull ghcr.io/chopratejas/headroom:latest
 ```
 
-Granular extras: `[proxy]`, `[mcp]`, `[ml]` (Kompress-base), `[code]`, `[memory]`, `[relevance]`, `[image]`, `[agno]`, `[langchain]`, `[evals]`, `[pytorch-mps]` (Apple-GPU memory-embedder offload — set `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps`). Requires **Python 3.10+**.
+Granular extras: `[proxy]`, `[mcp]`, `[ml]` (Kompress-base), `[code]`, `[memory]`, `[vector]` (optional HNSW backend — needs a C++ toolchain, not in `[all]`), `[relevance]`, `[image]`, `[agno]`, `[langchain]`, `[evals]`, `[pytorch-mps]` (Apple-GPU memory-embedder offload — set `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps`). Requires **Python 3.10+**.
 
 > **Note**: `[all]` covers the core stack but excludes framework adapters. Install them separately: `pip install "headroom-ai[langchain]"` (also `[agno]`, `[strands]`, `[anyllm]`, `[bedrock]`).
 

docs/content/docs/installation.mdx

@@ -41,7 +41,8 @@ pip install "headroom-ai[all]"
 | `proxy` | Proxy server, MCP tools, HTTP API | `pip install "headroom-ai[proxy]"` |
 | `ml` | Kompress (ModernBERT text compression, requires PyTorch) | `pip install "headroom-ai[ml]"` |
 | `code` | CodeCompressor (tree-sitter AST parsing) | `pip install "headroom-ai[code]"` |
-| `memory` | Persistent memory (hnswlib, sqlite-vec, sentence-transformers) | `pip install "headroom-ai[memory]"` |
+| `memory` | Persistent memory (sqlite-vec, sentence-transformers) — pure-Python default backend, no compiler | `pip install "headroom-ai[memory]"` |
+| `vector` | Optional HNSW vector backend (hnswlib) — needs a C++ toolchain; **not in `[all]`** | `pip install "headroom-ai[vector]"` |
 | `relevance` | fastembed-based relevance scoring (BAAI/bge-small-en-v1.5, ONNX) | `pip install "headroom-ai[relevance]"` |
 | `image` | Image compression (Pillow, ONNX runtime, OCR) | `pip install "headroom-ai[image]"` |
 | `reports` | HTML/Markdown report generation (Jinja2) | `pip install "headroom-ai[reports]"` |

pyproject.toml

@@ -107,12 +107,22 @@ ml = [
     # strands-agents`) drags huggingface-hub backwards.
     "huggingface-hub>=1.5.0,<2.0",
 ]
-# Memory system (hierarchical memory with vector search)
+# Memory system (hierarchical memory with vector search).
+# Uses the pure-Python sqlite-vec backend by default (VectorBackend.AUTO ->
+# SQLITE_VEC), so no C++ toolchain is required. The optional HNSW backend lives
+# in the [vector] extra below; installing it here would make `[all]` (which pulls
+# [memory]) fail on any machine without a compiler — see #1368.
 memory = [
-    "hnswlib>=0.8.0",
     "sqlite-vec>=0.1.6",
     "sentence-transformers>=2.2.0,<6.0",
 ]
+# Optional HNSW vector backend. Needs a C++ toolchain to build hnswlib, so it is
+# kept out of [memory] and [all]; opt in with `pip install headroom-ai[vector]`
+# and select it via MemoryConfig(vector_backend=VectorBackend.HNSW). The default
+# sqlite-vec backend needs no compiler.
+vector = [
+    "hnswlib>=0.8.0",
+]
 # Qdrant + Neo4j memory backend helpers
 memory-stack = [
     "mem0ai>=2.0.0,<3.0",