mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
1d2b76e72e
## Description Hardens persistent install startup and proxy compression behavior for issue #1843. Repeated `headroom install start` / scheduled ensure calls no longer spawn duplicate runtimes by default, and `/v1/compress` now fails open on compression timeout instead of returning a 503. The PR also adds a machine-readable platform feature matrix and app-level stabilization tests for health, compression functionality, timeout behavior, and matrix evidence. Refs #1843 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Wrapped direct persistent deployment starts with the existing profile-local runtime start lock. - Made `headroom install start` idempotent when the deployment is already healthy. - Added wedged-runtime handling: if a PID is running but `/readyz` does not recover inside the grace window, stop it before starting again. - Kept `install agent ensure` inside the already-held lock while delegating to the shared start helper. - Changed `/v1/compress` timeout behavior from `503 compression_timeout` to fail-open `200` with original messages, `compression_skipped: true`, and `skip_reason: compression_timeout`. - Added `tests/test_platform_stabilization_functional.py` covering real FastAPI health/compression routes, successful compression metrics, timeout fail-open speed, and a real JSON tool payload that reduces tokens. - Added `docs/platform-feature-matrix.json` and `docs/platform-stabilization.md` for Linux/macOS/Windows hardening coverage and known gaps. - Strengthened matrix tests so cited local test/workflow paths must exist. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # Clean source tree without copied Rust extension: functional module is skipped locally, as CI copies _core from the built wheel. > python -m pytest tests/test_install tests/test_cli/test_install_cli.py tests/test_platform_feature_matrix.py tests/test_platform_stabilization_functional.py -q collected 120 items / 1 skipped 119 passed, 2 skipped in 17.08s > python -m ruff check headroom/proxy/handlers/openai.py tests/test_platform_stabilization_functional.py tests/test_platform_feature_matrix.py All checks passed! # Local Windows compiled-core proof: > python -m maturin build --profile ci --out dist-local Built wheel for abi3 Python >= 3.10 to dist-local\headroom_ai-0.29.0-cp310-abi3-win_amd64.whl # Copied _core.pyd from the wheel into headroom/ for local route execution, then: > python -m pytest tests/test_platform_stabilization_functional.py -q collected 4 items 4 passed in 6.71s > python -m pytest tests/test_install tests/test_cli/test_install_cli.py tests/test_platform_feature_matrix.py -q collected 120 items 119 passed, 1 skipped in 17.16s Commit hooks: Sync plugin versions.....................................................Passed check for merge conflicts................................................Passed ruff.....................................................................Passed ruff-format..............................................................Passed mypy.....................................................................Passed ``` ## Real Behavior Proof - Environment: Windows 11, PowerShell, Python 3.13.13, worktree `C:\git\headroom-stabilization` on branch `jd/cross-platform-stabilization`. - Exact command / steps: built the Windows wheel with `maturin`, extracted `_core.pyd`, ran the new FastAPI route tests and install/matrix tests listed above, then removed generated artifacts before committing. - Observed result: direct start paths now no-op when healthy, skip spawning when the start lock is contended, and stop a wedged runtime before restart. `/v1/compress` now returns original messages quickly on timeout instead of a 503. The real JSON tool-payload smoke test returns `tokens_before > tokens_after`, `tokens_saved > 0`, `compression_ratio < 1.0`, and non-empty transforms through the public route. - Not tested: full native Windows persistent process e2e remains blocked by the upstream CRT/wheel issue already documented in workflows and in the matrix. No real OS service was installed locally; service manager behavior is covered by argument-level unit tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes CHANGELOG is not updated because this is an unreleased hardening/test/documentation pass. The platform matrix intentionally records partial/blocked Windows/macOS e2e gaps instead of claiming full coverage where the repo cannot currently run it.
28 lines
1.7 KiB
Markdown
28 lines
1.7 KiB
Markdown
# Platform Stabilization Matrix
|
|
|
|
This matrix is the hardening source of truth for install, startup, runtime, cache, and compression behavior across Linux, macOS, and Windows. The machine-readable matrix lives in `docs/platform-feature-matrix.json`; CI tests validate its shape so every feature has explicit platform status and test evidence.
|
|
|
|
## Status Values
|
|
|
|
- `covered`: unit, integration, or native e2e coverage exists for the platform.
|
|
- `partial`: coverage exists, but a named gap remains.
|
|
- `gap`: no meaningful coverage exists yet.
|
|
- `blocked`: coverage is intentionally excluded by a known external blocker.
|
|
|
|
## Current Priorities
|
|
|
|
1. Keep install/setup/run idempotent. Persistent starts must not create duplicate proxy instances unless a future explicit opt-in exists.
|
|
2. Keep compression fail-open. Cold model downloads, saturated executors, or timeout paths must pass through unchanged traffic instead of hanging agents.
|
|
3. Keep cache behavior visible. CCR persistence and TOIN skip recommendations must have restart and served-recommendation coverage.
|
|
4. Keep Windows honest. Windows-specific tests should run locally and in CI whenever they do not require the currently blocked native wheel build.
|
|
|
|
## Issue 1843 Coverage Map
|
|
|
|
- Windows service quoting: `install_windows_service`
|
|
- Duplicate startup processes: `single_instance_start`
|
|
- Slow compression and hangs: `compression_fail_open`
|
|
- CCR restart persistence: `ccr_persistence`
|
|
- TOIN recommendation wiring: `toin_skip_recommendations`
|
|
- Install/setup/run sanity: `install_apply_python`, `init_cli`, `wrap_prepare_only`
|
|
|
|
When adding or closing a hardening item, update `docs/platform-feature-matrix.json` in the same PR as the implementation or test change.
|