1 commit

Author	SHA1	Message	Date
Tejas Chopra	546ab553dc	feat(proxy): pilot hardening — inbound auth, security headers, audit log, air-gap switch (#1537) ... ## Description Tier-2 pilot security hardening from the engineering plan (Tier-1 landed in #1515). Four operator-facing controls, each verified open on `main` and grounded in a real exposure or enterprise requirement rather than a form answer: - **Optional inbound auth token (`HEADROOM_PROXY_TOKEN`).** When set, non-loopback callers to the data plane must present it (`Authorization: Bearer <token>` or `X-Headroom-Proxy-Token`); loopback callers and health probes are exempt. Constant-time (bytes) comparison. Closes the gap where the Docker image binds `0.0.0.0:8787` and exposes unauthenticated `/v1/*` routes to the pod network. A loud startup warning fires when binding a non-loopback host with no token set. - **Response security headers** (`X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: no-referrer`, HSTS) on every response, including 401s. - **Audit log for state-mutating admin endpoints.** A structured `headroom.audit` JSON event (source IP, method, path, status) for `/admin/*`, `/cache/clear`, `/stats/reset`; `/admin/runtime-env` additionally records the changed key names (values omitted so secrets are never logged). Logger-only — safe under `HEADROOM_STATELESS` (no new file writes). - **Air-gap master switch (`HEADROOM_OFFLINE=1`).** Hard-disables all outbound egress in one flag — telemetry beacon, update check, license/usage reporter, and HuggingFace model downloads (forces `HF_HUB_OFFLINE`/`TRANSFORMERS_OFFLINE`) — and logs an offline banner. The first three live in one outermost security middleware that wraps every inbound request; the offline switch is centralized in a new top-level `headroom/offline.py` predicate the egress paths consult. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `headroom/offline.py` (new): `is_offline()` predicate + `apply_offline_env()`; consulted by `beacon.is_telemetry_enabled`, `update_check.is_update_check_enabled`, and the license-reporter gate. - `headroom/proxy/audit.py` (new): `headroom.audit` structured logger + `record_admin_action` / `is_auditable_path`. - `headroom/proxy/server.py`: outermost `_security_gate` middleware (token enforcement + security headers + admin audit), offline activation + banner in `create_app`, non-loopback-no-token startup warning, `runtime-env` change auditing, env wiring in `_proxy_config_from_env`. - `headroom/proxy/models.py`: `ProxyConfig.proxy_token` and `ProxyConfig.offline`. - `headroom/cli/proxy.py`: env wiring + a Security banner line (flags the open-bind case). - `headroom/telemetry/beacon.py`, `headroom/update_check.py`: offline short-circuit. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check <changed files> All checks passed! $ mypy headroom/proxy/server.py headroom/offline.py headroom/proxy/audit.py headroom/proxy/models.py \ headroom/telemetry/beacon.py headroom/update_check.py Success: no issues found $ pytest tests/test_proxy_hardening.py -q 15 passed $ pytest tests/ -q (full suite, model/eval-dependent dirs ignored) 32 failed, 7131 passed, 126 skipped in 646s ``` The 32 failures are pre-existing/environmental, not introduced by this change — verified by running the same tests on `main` (they fail identically there). They are all `...Real` / `...live` / `real_api` integration tests that make live backend calls: AWS Bedrock returns "model is Legacy, access denied" on this host's credentials, plus a local tree-sitter version that requires `bytes`. On CI (no AWS/API creds) these tests skip. None touch the hardening code paths. ## Real Behavior Proof - Environment: macOS, Python 3.12, repo `.venv`; tests via FastAPI `TestClient` against `create_app`. - Exact command / steps: configure `ProxyConfig(proxy_token="...")`, then issue requests from a non-loopback client (`client=("203.0.113.5", ...)`) and a loopback client (`client=("127.0.0.1", ...)`). - Observed result: non-loopback request with no/!wrong token → 401; with correct `Authorization: Bearer` or `X-Headroom-Proxy-Token` → not 401; loopback and `/livez`/`/readyz` → never challenged. Every response (incl. the 401) carries `X-Content-Type-Options: nosniff` / `X-Frame-Options: DENY`. `POST /cache/clear` emits a `headroom.audit` JSON line with the source IP, path, and status. `HEADROOM_OFFLINE=1` makes `is_telemetry_enabled()` and `is_update_check_enabled()` return False and sets `HF_HUB_OFFLINE`/`TRANSFORMERS_OFFLINE`. - Not tested: WebSocket routes (see limitations); live upstream proxying of `/v1/*` (covered by existing integration tests / CI). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes Known limitations (by design / scope, documented in code): - WebSocket routes are not covered by the HTTP token middleware (`@app.middleware("http")` does not run for WS). The HTTP data plane is the main surface; the open-bind warning still applies. Follow-up. - The token keys off the direct peer IP — behind a same-host reverse proxy all requests appear loopback, so enforce auth at the reverse proxy in that topology (same property as the existing loopback guard). - OTEL metrics export is intentionally left on under offline mode — it targets the customer's own sink, not a Headroom phone-home. CHANGELOG not updated (handled by release tooling).	2026-06-28 12:09:00 -07:00