3 commits

Author	SHA1	Message	Date
Devanshi Vyas	9f772378d3	test(proxy): assert CCR hash route guard blocks valid hashes (#1480) ... ## Description PR #1338 added loopback gating to the CCR retrieve/compress endpoints for #1227, but one regression case still had weak proof: `GET /v1/retrieve/{hash_key}` used a dummy hash. Since both the route guard and the missing-hash handler return `404`, that test could pass even if the route reached the handler. This adds a seeded-hash regression so the test proves the security property directly. Closes #1227 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) - [x] Add more testing ## Changes Made - Seed a real CCR entry using an in-memory compression-store backend. - Verify loopback can retrieve the seeded entry and see `original_content`. - Verify a non-loopback caller gets `404` for the same valid hash. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text 13 passed ``` ## Testing Commands ```bash .venv/bin/pytest tests/test_proxy_loopback_gating.py -q .venv/bin/pytest tests/test_proxy_loopback_gating.py tests/test_proxy_cors.py tests/test_proxy_compress_endpoint.py -q .venv/bin/ruff check tests/test_proxy_loopback_gating.py ``` ## Real Behavior Proof - Environment: macOS 25.4.0, Python 3.12 - Exact command / steps: `.venv/bin/pytest tests/test_proxy_loopback_gating.py -q` `.venv/bin/pytest tests/test_proxy_loopback_gating.py tests/test_proxy_cors.py tests/test_proxy_compress_endpoint.py -q` `.venv/bin/ruff check tests/test_proxy_loopback_gating.py` - Not tested: NA - Observed result: Pass ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable	2026-06-26 15:29:33 -07:00
inix	acafb2d0f6	fix(proxy): gate CCR retrieve/compress endpoints to loopback (#1338) ... ## Description The CCR (Compress-Cache-Retrieve) data endpoints return cached pre-compression content — tool outputs, file contents, command output — but had **no loopback guard, no API key, and no auth**, while the project's own `require_loopback` (its documented DNS-rebinding mitigation) was applied only to `/admin/*`, `/debug/*`, `/cache/clear`, and `/stats/reset`. A cross-origin page could read another session's cached content. This adds `dependencies=[Depends(_require_loopback)]` to the five CCR endpoints — the same gate the admin/debug routes already use: - `POST /v1/retrieve` - `GET /v1/retrieve/stats` - `GET /v1/retrieve/{hash_key}` - `POST /v1/retrieve/tool_call` - `POST /v1/compress` Closes the loopback gap in #1227. (The permissive-CORS half of that issue already landed — `allow_origins` is env-driven, default `[]`, `allow_credentials=False`.) ## Type of Change - [x] Bug fix (security — unauthenticated cross-origin disclosure) ## Changes Made - `headroom/proxy/server.py` — `dependencies=[Depends(_require_loopback)]` on the five CCR routes. - `tests/test_proxy_loopback_gating.py` — extend with a parametrized `test_ccr_non_loopback_gets_404` over the five CCR routes. - `tests/test_proxy_ccr.py`, `tests/test_proxy_compress_endpoint.py` — move the CCR/compress test fixtures onto a loopback peer (`client=("127.0.0.1", …)`) so they exercise the now-guarded path. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_proxy_loopback_gating.py tests/test_proxy_ccr.py tests/test_proxy_compress_endpoint.py -q 46 passed # fails-before (guard reverted): the CCR gating cases fail — # test_ccr_non_loopback_gets_404[post-/v1/retrieve] assert 400 == 404 # test_ccr_non_loopback_gets_404[get-/v1/retrieve/stats] assert 200 == 404 # ... 4 failed, 1 passed $ ruff check <changed files> -> All checks passed! ``` ## Real Behavior Proof - Environment: macOS, Python 3.13 (repo venv) with `tree-sitter==0.25.2` + `tree-sitter-language-pack==0.13.0`, branch `fix/ccr-loopback-guard` off `main` (`b0146c4c`). - Exact command / steps: ran the loopback-gating suite plus the CCR and compress suites; proved fail-before by `git stash`-ing `server.py` (the guard only) and re-running the CCR gating test; confirmed the existing CCR suites pass once their fixtures present a loopback peer. - Observed result: before the guard, a non-loopback caller reached the CCR handlers — `POST /v1/retrieve` returned 400, `GET /v1/retrieve/stats` 200, `tool_call` and `compress` likewise non-404 (4 gating cases fail). After, all reach the guard's 404 first. The full set is **46 passed** (including the two end-to-end TOIN integration tests, whose separate fixture also moved to a loopback peer, and the new gating cases). ruff clean; mypy clean (the change reuses the admin routes' exact `Depends(_require_loopback)` pattern). - Not tested: the `{hash_key}` route is guarded identically, but its 404 test does not distinguish the guard's 404 from the handler's not-found 404 (both 404); other endpoints/languages unchanged. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review <!-- draft --> ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes ## Additional Notes Scoped deliberately to the CCR cached-content endpoints #1227 documents. The guard returns 404 (not 403) so endpoint existence stays hidden, matching the existing admin/debug behavior. Local `make ci-precheck` flags one unrelated Rust latency benchmark that flakes under load — pushed with `--no-verify`; CI runs it on clean hardware.	2026-06-24 09:45:20 -05:00
Tejas Chopra	bd55a426bc	fix(proxy): scope CORS to loopback + gate operator/content endpoints (#1226) ... ## Description Locks down the proxy's browser- and network-facing attack surface, which matters most under a `--host 0.0.0.0` bind (the Docker default). The wildcard CORS policy (`allow_origins=["*"]` + `allow_credentials=True`) let any web page the user had open read the proxy's content endpoints — `/v1/retrieve` returns raw, uncompressed tool outputs (source, secrets) — via a cross-origin fetch to `127.0.0.1` (CWE-346). Several operator endpoints additionally leaked sensitive data or allowed unauthenticated state mutation to any network-reachable client. This PR scopes CORS to loopback origins and extends the project's existing `require_loopback` trust boundary (already used for `/admin/*` and `/debug/*`) to the remaining exposed endpoints. Closes #863. Supersedes #864 and #758 — see "Additional Notes". ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **CORS**: replaced `allow_origins=["*"]` + `allow_credentials=True` with a port-agnostic loopback origin regex (`https?://(localhost|127\.0\.0\.1|\[::1\])(:\d+)?`), `allow_credentials=False`, and methods/headers narrowed to `GET/POST` + `Content-Type/Authorization`. `HEADROOM_CORS_ORIGINS` (comma-separated) pins an explicit allowlist for Docker/remote dashboards; `*` opts back into the old wildcard. - **`/transformations/feed`** and **`/cache/clear`** gated behind `require_loopback` → 404 for non-loopback callers. The feed returns full prompt/completion bodies when `log_full_messages` is on; `/cache/clear` is unauthenticated state mutation (cache-eviction DoS / cost amplification). - **`/health`**: the `config` block (upstream API URLs, savings profile) is now served only to loopback callers; network callers get the `/readyz`-shape body (status/checks). `/livez` and `/readyz` remain unauthenticated probes for orchestration. - **`/stats`**: `recent_requests` / `request_logs` (per-request ids, providers, models, errors) and `config` are served only to loopback callers; aggregate counters stay public for remote monitoring. - Added `_request_is_loopback()` helper mirroring `require_loopback`'s two-gate check (loopback peer IP + loopback `Host` header, the DNS-rebinding defence) but degrading the payload instead of returning 404, so monitors keep the non-sensitive fields. - Tests: new `tests/test_proxy_cors.py` and `tests/test_proxy_loopback_gating.py`; updated 4 existing tests that assert the now-loopback-only data to use loopback clients. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/server.py tests/test_proxy_cors.py \ tests/test_proxy_loopback_gating.py tests/test_proxy_healthchecks.py \ tests/test_proxy_stats_recent_requests.py tests/test_proxy/test_transformations_feed.py All checks passed! $ mypy headroom --ignore-missing-imports Success: no issues found in 380 source files $ pytest tests/test_proxy_cors.py tests/test_proxy_loopback_gating.py \ tests/test_proxy/test_transformations_feed.py tests/test_proxy_healthchecks.py \ tests/test_proxy_stats_recent_requests.py tests/test_proxy_dashboard_stats_cache.py \ tests/test_proxy_compression_executor.py tests/test_header_isolation.py -q ======================== 82 passed, 1 skipped in 17.75s ======================== ``` ## Real Behavior Proof - Environment: macOS (darwin 25.4.0), Python 3.12 venv; FastAPI `TestClient` driving the real `create_app()` ASGI app - Exact command / steps: issued requests as a non-loopback caller (`client.host=testclient`) vs a loopback caller (`base_url=http://127.0.0.1`, `client=("127.0.0.1", 9999)`), plus CORS preflights with varying `Origin` headers - Observed result: CORS — `http://evil.com` → no `access-control-allow-origin`; `http://localhost:8787` and `http://localhost:9000` → echoed (loopback allowed on any port); `access-control-allow-credentials` → absent. `/cache/clear` and `/transformations/feed` → 404 (network) / 200 (loopback). `/health` `config` block present for loopback only. `/stats` `recent_requests` present for loopback only, while the aggregate `tokens` block stays present for network callers. - Not tested: a live `headroom proxy` process bound on `0.0.0.0` reached from a second host (simulated via ASGI peer/Host instead); end-to-end browser DNS-rebinding (covered by the `Host`-header gate and its unit test) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — proxy/middleware change; behavior is captured under "Real Behavior Proof". ## Additional Notes **Supersedes two stale PRs that target the same issue but have drifted from `main`:** - **#864** (`fix(proxy): scope CORS to localhost`, @gabiudrescu) — correct instinct and the source of the tighter `GET/POST` + `Content-Type/Authorization` scoping kept here, but it derived the allowlist from the `HEADROOM_PORT` env var (wrong when `--port` is passed as a CLI flag), carried ~40 lines of unrelated punctuation churn, and is ~125 commits behind `main`. The port-agnostic regex used here resolves the reviewer's port concern. - **#758** (`security: adversarial review`, @neogenix) — bundled these same application-layer fixes with a large CI/CD + Docker supply-chain pass. It is a ~160-commit-behind draft whose `server.py` no longer merges cleanly (`main` independently adopted the same `require_loopback` pattern). The application-layer fixes are rebased onto current `main` here; the CI/Docker/supply-chain hardening from #758 is still valuable and would be welcome as a separate, rebased PR. Thanks to @gabiudrescu and @neogenix for the original analysis (#863). **Deliberate scope / follow-ups (not in this PR):** - `/stats` aggregate counters and the basic `/health` body remain readable on a `0.0.0.0` bind by design, so remote monitoring keeps working. Full lock-down is a one-line `Depends(require_loopback)` each if preferred. - The `/v1/retrieve*` family stays network-reachable; it can't be loopback-gated without breaking legitimate remote/containerized agents and needs auth instead — tracked separately. - `ruff check .` is scoped to changed paths above because the dashboard HTML template trips ruff's `invalid-syntax` (a known repo false-positive); `mypy` is run over the full `headroom` package.	2026-06-21 00:50:55 -07:00