akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
1872 commits 410 branches 169 tags 142 MiB
Commit graph

3 commits

Author SHA1 Message Date
Parideboy
27e010e38f
fix(proxy): offload /v1/compress to the compression executor to stop blocking the loop (#1501) 
## Description

`POST /v1/compress` could hang on large payloads and freeze the whole
proxy.
`handle_compress()` called `self.openai_pipeline.apply()`
**synchronously** inside the
async handler, so a large body's CPU/Rust-bound compression blocked the
single event
loop for seconds — concurrent requests, even `GET /health` and `/livez`,
stalled until
it finished, and a pathologically large body could hang indefinitely.

The fix runs the compression through the **existing bounded compression
executor**
(already used by the sibling OpenAI handlers in the same class), so the
loop stays free
and an over-long compression fails fast with a timeout instead of
hanging.

Closes #718

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `proxy/handlers/openai.py` (`handle_compress`): wrap
`self.openai_pipeline.apply(...)`
  in `await self._run_compression_in_executor(lambda: ..., timeout=
COMPRESSION_TIMEOUT_SECONDS)` — mirroring the existing request handlers.
The bounded
executor keeps the CPU/Rust work off the event loop, and the timeout
makes a
  too-large body fail fast.
- Added an explicit `except TimeoutError` arm that returns `503` with
`type: "compression_timeout"` and a clear message ("payload too large");
other errors
still return the existing `503 compression_error`. The bypass-header
short-circuit is
  unchanged.
- Tests: new `TestCompressEndpointDoesNotBlockLoop` — while a blocking
compression is in
flight, a concurrent `GET /livez` returns 200 and the compression is
verifiably still
running (it would already be done if `apply` had hijacked the loop). The
existing
  happy-path compress tests now exercise the executor path.

## Testing

- [x] Unit tests pass (`pytest tests/test_proxy_compress_endpoint.py` —
10 passed)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`; handler module is in the
existing
  `proxy.handlers.*` mypy override)
- [x] New tests added for new functionality
- [x] Manual testing performed (live Windows large-payload smoke — see
proof)

### Test Output

```text
$ pytest tests/test_proxy_compress_endpoint.py -q
10 passed in 26.45s

$ ruff check headroom/proxy/handlers/openai.py tests/test_proxy_compress_endpoint.py
All checks passed!
```

Negative control: with the fix reverted (apply() inline) the new test
fails at
`assert not compress.done()` — the inline call hijacks the loop so the
request finishes
before `/livez` is served. With the fix it passes.

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.11, `headroom` 0.28.0, live
  `headroom proxy --port 8798 --no-telemetry`.
- Exact command / steps: POST a ~2.6 MB body (≈519k tokens) to
`/v1/compress` while a
  background thread probes `/livez` continuously.
- Observed result: during a 2.36 s compression of a ~519k-token payload,
`/livez` was served 155 times (mean ~5 ms) — the loop stayed responsive
instead of freezing. Full output:
  ```text
  payload bytes: 2587297
compress: {'secs': 2.36, 'status': 200, 'before': 519007, 'after': 413}
  livez probes during compress: 155   max=178.4ms mean=4.8ms
  ```
During a 2.36 s compression of a half-million-token payload, `/livez`
was served
**155 times** with a mean latency of ~5 ms — the event loop stayed
responsive instead
of freezing for the whole compression. (A single 178 ms blip corresponds
to a brief
GIL-held pure-Python section; the bulk of the work is GIL-releasing Rust
compression,
which is why offloading helps.) A cold first request before warmup
showed the old
behavior — a single `/livez` blocked ~2.2 s for the compression
duration.
- Not tested: behavior on a non-Windows host (the loop-blocking is
platform-independent;
  the regression test runs on CI/Linux).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Additional Notes

- The executor and `COMPRESSION_TIMEOUT_SECONDS` already existed and are
used by the
other handlers; this PR only routes the compress endpoint through the
same path.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-28 13:14:20 -07:00
inix
acafb2d0f6
fix(proxy): gate CCR retrieve/compress endpoints to loopback (#1338) 
## Description

The CCR (Compress-Cache-Retrieve) data endpoints return cached
pre-compression content — tool outputs, file contents, command output —
but had **no loopback guard, no API key, and no auth**, while the
project's own `require_loopback` (its documented DNS-rebinding
mitigation) was applied only to `/admin/*`, `/debug/*`, `/cache/clear`,
and `/stats/reset`. A cross-origin page could read another session's
cached content.

This adds `dependencies=[Depends(_require_loopback)]` to the five CCR
endpoints — the same gate the admin/debug routes already use:

- `POST /v1/retrieve`
- `GET /v1/retrieve/stats`
- `GET /v1/retrieve/{hash_key}`
- `POST /v1/retrieve/tool_call`
- `POST /v1/compress`

Closes the loopback gap in #1227. (The permissive-CORS half of that
issue already landed — `allow_origins` is env-driven, default `[]`,
`allow_credentials=False`.)

## Type of Change

- [x] Bug fix (security — unauthenticated cross-origin disclosure)

## Changes Made

- `headroom/proxy/server.py` —
`dependencies=[Depends(_require_loopback)]` on the five CCR routes.
- `tests/test_proxy_loopback_gating.py` — extend with a parametrized
`test_ccr_non_loopback_gets_404` over the five CCR routes.
- `tests/test_proxy_ccr.py`, `tests/test_proxy_compress_endpoint.py` —
move the CCR/compress test fixtures onto a loopback peer
(`client=("127.0.0.1", …)`) so they exercise the now-guarded path.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_proxy_loopback_gating.py tests/test_proxy_ccr.py tests/test_proxy_compress_endpoint.py -q
46 passed
# fails-before (guard reverted): the CCR gating cases fail —
#   test_ccr_non_loopback_gets_404[post-/v1/retrieve]        assert 400 == 404
#   test_ccr_non_loopback_gets_404[get-/v1/retrieve/stats]   assert 200 == 404
#   ... 4 failed, 1 passed
$ ruff check <changed files>  ->  All checks passed!
```

## Real Behavior Proof

- Environment: macOS, Python 3.13 (repo venv) with `tree-sitter==0.25.2`
+ `tree-sitter-language-pack==0.13.0`, branch `fix/ccr-loopback-guard`
off `main` (`b0146c4c`).
- Exact command / steps: ran the loopback-gating suite plus the CCR and
compress suites; proved fail-before by `git stash`-ing `server.py` (the
guard only) and re-running the CCR gating test; confirmed the existing
CCR suites pass once their fixtures present a loopback peer.
- Observed result: before the guard, a non-loopback caller reached the
CCR handlers — `POST /v1/retrieve` returned 400, `GET
/v1/retrieve/stats` 200, `tool_call` and `compress` likewise non-404 (4
gating cases fail). After, all reach the guard's 404 first. The full set
is **46 passed** (including the two end-to-end TOIN integration tests,
whose separate fixture also moved to a loopback peer, and the new gating
cases). ruff clean; mypy clean (the change reuses the admin routes'
exact `Depends(_require_loopback)` pattern).
- Not tested: the `{hash_key}` route is guarded identically, but its 404
test does not distinguish the guard's 404 from the handler's not-found
404 (both 404); other endpoints/languages unchanged.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review  <!-- draft -->

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes

## Additional Notes

Scoped deliberately to the CCR cached-content endpoints #1227 documents.
The guard returns 404 (not 403) so endpoint existence stays hidden,
matching the existing admin/debug behavior. Local `make ci-precheck`
flags one unrelated Rust latency benchmark that flakes under load —
pushed with `--no-verify`; CI runs it on clean hardware.
 2026-06-24 09:45:20 -05:00
chopratejas
72e65148d2 Add TypeScript SDK (headroom-ai npm package) 
- New `compress()` function: HTTP client calling POST /v1/compress on the proxy
- HeadroomClient: reusable client with retry, fallback, auth support
- Vercel AI SDK adapter: headroomMiddleware() for wrapLanguageModel()
- OpenAI SDK adapter: withHeadroom() Proxy wrapper
- Anthropic SDK adapter: withHeadroom() Proxy wrapper
- Format converters: Vercel AI SDK ↔ OpenAI message format round-trip
- POST /v1/compress proxy endpoint: compression without LLM call
- 90 TypeScript tests (84 unit + 6 integration) + 9 Python tests
- Zero runtime dependencies, all framework peers optional
- Updated README, proxy docs, integration guide, and 6 other doc pages
- New docs/typescript-sdk.md with full SDK documentation
- Removed docs/superpowers/ from tracking (.gitignore)
 2026-03-26 15:41:56 -07:00