mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
5 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Abhinav Kumar Singh
|
739fdef423
|
fix(proxy): cancel periodic TOIN task on shutdown
## Description Retains the periodic TOIN statistics task on application state and reaps it during proxy lifespan shutdown. Fixes #2896 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Store the periodic TOIN task as `app.state.periodic_toin_stats_task` when enabled. - Cancel and await the task with the existing bounded shutdown helper before stopping proxy resources. - Clear the application state reference after shutdown. - Add regression coverage proving the task is canceled and reaped when the FastAPI lifespan exits. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text python -m pytest -q tests/test_proxy_telemetry_env.py 0 items / 1 error ModuleNotFoundError: No module named 'headroom._core' Temporary in-process native-core stub + real FastAPI TestClient: python -m pytest -q tests/test_proxy_telemetry_env.py 8 passed ruff check . All checks passed! ruff format --check . 1382 files already formatted python -m mypy headroom Success: no issues found in 515 source files python -m pytest -q Collected 8878 items / 174 errors / 18 skipped. Interrupted during collection because this Windows environment lacks the compiled headroom._core extension. ``` ## Real Behavior Proof - Environment: Windows, Python 3.12, real FastAPI `TestClient` lifespan; only the unavailable native `headroom._core` import was replaced with an in-process test stub. - Exact command / steps: Ran the telemetry test module with the temporary core stub. The new test enabled periodic TOIN stats, held the real lifespan open, observed the stored task, exited the `TestClient` context, and checked that the task was canceled and the state reference cleared. - Observed result: 8 telemetry tests passed, including the new shutdown regression test; the periodic task reported canceled after lifespan exit and no task reference remained on application state. - Who maintains it: Headroom Labs maintains this active upstream repository and proxy lifecycle. - Install surface: No dependencies or install behavior changed. The fix uses existing asyncio and FastAPI lifecycle APIs; no native code or runtime network access is introduced. - Not tested: The complete suite and the unmodified proxy test command cannot run in this Windows environment without the compiled `headroom._core` extension. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes (full suite blocked by missing native extension; stubbed focused tests pass) - [x] I did not edit `CHANGELOG.md` - it is generated by release-please from my Conventional Commit PR title. ## Screenshots (if applicable) Not applicable. ## Additional Notes The shutdown uses the existing three-second `_timed()` bound and handles the disabled configuration without creating a task. |
||
|
Tejas Chopra
|
53be64ca12
|
chore(telemetry): remove Supabase anonymous beacon; fix contact domain to headroomlabs.ai (#1526)
## Description
Removes the anonymous-telemetry **beacon** — the only external,
third-party data flow Headroom ever initiated. When telemetry was opted
in, it POSTed aggregate `/stats` to a hardcoded **Supabase** REST
endpoint (with an embedded anon API key in the source). For
enterprise/on-prem deployments this is exactly the kind of
vendor-controlled data egress a security review flags, so it's gone
entirely — **zero "Supabase" references remain in the codebase.**
What stays (by design): the **local** telemetry collector + the
`HEADROOM_TELEMETRY` opt-in (it only feeds `/stats` and `/v1/telemetry`
— nothing leaves the process), **OpenTelemetry export**
(`HEADROOM_OTEL_METRICS_*`, so operators send operational metrics to
*their own* collector), and the license usage reporter (your own domain,
license-key-gated).
Also fixes the contact domain: `headroom.dev` → `headroomlabs.ai`
everywhere.
Closes # (no tracking issue)
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)
> Non-breaking: `HEADROOM_TELEMETRY` is still accepted (now gates local
collection only). The only behavior change is that no telemetry is ever
sent externally.
## Changes Made
- **Deleted the Supabase beacon**: `TelemetryBeacon` class,
`_SUPABASE_URL`/`_SUPABASE_KEY`/`_TABLE`/`_ENDPOINT`, the JSONB
projection helper, the proxy-lifespan beacon wiring, the `SUPABASE_`
install env passthrough, and `tests/test_strategy_stats_supabase.py`.
- **Kept** the local opt-in predicate (`is_telemetry_enabled` etc.) in
`beacon.py` — still used by the local collector + CLI — reworded to
"local only".
- **Retained** the single-worker-owner file lock (the cc-switch
reconciler depends on it); updated its comments to drop the beacon
framing.
- `/stats` `anon_telemetry_shipping` is now always `False` (nothing
ships externally); startup log reworded to "Local telemetry".
- Reworded remaining "Supabase" comments in `collector.py`,
`context.py`, `prometheus_metrics.py`, and two test docstrings.
- Contact domain: `security@headroom.dev` → `security@headroomlabs.ai`,
`conduct@headroom.dev` → `conduct@headroomlabs.ai`, FUNDING.yml sponsor
URL.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ grep -rniI "supabase" --include=*.py --include=*.md --include=*.mdx . # (excl .venv/sbom)
>>> ZERO Supabase references
$ grep -rniI "headroom.dev" .
>>> ZERO headroom.dev references
$ ruff check <changed files> -> All checks passed!
$ ruff format --check <changed files> -> 10 files already formatted
$ mypy <changed telemetry files> -> Success: no issues found
$ pytest tests/test_telemetry.py tests/test_telemetry_warning.py \
tests/test_proxy_telemetry_env.py tests/test_compression_observability.py \
tests/test_paths.py tests/test_paths_backward_compat.py -q
============================= 173 passed in 6.67s ==============================
```
## Real Behavior Proof
- **Environment:** macOS, Python 3.12 (`.venv`).
- **Exact command / steps:** repo-wide grep for
`supabase`/`headroom.dev`; `create_app(...)` driven through a full
`TestClient` lifespan (startup + shutdown) in
`test_proxy_telemetry_env.py`; `/stats` exercised in
`test_telemetry_warning.py`.
- **Observed result:** zero `supabase`/`headroom.dev` strings remain;
the proxy starts and shuts down cleanly with the beacon removed (the
worker-owner lock + reconciler still elect a single owner);
`/stats.anon_telemetry_shipping` is `False` even with
`HEADROOM_TELEMETRY=on`; local collector + OTEL paths unchanged.
- **Not tested:** no live network call was ever made (the point — the
external POST is gone). OTEL export and the license reporter were not
exercised (unchanged by this PR).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- The license usage reporter (`reporter.py` → `app.headroomlabs.ai`) is
intentionally **kept** — it's license-key-gated (dormant for
unlicensed/OSS deployments) and goes to your own domain, not a third
party.
- Docs/CHANGELOG left unchecked: a couple of docs mention the telemetry
beacon and may want a follow-up note that it now collects locally only;
happy to add.
|
||
|
wstczyw
|
b514695efd
|
test(proxy): cover enabled periodic TOIN stats startup (#1268)
## Description Follow-up to #1265. Add coverage for the enabled branch of `periodic_toin_stats_enabled` during proxy lifespan startup. The original PR added the opt-out and disabled-path coverage. This test covers the default/enabled path so the new lifespan guard is not left partially covered. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `test_lifespan_schedules_periodic_toin_stats_when_enabled`. - The test patches `_log_toin_stats_periodically` with a short noop coroutine and verifies the proxy lifespan requests it when `periodic_toin_stats_enabled=True`. - This complements the existing disabled-path test from #1265. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_proxy_telemetry_env.py -q ============================= test session starts ============================= platform win32 -- Python 3.11.15, pytest-9.0.3, pluggy-1.6.0 rootdir: C:\Users\wstcz\AppData\Local\Temp\headroom-main-20260622-073524 configfile: pyproject.toml plugins: anyio-4.12.1, langsmith-0.8.0, asyncio-1.3.0, cov-7.0.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 10 items tests\test_proxy_telemetry_env.py .......... [100%] ============================== warnings summary =============================== .venv\Lib\site-packages\fastapi\testclient.py:1 C:\Users\wstcz\AppData\Local\Temp\headroom-main-20260622-073524\.venv\Lib\site-packages\fastapi\testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. from starlette.testclient import TestClient as TestClient # noqa -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ======================== 10 passed, 1 warning in 2.02s ======================== $ git diff --check origin/main..HEAD # no output; command exited 0 ``` ## Real Behavior Proof - Environment: Windows, Python 3.11.15 uv-managed `.venv`, branch based on current `origin/main`. - Exact command / steps: ran `uv run pytest tests/test_proxy_telemetry_env.py -q`. - Observed result: all 10 tests in `tests/test_proxy_telemetry_env.py` passed, including the enabled periodic TOIN stats lifespan branch. - Not tested: full repository pytest, ruff, and mypy were not run for this test-only follow-up. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes - Test-only follow-up to #1265. - No production behavior changes. - The focused pytest run still emits the existing Starlette/FastAPI TestClient deprecation warning from dependencies, so `My changes generate no new warnings` is intentionally left unchecked. |
||
|
wstczyw
|
b5f63d8fa9
|
fix(proxy): allow disabling periodic TOIN stats logging (#1265)
## Description
Add an explicit proxy configuration toggle for the periodic TOIN stats
logging loop.
Long-lived proxy workers currently schedule
`_log_toin_stats_periodically()` unconditionally at startup. This change
lets operators disable only that 5-minute stats logging loop via
`HEADROOM_PERIODIC_TOIN_STATS=0` when periodic stats collection creates
avoidable resource pressure. The default remains enabled, and this does
not disable TOIN learning or request-time feedback.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Added `ProxyConfig.periodic_toin_stats_enabled`, defaulting to `True`.
- Wired `HEADROOM_PERIODIC_TOIN_STATS` through
`_proxy_config_from_env()`.
- Guarded the proxy lifespan startup so `_log_toin_stats_periodically()`
is only scheduled when the config is enabled.
- Added tests for the default env behavior, disabled env values, and the
disabled lifespan behavior.
- Documented `HEADROOM_PERIODIC_TOIN_STATS` in the configuration
reference.
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv sync --extra dev
Resolved 256 packages in 1m 05s
Built headroom-ai @ file:///C:/Users/wstcz/AppData/Local/Temp/headroom-main-20260622-073524
Installed 124 packages in 50.87s
$ .\.venv\Scripts\python.exe -c "import headroom._core; print('core ok')"
core ok
$ uv run pytest tests/test_proxy_telemetry_env.py -q
============================= test session starts =============================
platform win32 -- Python 3.11.15, pytest-9.0.3, pluggy-1.6.0
rootdir: C:\Users\wstcz\AppData\Local\Temp\headroom-main-20260622-073524
configfile: pyproject.toml
plugins: anyio-4.12.1, langsmith-0.8.0, asyncio-1.3.0, cov-7.0.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 9 items
tests\test_proxy_telemetry_env.py ......... [100%]
============================== warnings summary ===============================
.venv\Lib\site-packages\fastapi\testclient.py:1
C:\Users\wstcz\AppData\Local\Temp\headroom-main-20260622-073524\.venv\Lib\site-packages\fastapi\testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================== 9 passed, 1 warning in 4.30s =========================
$ python -m py_compile headroom\proxy\models.py headroom\proxy\server.py tests\test_proxy_telemetry_env.py
# no output; command exited 0
$ git diff --check
# no output; command exited 0
```
## Real Behavior Proof
- Environment: Windows, Python 3.11.15 uv-managed `.venv`, source
checkout at `
|
||
|
Garm
|
8232e7822e | feat: allow overriding proxy telemetry sdk via HEADROOM_SDK |