mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
228 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
JD Davis
|
2e29c7223f
|
fix(ci): guarantee model present in test shards to end cache-miss flakiness (#1399)
## Description
Fixes intermittent (`~25-test`) failures in `test` shards caused by a
GitHub Actions cache race between the `prefetch-model` job and the four
parallel `test` shards.
Closes #<!-- no upstream issue number yet -->
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- Added `id: restore-hfcache` to the "Restore HuggingFace model cache"
step in the `test` job so its cache-hit outcome is observable.
- Added a conditional "Fallback model download if cache missed" step
immediately after the restore, gated on
`steps.restore-hfcache.outputs.cache-hit != 'true'`. When the cache
misses it runs the same authenticated `snapshot_download` retry loop
that `prefetch-model` already uses (same
`snapshot_download('sentence-transformers/all-MiniLM-L6-v2')`, same
default `~/.cache/huggingface` cache root, same unpinned
`huggingface_hub` — byte-for-byte the warm path's mechanism), with
`HF_HUB_OFFLINE=0` / `TRANSFORMERS_OFFLINE=0` scoped to that step only,
so the model lands where pytest looks before pytest starts.
- `TRANSFORMERS_OFFLINE: "1"` on the actual `pytest` step is unchanged.
- The `prefetch-model` job and shared cache key remain the warm-path
optimisation.
- **Added `.github/workflows/**` to the `code` paths-filter group** (the
gate `test` / `prefetch-model` / `build-wheel` / `lint` read via
`needs.changes.outputs.code == 'true'`). Rationale: a change to *how the
tests run* must be validated by the test suite it governs. Without this,
a PR that only touches `ci.yml` matches only the separate `workflows`
filter, so `code=false` and every test job is **skipped** — a CI change
would merge on a hollow green having never executed the pipeline it
modifies. With this line **this PR is self-validating**: the four `test`
shards and `prefetch-model` actually run and exercise the new cache-miss
fallback path. The separate `workflows` filter is left unchanged.
- Polish: the fallback retry loop no longer sleeps after its final (6th)
attempt — it only backs off when another attempt will follow, saving up
to 30s of wasted runner time on a hard failure.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
YAML validation: python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/ci.yml')); print('YAML valid')"
→ YAML valid
pre-commit hooks (Sync plugin versions, ruff, ruff-format, mypy): all Passed/Skipped
```
## Real Behavior Proof
- **Root cause**: GitHub Actions cache is eventually-consistent. The
`prefetch-model` job saves the model under `Linux-models-allMiniLM-v2`.
The four `test` shards are independent runner VMs that restore from that
key concurrently. If a shard reaches the restore step before the cache
entry has propagated to the storage layer it gets a cache miss. With
`TRANSFORMERS_OFFLINE=1` on the runner and no model on disk, any test
that instantiates `LocalEmbedder` (≈25 tests) crashes with
`OSError`/`LocalEntryNotFoundError`. Since only some shards miss per run
the failure appears random.
- **Fix rationale**: The inline fallback approach (adding an `id` to the
restore step + a conditional download step) is the smallest possible
diff — two logical additions inside the existing `test` job, no new
jobs, no new artifacts, no changes to any other job. The alternative
(artifact-based sharing via `upload-artifact` / `download-artifact`)
would have been more reliable but required restructuring
`prefetch-model` and the `test` job more significantly. Given the
existing retry loop in `prefetch-model` already handles transient
HuggingFace failures, reusing it as a fallback is the right call.
- **Validated on this PR**: by adding `.github/workflows/**` to the
`code` filter, the `test` shards (×4) and `prefetch-model` execute on
this very PR and pass — so the modified pipeline is proven, not skipped.
- **Not tested**: a live cache miss is not deterministically
reproducible on-demand (it depends on Actions cache propagation timing);
the fallback is byte-for-byte the prefetch-model job's proven download
path, so its correctness rests on that parity.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works (N/A — CI-only change, no Python source modified)
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable (N/A — CI
infrastructure fix)
## Additional Notes
The `prefetch-model` job is preserved as the warm-path optimisation: on
a typical run the cache hits and the fallback step is skipped entirely
(no extra cost). The fallback only fires on the rare cache-consistency
miss that was previously causing flakiness.
|
||
|
Lakshya Sharma
|
88e67edf03
|
ci(release): publish win_amd64 wheel so Windows installs need no Rust (#1328) (#1335)
## Description We ship wheels for macOS arm64 and manylinux x86_64/aarch64, but there's no `win_amd64` wheel on PyPI for any Python version. So on Windows, pip/uv can't find a binary and try to build from the sdist with maturin, which pulls the Rust toolchain from static.rust-lang.org and crates from crates.io. On locked-down machines (corporate proxies, CI runners, the GitHub Copilot CLI sandbox, anything air-gapped) those hosts aren't reachable and the install just dies: ``` error: could not download file from 'https://static.rust-lang.org/dist/channel-rust-stable.toml.sha256' error: failed to get pyo3-macros as a dependency of package pyo3 v0.24.2 [28] Timeout was reached (Failed to connect to index.crates.io port 443) ``` This adds the Windows wheel to the release matrix so `pip install headroom-ai` works on Windows without a local Rust install. Closes #1328 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a `windows-latest` / `x86_64-pc-windows-msvc` row to the `build-wheels` matrix. The runner already has MSVC and maturin-action sets up Rust, so it produces `headroom_ai-*-win_amd64.whl` on every release. I checked `crates/headroom-core/Cargo.toml` first — the Windows ONNX path is already on `ort-load-dynamic` under `cfg(windows)`, so the wheel loads ORT at runtime instead of linking the DirectML SDK libs. Nothing else was needed on the Rust side. - Added a matching `windows-latest` row to `smoke-import-wheels` so a broken Windows wheel blocks publish like the other platforms do. Windows needed its own step: the venv puts Python under `Scripts\` not `bin/`, and the runner defaults to pwsh. I also pinned the shared script-staging step to `shell: bash` since it uses a heredoc that pwsh can't run (Git Bash is on the runner), and added a `setup-python` step to get the right minor version. - Updated the README install section so the "install Rust first" workaround is clearly only for the sdist fallback (e.g. Intel macOS) now that Windows/Linux/macOS-arm64 all have prebuilt wheels. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed This is a CI workflow + docs change, no Python runtime code. I leaned on the existing `tests/test_release_workflows.py` structural gates plus a YAML parse and matrix-shape sanity check. ### Test Output ```text $ python -m pytest tests/test_release_workflows.py -q 28 passed, 1 skipped, 1 failed # The one failure, test_no_native_tls_in_wheel_build_tree, shells out to cargo, which # isn't installed here. I confirmed with `git stash` that it fails the same way on main # without my changes, so it's pre-existing and unrelated. $ python -c "import yaml; d=yaml.safe_load(open('.github/workflows/release.yml',encoding='utf-8')); \ j=d['jobs']; print('build-wheels rows:', len(j['build-wheels']['strategy']['matrix']['include'])); \ print('smoke rows:', len(j['smoke-import-wheels']['strategy']['matrix']['include']))" build-wheels rows: 4 smoke rows: 6 ``` ## Real Behavior Proof - Environment: Windows 11 local clone; CI runs on GitHub-hosted `windows-latest`. - Exact command / steps: edited the build-wheels and smoke-import-wheels matrices in `.github/workflows/release.yml` and the README, then ran the release-workflow tests and the YAML/matrix-shape check above. - Observed result: tests pass, YAML parses, build matrix is now 4 rows (Linux x64, Linux arm64, macOS arm64, Windows x64) and the smoke matrix is 6 rows including the new native Windows row. - Not tested: the actual win_amd64 build + PyPI publish. Those jobs only run in the release workflow on a tag or workflow_dispatch, not on a feature PR. The PR-time release dry-run will exercise the new rows once a maintainer approves the workflow run. I couldn't run `maturin build --target x86_64-pc-windows-msvc` end to end here. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - No new test file: the existing structural gates in `tests/test_release_workflows.py` (`test_build_wheels_matrix_excludes_intel_macos`, `test_aarch64_wheel_uses_native_arm64_runner`, the smoke-import gate test) already assert the matrix contract and still pass with the Windows row added. - I didn't touch CHANGELOG.md — release-please generates it from the Conventional Commit subject, so the `ci(release):` commit gets picked up automatically. - The win_amd64 wheel actually shows up on PyPI on the next tagged release. |
||
|
Parideboy
|
d633e8172c
|
fix(windows): pin UTF-8 encoding on text-mode subprocess calls (#1311)
Fixes #1310. ## Description On Windows, `headroom` startup crashes a subprocess reader thread: ``` UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 7894: character maps to <undefined> ... subprocess.py _readerthread -> buffer.append(fh.read()) ... encodings/cp1252.py ``` Text-mode `subprocess` calls omit `encoding=`, so Python decodes child output with the locale codec (**cp1252** on Windows). Children that emit UTF-8 ??? `cbm index_repository` (indexing sources with chars like `???`/`???`), `claude mcp get/add`, the memory-sync process ??? produce bytes invalid in cp1252 and kill the reader thread. Linux/macOS default to UTF-8, so it's invisible there. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Add `encoding="utf-8", errors="replace"` to every text-mode (`text=True` / `universal_newlines=True`) subprocess call in the `headroom/` package (~50 call sites; several already had it). - `errors="replace"` (not `ignore`) so corrupt bytes surface as `???` rather than vanishing from parsed output. - Add `tests/test_cli/test_subprocess_utf8_encoding.py`: an AST guard asserting every text-mode subprocess call pins `encoding=`. The runtime crash can't reproduce on UTF-8 CI, so the invariant is enforced at the source level instead. ## Testing - [x] Unit tests pass (`pytest`) - New guard test passes (validates 51 call sites). - `tests/test_install`, `tests/test_cli/test_mcp.py`, `tests/test_mcp_registry` pass. (`test_runtime_start_lock_blocks_another_process` fails on this Windows box, but it fails identically on unmodified `main` ??? a pre-existing `msvcrt` lock flake, unrelated.) ### Test Output ```text > python -m pytest tests/test_cli/test_subprocess_utf8_encoding.py -q 1 passed in 0.12s > python -m pytest tests/test_install/ tests/test_cli/test_mcp.py tests/test_mcp_registry/ -q 133 passed, 2 skipped in 15.34s ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13.13. - Exact command / steps: Started `headroom` without `PYTHONUTF8=1` on a repo with UTF-8 chars in indexable files. Observed the `UnicodeDecodeError` crash. Applied the fix (pinning `encoding="utf-8"` on all text-mode subprocess calls). Re-ran. No crash. The AST guard enforces the invariant on CI (which runs UTF-8 locales and cannot reproduce the cp1252 crash natively). - Observed result: Subprocess reader threads no longer crash on UTF-8 output under cp1252 locale. - Not tested: All third-party tools that `headroom` shells out to; each was given `errors="replace"` as a safety net. ## Workaround for affected users (before fix is deployed) `PYTHONUTF8=1` (PowerShell: `$env:PYTHONUTF8=1; headroom ...`). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Rod Boev
|
ad7993bf15
|
fix(codex): stop pinning Codex memory MCP to one project db (#1269)
## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic. |
||
|
Ashish
|
23d73ae070
|
test(evals): add offline fidelity regression gate (recall-based, zero-model) (#1187)
## Description Headroom's lossy compression drops rows/lines using statistical heuristics but **never checks that meaning survived** — a dropped `OOM killed worker 3` line can silently flip a model's answer with no signal that compression caused it. The repo already ships a quality-metric toolkit (`headroom/evals/metrics.py`) and a `weekly-suite` eval job, but neither gates the compression path on a PR. This adds a **per-PR fidelity regression gate**: compress vendored golden tool-outputs through SmartCrusher's lossy path and assert the evidence that answers each case's question survives. It is the first of a planned trio (this is the "offline gate" half of the fidelity work); query-aware retention and a hard token-budget API are documented follow-ups. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Blocking gate** (`tests/test_compression_fidelity_regression.py`): compresses each golden case via `smart_crush_tool_output(..., with_compaction=False)` and scores with `compute_information_recall`. Two assertions: - **Per-case critical recall == 1.0** — every `answer_evidence` string (placed in error/anomaly rows, the documented SmartCrusher retention guarantee) must survive. - **Aggregate recall ≥ committed baseline** (`baseline.json`, tol 0.02) — catches softer regressions. - **Vendored fixtures** (`tests/fixtures/fidelity_golden/`): deterministic `_generate.py` emits `cases.json` (4 cases: OOM crash, payment exception, latency anomaly, CI failure) + `baseline.json`. - **Non-blocking weekly report** (`.github/workflows/eval.yml`): one step in the existing `weekly-suite` job (schedule/manual only) reuses the existing `evaluate_information_retention` runner for a recall report on the production routing path. - **Pure reuse**: scoring (`evals/metrics.py`), compressor (`smart_crush_tool_output`), and the weekly runner (`evaluate_information_retention`) all already existed. ### Design notes - **Zero new CI setup.** The blocking gate runs in the existing `[dev]` test shard — no new workflow, no new deps, **no model, no network, no secrets** (verified under `HF_HUB_OFFLINE=1`). It deliberately uses small hand-made structured fixtures rather than the repo's HuggingFace dataset loaders, which would require a network download + ModernBERT and don't belong in a fast PR gate. - **Scope:** structured JSON tool-output (the dominant, deterministic, model-free case). Real-dataset (HotpotQA/BFCL) recall — which needs `[all]` + a local model — is a **documented follow-up PR**, and the `weekly-suite` job (which genuinely runs every Monday) is its natural home. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ HF_HUB_OFFLINE=1 python -m pytest tests/test_compression_fidelity_regression.py -v tests/test_compression_fidelity_regression.py::test_critical_evidence_survives_compression[logs_oom] PASSED tests/test_compression_fidelity_regression.py::test_critical_evidence_survives_compression[payment_exception] PASSED tests/test_compression_fidelity_regression.py::test_critical_evidence_survives_compression[latency_anomaly] PASSED tests/test_compression_fidelity_regression.py::test_critical_evidence_survives_compression[ci_test_failures] PASSED tests/test_compression_fidelity_regression.py::test_aggregate_recall_not_regressed PASSED ============================== 5 passed in 0.18s =============================== ``` ## Real Behavior Proof - **Environment:** local checkout of `feat/fidelity-regression-gate`, `pip install -e ".[dev]"`, `HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1` (proves no model/network). - **Exact command / steps:** `HF_HUB_OFFLINE=1 python -m pytest tests/test_compression_fidelity_regression.py -q` → `5 passed in 0.14s`. - **Negative control (proves the gate has teeth):** compressing `logs_oom` and probing for a benign row that compression legitimately drops returns `recall = 0.00, lost = ['heartbeat ping 25']` — i.e. the gate fires when critical evidence is dropped, so it is not trivially green. - **Weekly (non-blocking) step verified locally:** ```text Information retention: 50/50 cases >=0.9 recall, avg compression 65.7% ``` - **Not tested:** real-dataset (HotpotQA/BFCL) recall and prose/ModernBERT compression — intentionally deferred to a follow-up PR targeting the weekly job. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - CHANGELOG/version intentionally untouched: repo uses **release-please**. - **Follow-up PR (planned):** wire the real HotpotQA/BFCL loaders (`headroom/evals/datasets.py`) into the `weekly-suite` job for genuine benchmark-scale recall coverage (model-allowed, non-blocking). Further follow-ups from the same design: a live per-request fidelity guardrail, query-aware lossy retention, and a hard `target_tokens` budget API. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Parideboy
|
5194388b66
|
fix(ci): normalize Windows CRLF line endings in PR governance script (#1012)
## Description The `CODE_BLOCK_RE` regex in `scripts/pr-governance.py` expects LF after the opening fenced code block. PR bodies authored on Windows can arrive with CRLF line endings, which leaves a `\r` before the `\n` and prevents `has_test_output()` from detecting a valid Test Output block. This normalizes CRLF to LF once when loading the pull request body, before section extraction and code-block matching. A regression test now verifies that a valid PR body with CRLF line endings still passes governance. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Tests only ## Changes Made - Normalize Windows CRLF line endings in `scripts/pr-governance.py` before regex-based validation runs. - Added `test_validate_pull_request_accepts_crlf_test_output_code_block` to prevent regressions. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Formatting passes (`ruff format --check`) - [x] New tests added for new functionality ### Test Output ```text python -m pytest scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py -q 8 passed in 0.06s ruff check scripts/pr-governance.py scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py All checks passed! ruff format --check scripts/pr-governance.py scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py 4 files already formatted ``` ## Real Behavior Proof - Environment: local Windows 11 checkout, Python 3.13.13. - Exact command / steps: Converted the known-valid governance test body to CRLF line endings and passed it through `validate_pull_request` in the new regression test. - Observed result: The report is valid with no problems, proving the fenced Test Output block is recognized after normalization. - Not tested: GitHub-hosted Windows PR authoring path end to end; the unit test covers the exact CRLF body shape consumed by the validator. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Parideboy
|
487aa71a3c
|
ci: restore green lint (reformat for ruff 0.15.17, fix mypy no-any-return, pin linters) (#1295)
## Description
The CI lint job (`ruff check .` → `ruff format --check .` → `mypy
headroom`) was red on `main`
and therefore on every open PR, for two unrelated reasons that the early
ruff failure was masking:
1. **ruff**: the lint job installs `ruff` unpinned, and ruff 0.15.17
began enforcing import-block
sorting (`I001`) and formatting that older ruff accepted → `ruff check
.` / `ruff format --check .`
fail on files nobody touched.
2. **mypy**: `headroom/providers/opencode/config.py` had two `return
json.loads(...)` statements in
a function declared `-> dict[str, Any]`; `json.loads` is typed `Any`, so
`mypy headroom` fails
with `no-any-return` (reproduced on mypy 1.20.2 — not a version-specific
quirk).
This restores a green lint baseline and pins both linters so a future
release can't silently break
CI again.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `.github/workflows/ci.yml`: pin `ruff==0.15.17` and `mypy==1.20.2` in
the lint job.
- Applied `ruff check --fix .` (6 `I001` import-sort fixes) and `ruff
format .` (10 files) across
the repo — import ordering and whitespace only, no behavior change.
- `headroom/providers/opencode/config.py`: narrow both
`_parse_json_loose` return sites with an
`isinstance(parsed, dict)` guard, so the `dict[str, Any]` annotation is
true at runtime
(non-dict JSON falls back to `{}`) and mypy's `no-any-return` is
resolved.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`, `ruff format --check .`, `mypy
headroom --ignore-missing-imports`)
### Test Output
```text
$ python -m ruff check .
All checks passed!
$ python -m ruff format --check .
913 files already formatted
$ python -m mypy headroom/providers/opencode/config.py --ignore-missing-imports
Success: no issues found in 1 source file
$ python -m pytest tests/test_providers_opencode_config.py -q
37 passed
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.13, ruff 0.15.17, mypy 1.20.2,
branch ci/fix-ruff-lint off
headroomlabs-ai/main
- Exact command / steps: reproduced the red lint (latest ruff: 6 `I001`
+ 10 unformatted files;
the mypy failure was read from the #1295 CI lint log —
`config.py:125,133 no-any-return`, and
re-confirmed locally on mypy 1.20.2). Applied the ruff auto-fix/format,
added the dict guard,
pinned both linters, and re-ran each lint step.
- Observed result: `ruff check .` → "All checks passed!"; `ruff format
--check .` → "913 files
already formatted"; `mypy` on the fixed file → "Success: no issues
found"; full `mypy headroom`
reports only Unix `fcntl` attributes that don't exist on this Windows
box (present on the Linux
CI runner, where the prior run showed exactly the two now-fixed errors).
37 opencode-config
tests pass.
- Not tested: did not run the full OS/Python test matrix — the change is
formatting + two CI
dependency pins + a two-line type-narrowing guard, with no runtime
behavior change for dict JSON.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Ashish
|
d789a7c528
|
feat(transforms): tabular + spreadsheet (.xlsx/.xls) compression (#1128)
## Description Adds a content-type-aware path for **tabular data** — CSV/TSV, markdown tables, fixed-width text, and binary `.xlsx`/`.xls` spreadsheets — by routing them through the existing, battle-tested `SmartCrusher` instead of letting them fall through to `PLAIN_TEXT → Kompress`. The pipeline already compressed tables losslessly when handed a JSON array of records. This wires up the missing front door: detect tabular text (and ingest binary spreadsheets), convert to JSON records, and reuse `SmartCrusher.crush()`. No new compression algorithm. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Detection** (`content_detector.py`): new `ContentType.TABULAR` + `_try_detect_tabular()` for CSV/TSV, markdown tables, and fixed-width columns. Ordered after search/log (which also look "delimited") and before code, with a prose-rejection guard so it never steals `file:line:content` search output, `key: value` logs, or sentences with incidental commas. Rust backend returns `plain_text` for unknown types and the router already falls back to the Python detector, so **no Rust change**. - **Bridge** (`tabular_ingest.py`): stdlib parsers + `to_records()` + a `TabularCompressor` that parses → JSON records → `SmartCrusher` (lossless `csv-schema` first; lossy row-drop with reversible `<<ccr:HASH>>` markers stays SmartCrusher's built-in fallback). Only adopts a result when it actually saves bytes. - **Spreadsheets** (`spreadsheet_ingest.py`): `.xlsx`/`.xls` → per-sheet CSV text at the SDK boundary. Optional deps (`pip install headroom-ai[spreadsheet]`) fail loudly with an install hint, never silently degrade. - **Routing** (`content_router.py`): `CompressionStrategy.TABULAR`, `enable_tabular_compressor` flag, lazy getter, apply branch, strategy maps, Kompress fallback eligibility. - **SDK** (`compress.py`): `compress_spreadsheet(path, ...)` helper (one message per sheet). - **Packaging** (`pyproject.toml`): new `[spreadsheet]` extra; `openpyxl` added to `[dev]` so the xlsx path is exercised in CI. - **Docs/demo**: `examples/tabular_compression_demo.py` + README entry. ### Design note: lossless-only Compact, all-unique tables with no query yield ~0 savings — this is correct, not a bug. SmartCrusher returns `skip:unique_entities_no_signal` and won't drop unique rows without a duplicate/relevance signal. Real wins come from verbose/redundant tables and query-driven selection. A pressure-driven lossy row sampler was considered and intentionally not added. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_transforms_tabular.py -q collected 20 items tests/test_transforms_tabular.py .................... [100%] ============================== 20 passed in 7.15s ============================== $ ruff check headroom/transforms/tabular_ingest.py headroom/transforms/spreadsheet_ingest.py All checks passed! $ mypy headroom/transforms/tabular_ingest.py headroom/transforms/spreadsheet_ingest.py Success: no issues found in 2 source files ``` `tests/test_transforms_tabular.py` (20 tests): detection true positives + no-misroute negatives (search/log/JSON/prose), parser units (incl. fixed-width), the CSV→SmartCrusher bridge, router routing + disable flag, and `.xlsx` ingestion (skipif openpyxl missing) + error paths. `spreadsheet_ingest` 100% / `tabular_ingest` 90% line coverage. ## Real Behavior Proof - **Environment:** local checkout of `feat/tabular-compression`, Python 3.x, `pip install -e ".[dev]"`. - **Exact command / steps:** `python examples/tabular_compression_demo.py` (no API key required). - **Observed result:** ```text === Raw tabular text (ContentRouter, char-level) === compact unique CSV strat=tabular chars 1306 -> 1072 ( 17.9% saved) redundant CSV strat=tabular chars 2661 -> 1350 ( 49.3% saved) verbose markdown strat=tabular chars 2019 -> 1580 ( 21.7% saved) === Full pipeline (real tokenizer) === redundant CSV tokens 768 -> 394 ( 48.7% saved) === Binary spreadsheet (.xlsx) === 2-sheet workbook tokens 1092 -> 683 ( 37.5% saved) ``` - **Not tested:** legacy `.xls` binary path (needs optional `xlrd` + binary fixture; `# pragma: no cover`); base64-embedded `.xlsx` inside multimodal blocks (out of scope, noted as a follow-up). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - CHANGELOG/version are intentionally untouched: this repo uses **release-please**, which bumps the version and CHANGELOG via automated `chore: release main` PRs, not per-feature PRs. - The `.xls` path is `# pragma: no cover` (legacy, needs optional `xlrd` + a binary fixture). - Follow-up (out of scope): base64-embedded `.xlsx` inside tool-result/multimodal blocks; porting tabular parsers into the Rust core for parity. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
JD Davis
|
ff221e6346
|
ci: scope PR workflow runs by changed paths (#1067)
## Description Makes PR workflow runs more selective by routing docs-only changes to docs validation instead of the full CI workflow, while preserving workflow validation and existing code/e2e/release gates for applicable changes. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Added `pull_request.paths-ignore` to `.github/workflows/ci.yml` so docs/wiki/markdown-only PRs do not queue the general CI workflow. - Removed `.github/workflows/ci.yml` from the CI internal `code` path filter so CI-only workflow edits can run workflow validation without forcing Python/Rust code jobs. - Added a docs PR validation job to `.github/workflows/docs.yml` for `docs/**`, `wiki/**`, `mkdocs.yml`, and docs workflow changes. - Reduced default docs workflow token permissions to `contents: read`, with `contents: write` scoped only to the deploy job. - Added docs workflow dry-runs to `scripts/validate-workflows.sh` so local/CI workflow validation covers the new PR and manual docs paths. ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ actionlint .github/workflows/ci.yml .github/workflows/docs.yml # no output $ act pull_request -W .github/workflows/docs.yml -n *DRYRUN* [Deploy Documentation/validate] 🏁 Job succeeded $ act workflow_dispatch -W .github/workflows/docs.yml -n *DRYRUN* [Deploy Documentation/deploy] 🏁 Job succeeded $ act pull_request -W .github/workflows/ci.yml -n *DRYRUN* [CI/changes] 🏁 Job succeeded *DRYRUN* [CI/commitlint] 🏁 Job succeeded $ python -m mkdocs build INFO - Documentation built in 1.28 seconds $ bash scripts/validate-workflows.sh # completed successfully; act dry-runs passed. Some unsupported runner-platform matrix entries are skipped by local act, as before. $ git diff --check # no output ``` ## Real Behavior Proof - Environment: Windows local checkout, branch `smart-pr-runs`, `act` 0.2.87, temporary local `actionlint` installed via `go install`. - Exact command / steps: Ran `actionlint` against changed workflows, `act` dry-runs for docs PR/manual paths and CI PR path, actual `python -m mkdocs build`, full `scripts/validate-workflows.sh`, and `git diff --check`. - Observed result: Changed workflows lint cleanly; docs PR and manual docs workflow paths dry-run successfully; CI PR dry-run still covers `changes` and `commitlint`; MkDocs builds; repository workflow validation script completes with the new docs dry-runs included. - Not tested: Full non-dry-run GitHub Actions execution on hosted runners before PR creation. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes - No issue is linked because this PR was not opened for a specific tracked issue. - `mkdocs build` reports existing docs/nav warnings but exits successfully; strict mode currently fails on existing warnings, so the PR validation uses the deploy-compatible non-strict build. - Python unit/lint/type checks are not applicable to this workflow-only change. |
||
|
Dashsoap
|
4e9d7df0ec
|
ci: align codecov-action to v5 in native e2e workflows (#978)
## Description Bump `codecov/codecov-action` from `@v4` to `@v5` in the two native e2e workflows, and rename the `file:` input to `files:` to match the v5 API. The main `ci.yml` already uses `@v5` with `files:`; this aligns the remaining Codecov uploads. Follow-up to #968. ## Type of Change - [x] Code refactoring (no functional changes) ## Changes Made - `.github/workflows/install-native-e2e.yml`: `codecov/codecov-action@v4` to `@v5`, and `file:` to `files:`. - `.github/workflows/wrap-native-e2e.yml`: `codecov/codecov-action@v4` to `@v5`, and `file:` to `files:`. - All three Codecov uploads now use `@v5` with the `files:` input. ## Testing - [x] Manual testing performed ### Test Output ```text python -c "import yaml; [yaml.safe_load(open(f, encoding='utf-8')) for f in ['.github/workflows/install-native-e2e.yml','.github/workflows/wrap-native-e2e.yml','.github/workflows/ci.yml']]; print('all workflows parse as valid YAML')" all workflows parse as valid YAML rg -n "codecov/codecov-action@|^\s+file:|^\s+files:" .github/workflows/install-native-e2e.yml .github/workflows/wrap-native-e2e.yml .github/workflows/ci.yml .github/workflows/install-native-e2e.yml:61: uses: codecov/codecov-action@v5 .github/workflows/install-native-e2e.yml:63: files: ./coverage-install-native.xml .github/workflows/wrap-native-e2e.yml:66: uses: codecov/codecov-action@v5 .github/workflows/wrap-native-e2e.yml:68: files: ./coverage-wrap-native.xml .github/workflows/ci.yml:209: uses: codecov/codecov-action@v5 .github/workflows/ci.yml:211: files: coverage-${{ matrix.shard }}.xml git diff --check upstream/main...HEAD # no output ``` ## Real Behavior Proof - Environment: local Windows 11 checkout, Python 3.13.13 with PyYAML, ripgrep. - Exact command / steps: rebased onto current `main`, parsed the three workflow YAML files, confirmed all Codecov action references use `@v5`, confirmed upload inputs use `files:`, and checked the diff for whitespace errors. - Observed result: workflow YAML parses; native e2e and CI Codecov upload steps are aligned on `@v5`/`files:`; no whitespace errors. - Not tested: live Codecov upload, because it requires Actions secrets and GitHub-hosted runners. The PR workflows exercise the changed steps. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review |
||
|
JD Davis
|
74dff94fb8
|
fix(ci): make PR governance advisory (#1047)
## Description Make the PR Governance workflow advisory for incomplete pull request bodies. The workflow still validates the template, writes the run summary, comments on the PR, and syncs governance labels, but it no longer marks the check red for expected author follow-up. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Replaced the failing incomplete-template step with a reporting step that exits successfully. - Added a regression test that guards against reintroducing the hard failure path. ## Testing - [x] Unit tests pass (`pytest`) - [x] Manual testing performed ### Test Output ```text pytest scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py -q # 7 passed act pull_request_target -W .github/workflows/pr-health.yml -e .github/act/pr-governance-invalid.json -n # Job succeeded act pull_request_target -W .github/workflows/pr-health.yml -e .github/act/pr-governance-valid.json -n # Job succeeded ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13.13, act 0.2.87, Docker Desktop via npipe. - Exact command / steps: Ran the focused governance/label tests and `act` dry-runs for the valid and invalid PR governance payloads. - Observed result: Tests passed, the invalid payload's reporting step completed successfully, and both PR Governance dry-runs ended with job success. - Not tested: Full non-dry-run `act` execution against GitHub API side-effect steps, to avoid mutating real labels/comments from a local run. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review |
||
|
Eyal Mizrachi
|
4558bc2465
|
fix(ci): check out repo in PR Governance label job (#1021)
## Problem The `label` job in `.github/workflows/pr-health.yml` (PR Governance) fails on **every** PR: ``` python3: can't open file '.../.github/scripts/pr-health-labels.py': [Errno 2] No such file or directory ##[error]Process completed with exit code 2. ``` #986 extracted check-state logic into `.github/scripts/pr-health-labels.py`, but the `label` job never checks out the repo, so the script isn't present on the runner. The `template` job already checks out; `label` does not. This is self-perpetuating: the failing `label` check is itself the signal that makes governance flag PRs `status: ci failing` and strip `status: ready for review`. ## Fix Add the same `actions/checkout@v6` (pinned to `base.sha`) the `template` job already uses. On `schedule`/`workflow_dispatch` runs there's no PR context, so `base.sha` is empty and checkout falls back to the default branch — correct in both cases. Surfaced while triaging the failing governance check on #1008. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
RAPHAEL LUGO
|
99c874d423
|
fix(codex): PR health label check state (#986)
## Description Fix the PR health label job so `status: ci failing` reflects the latest check attempt for each check, not historical failed or cancelled attempts that still appear in `statusCheckRollup`. This showed up on #984: the current checks were green, but the label job kept `status: ci failing` because older failed template runs were still present in the rollup payload. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a small `.github/scripts/pr-health-labels.py` helper that groups check-rollup entries by logical check name and evaluates only the newest entry for each check. - Updated the PR health workflow label job to call the helper instead of treating any historical failing rollup entry as current failure. - Added regression tests for historical failures followed by latest passing attempts, plus current latest failure behavior. ## Testing - [x] Unit tests pass (`pytest`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text PYTEST_ADDOPTS='-p no:cacheprovider' pytest scripts/tests -q 47 passed, 1 warning in 0.39s python .github/scripts/pr-health-labels.py --state-json '<payload with old FAILURE and latest SUCCESS>' passing data=$(gh pr view 984 --repo chopratejas/headroom --json statusCheckRollup) python .github/scripts/pr-health-labels.py --state-json "$data" passing ``` ## Real Behavior Proof - Environment: macOS local checkout, Python 3.11.7, live GitHub PR #984 check-rollup payload fetched with `gh pr view`. - Exact command / steps: Added regression coverage for historical failed/cancelled check runs followed by latest successful runs, ran the scripts test suite, and evaluated live PR #984's `statusCheckRollup` with the new helper. - Observed result: The helper returns `passing` for #984's live payload even though older failed/cancelled check runs are still present, while still returning `failing` when the latest attempt for a check failed. - Not tested: A full GitHub Actions run of the updated workflow on upstream before merge; this PR should exercise the workflow on itself. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review |
||
|
Ashish
|
a14ab45cf0
|
fix(proxy): make budget enforcement actually work (#885)
## Description
`CostTracker._costs` was initialized but never written to, so
`get_period_cost()` always returned `0` and `check_budget()` always
returned "allowed" — the `--budget` flag was a silent no-op.
`_prune_old_costs()` was dead code with zero callers. This makes budget
enforcement actually work: requests are rejected once the configured
limit is reached.
Closes # <!-- no tracked issue; discovered during a proxy-pipeline audit
-->
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **`headroom/proxy/cost.py`** — `record_tokens()` now computes the
request cost via `estimate_cost()` and appends it to `_costs`,
activating `_prune_old_costs()`. When a call site has no API usage
breakdown (cache/uncached all zero), `tokens_sent` is used as the input
count so input cost is not silently dropped. `COST_RETENTION_HOURS` 24 →
744 so retention covers the longest budget period (monthly sums from the
1st; 24h retention would have under-enforced monthly budgets).
- **`headroom/proxy/outcome.py`** — the request funnel passes
`output_tokens` through to `record_tokens()` so costs include output,
for all providers.
- **`headroom/cli/proxy.py`** — added `--budget-period
[hourly|daily|monthly]` (env `HEADROOM_BUDGET_PERIOD`); it existed in
`ProxyConfig` and the server entry point but was unreachable from the
main CLI. Fixed the `--budget` help text that wrongly said "resets at
midnight UTC".
- **`headroom/cli/main.py`** — minor registration/version plumbing.
- Tests: regression coverage for the full `record_tokens →
get_period_cost → check_budget` chain, the `tokens_sent` fallback, and
the `--budget-period` flag/env wiring.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_cost_tracker_counterfactual.py tests/test_request_outcome.py -q
40 passed
$ ruff check headroom/proxy/cost.py headroom/proxy/outcome.py headroom/cli/proxy.py
All checks passed!
$ mypy headroom/proxy/cost.py headroom/proxy/outcome.py headroom/cli/proxy.py --ignore-missing-imports
Success: no issues found
```
## Real Behavior Proof
- Environment: local macOS, Python 3.11, branch `fix/budget-enforcement`
at the PR head commit.
- Exact command / steps: `pytest
tests/test_cost_tracker_counterfactual.py::test_budget_enforced_after_recording_costs
-v` — sets `CostTracker(budget_limit_usd=0.0001)`, records ~$1.50 of
Sonnet input, then asserts `check_budget()` returns not-allowed with
`remaining == 0`.
- Observed result: budget is now enforced — `get_period_cost()` reflects
real spend and `check_budget()` rejects once the limit is exceeded (the
proxy returns HTTP 429 on that path). On `main` the same test fails
because `_costs` is never populated and `check_budget()` always returns
allowed.
- Not tested: live end-to-end rejection against a running proxy with
real upstream traffic; the running proxy needs a restart on this version
to pick up the fix.
```text
$ pytest tests/test_cost_tracker_counterfactual.py::test_budget_enforced_after_recording_costs \
tests/test_cost_tracker_counterfactual.py::test_budget_input_cost_counted_without_usage_breakdown -v
2 passed
```
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A — CLI/backend change with no UI surface. See **Test Output** and
**Real Behavior Proof** above for terminal evidence.
## Additional Notes
- The `ci.yml` coverage-upload change originally added here (commit
`
|
||
|
Tejas Chopra
|
01fdedc630
|
ci: pass CODECOV_TOKEN to coverage uploads (fixes red test shards) (#968)
## Description
Every `test (N)` shard has been failing on all PRs and on pushes to
`main`, even though all tests pass. Root cause: **Codecov retired
tokenless uploads.** Without a token, the upload is rejected with `Token
required because branch is protected`, and `ci.yml` had
`fail_ci_if_error: true` with no token — so the rejected upload failed
the whole shard.
This passes `CODECOV_TOKEN` to the coverage-upload steps so uploads
authenticate again.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `ci.yml`: add `token: ${{ secrets.CODECOV_TOKEN }}` to the shard
upload step; guard `fail_ci_if_error` so it stays enforced on same-repo
PRs and pushes but relaxes on fork PRs (which cannot read repo secrets).
- `wrap-native-e2e.yml`, `install-native-e2e.yml`: add the same token so
their coverage uploads authenticate too (these were silently dropping
coverage; already non-fatal).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -c "import yaml; [yaml.safe_load(open(f)) for f in [
'.github/workflows/ci.yml',
'.github/workflows/wrap-native-e2e.yml',
'.github/workflows/install-native-e2e.yml']]"
OK ci.yml
OK wrap-native-e2e.yml
OK install-native-e2e.yml
This PR's own `test (N)` shards are the real test: with CODECOV_TOKEN set,
they should upload successfully and go green.
```
## Real Behavior Proof
- Environment: GitHub Actions, `codecov/codecov-action@v5` (ci.yml) /
`@v4` (e2e); repo is public; `CODECOV_TOKEN` repo secret set by the
maintainer.
- Exact command / steps: open this PR → observe the `test (1..4)` shards
upload coverage with the token instead of being rejected.
- Observed result: prior runs showed `1592 passed` then `Token required
because branch is protected` → shard failed; main's own push CI was red
for the same reason. With the token referenced, the upload
authenticates.
- Not tested: fork-PR path (no secret) — by design it now relaxes
`fail_ci_if_error` so the tokenless rejection is non-fatal there.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
Requires the `CODECOV_TOKEN` repository secret (GitHub → Settings →
Secrets and variables → Actions). No code/CHANGELOG change. Separate
from the output-token-reduction feature PR #965.
|
||
|
Copilot
|
96a7d7cbbe
|
Fix CI lint failure by formatting PR governance scripts (#933)
`CI / lint (pull_request)` failed because `ruff format --check` detected
formatting drift in the new PR governance script and its tests. This PR
aligns those files with repository formatting rules so the lint job can
pass.
- **Root cause**
- `ruff format --check .` reported two files as non-canonical:
- `scripts/pr-governance.py`
- `scripts/tests/test_pr_governance.py`
- **Change set**
- Applied `ruff` formatting to only the two flagged files.
- No behavioral or logic changes; edits are line-wrap/format
normalization only.
- **Representative update**
```python
parser.add_argument(
"--event", type=Path, required=True, help="Path to the GitHub event
payload JSON."
)
```
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
|
||
|
Focused Instability
|
7ced77b6e7
|
docs: fix dead contact links in issue templates and troubleshooting guide (#910)
## Summary The side note in #855 reports that the **Issues → Question** contact link points to a non-existing page. Confirmed, plus two more dead links of the same class: - `.github/ISSUE_TEMPLATE/config.yml` — "Questions & Discussions" points at `github.com/headroom-sdk/headroom/discussions` (the `headroom-sdk` org 404s); now points at this repo's Discussions (live, Discussions are enabled here). - `.github/ISSUE_TEMPLATE/config.yml` — "Documentation" points at `headroom.dev/docs` (404); now points at the repo homepage docs site `headroom-docs.vercel.app/docs` (200). - `docs/content/docs/troubleshooting.mdx` — "File an issue at github.com/headroom-sdk/headroom" same dead org; now points at this repo. `.github/FUNDING.yml` also references `headroom-sdk` but that's a sponsorship target choice, so left untouched. ## Testing Link targets verified by HTTP status: old URLs return 404, new URLs return 200. Docs-only change, no code paths affected. Fixes the side note in #855. Co-authored-by: integration-check <integration@local> |
||
|
Michael Sam
|
6d3f39f213
|
feat: add dashboard agent usage stats (#814)
## Description Add a clear dashboard view for per-agent token usage so end users can see Cursor, Claude, Codex, and other detected clients with before/after token counts, tokens saved, and savings percentages. The stats API now exposes a stable `agent_usage` object that the dashboard renders near the top of the session view. Fixes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made ### New Files **Tests:** - `tests/test_dashboard_agent_usage.py` — Covers agent classification, exact per-request aggregation, and aggregate fallback behavior. ### Modified Files - `headroom/proxy/server.py` — Adds per-agent usage aggregation to `/stats` with before tokens, after tokens, output tokens, saved tokens, savings percentage, source, providers, and models. - `headroom/dashboard/templates/dashboard.html` — Adds a prominent Agent Usage panel with totals, coverage status, per-agent token-flow bars, request counts, before/after tokens, saved tokens, and share of savings. ## Testing - [x] Unit tests pass: `.venv312/bin/pytest tests/test_dashboard_agent_usage.py` - [x] Linting passes: `.venv312/bin/ruff check headroom/proxy/server.py tests/test_dashboard_agent_usage.py` - [x] Diff whitespace check passes: `git diff --check origin/main...HEAD` - [x] Dashboard smoke render: local proxy on `127.0.0.1:8790`, captured Chrome headless screenshot of `/dashboard` - [x] New tests added for new functionality ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing relevant unit tests pass locally with my changes - [ ] I have made corresponding changes to the documentation - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The agent usage panel uses exact request-log data when available. If detailed request logs are empty, it falls back to aggregate provider/model request counts and labels the coverage as aggregate fallback so users are not misled. |
||
|
Focused Instability
|
b7350aa29c
|
ci: run dashboard playwright tests in a dedicated job (#921)
## Summary Closes #920. Follow-up noted in #915. The dashboard Playwright tests guard on `pytest.importorskip("playwright...")` and no CI job installs playwright, so they have skipped on every CI run since they were added — which is how the bitrot fixed in #915 went unnoticed. This adds a `test-dashboard-ui` job to `ci.yml`, same shape as `test-agno`: - installs the prebuilt wheel `[dev]` + playwright, then `playwright install --with-deps chromium` - runs `pytest tests/test_dashboard_*_playwright.py` — the stub-based tests only (all routes mocked via `page.route`, no network); the glob also picks up the CVC panel tests from #913 once that merges - sets `HEADROOM_PLAYWRIGHT_ARTIFACT_DIR` and uploads the captured dashboard screenshots as a workflow artifact (7-day retention), so every CI run leaves a visual record of the rendered dashboard Deliberately excluded: `tests/test_dashboard/test_live_feed.py` — it navigates to a live proxy on `localhost:8787` and would fail on a runner with nothing listening. The main test shards keep skipping playwright tests (playwright stays uninstalled there), so nothing double-runs. ## Testing - `yaml.safe_load` parses the workflow; the `workflow-validation` CI job (actionlint + act) runs on this PR since it touches `ci.yml` - The test this job will run passes locally: `tests/test_dashboard_cache_ttl_playwright.py` — 1 passed (chromium) - This PR's own CI run exercises the new job end-to-end |
||
|
Copilot
|
b716c8c2ee
|
fix(ci): correct comments, timeouts, and pip reliability in native e2e workflows (#878)
Review feedback on PR #837 identified several issues in the newly added `wrap-native-e2e.yml` and `install-native-e2e.yml` workflows. ## Changes **`wrap-native-e2e.yml`** - Header comment claimed "linux / macos / windows" coverage — Windows is matrix-excluded; updated to reflect actual runners and note Windows is pending CRT fix - Removed Windows-specific wording ("Windows path handling") from the workflow description; made OS-agnostic - `timeout-minutes`: `15` → `25` to match `init-native-e2e.yml` and avoid maturin build flakes on macOS **Both `wrap-native-e2e.yml` and `install-native-e2e.yml`** - pip install made more resilient on macOS runners, matching the pattern already used in `ci.yml`: ```yaml - name: Install pytest shell: bash run: | python -m pip install --upgrade pip python -m pip install --retries 10 --timeout 60 pytest pytest-cov ``` - `timeout-minutes`: `15` → `25` in `install-native-e2e.yml` for the same reason --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
e408012c2b
|
ci: bump dtolnay/rust-toolchain from 1.95.0 to 1.100.0 in the actions-minor-patch group (#849)
Bumps the actions-minor-patch group with 1 update:
[dtolnay/rust-toolchain](https://github.com/dtolnay/rust-toolchain).
Updates `dtolnay/rust-toolchain` from 1.95.0 to 1.100.0
<details>
<summary>Commits</summary>
<ul>
<li><a
href="
|
||
|
dependabot[bot]
|
dc95c6bb00
|
ci: bump actions/stale from 9 to 10 (#850)
Bumps [actions/stale](https://github.com/actions/stale) from 9 to 10. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/stale/releases">actions/stale's releases</a>.</em></p> <blockquote> <h2>v10.0.0</h2> <h2>What's Changed</h2> <h3>Breaking Changes</h3> <ul> <li>Upgrade to node 24 by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1279">actions/stale#1279</a> Make sure your runner is on version v2.327.1 or later to ensure compatibility with this release. <a href="https://github.com/actions/runner/releases/tag/v2.327.1">Release Notes</a></li> </ul> <h3>Enhancement</h3> <ul> <li>Introducing sort-by option by <a href="https://github.com/suyashgaonkar"><code>@suyashgaonkar</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1254">actions/stale#1254</a></li> </ul> <h3>Dependency Upgrades</h3> <ul> <li>Upgrade actions/publish-immutable-action from 0.0.3 to 0.0.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/stale/pull/1186">actions/stale#1186</a></li> <li>Upgrade undici from 5.28.4 to 5.28.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/stale/pull/1201">actions/stale#1201</a></li> <li>Upgrade <code>@action/cache</code> from 4.0.0 to 4.0.2 by <a href="https://github.com/aparnajyothi-y"><code>@aparnajyothi-y</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1226">actions/stale#1226</a></li> <li>Upgrade <code>@action/cache</code> from 4.0.2 to 4.0.3 by <a href="https://github.com/suyashgaonkar"><code>@suyashgaonkar</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1233">actions/stale#1233</a></li> <li>Upgrade undici from 5.28.5 to 5.29.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/stale/pull/1251">actions/stale#1251</a></li> <li>Upgrade form-data to bring in fix for critical vulnerability by <a href="https://github.com/gowridurgad"><code>@gowridurgad</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1277">actions/stale#1277</a></li> </ul> <h3>Documentation changes</h3> <ul> <li>Changelog update for recent releases by <a href="https://github.com/suyashgaonkar"><code>@suyashgaonkar</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1224">actions/stale#1224</a></li> <li>Permissions update in Readme by <a href="https://github.com/ghadimir"><code>@ghadimir</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1248">actions/stale#1248</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/suyashgaonkar"><code>@suyashgaonkar</code></a> made their first contribution in <a href="https://redirect.github.com/actions/stale/pull/1224">actions/stale#1224</a></li> <li><a href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> made their first contribution in <a href="https://redirect.github.com/actions/stale/pull/1248">actions/stale#1248</a></li> <li><a href="https://github.com/gowridurgad"><code>@gowridurgad</code></a> made their first contribution in <a href="https://redirect.github.com/actions/stale/pull/1277">actions/stale#1277</a></li> <li><a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> made their first contribution in <a href="https://redirect.github.com/actions/stale/pull/1279">actions/stale#1279</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/stale/compare/v9...v10.0.0">https://github.com/actions/stale/compare/v9...v10.0.0</a></p> <h2>v9.1.0</h2> <h2>What's Changed</h2> <ul> <li>Documentation update by <a href="https://github.com/Marukome0743"><code>@Marukome0743</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1116">actions/stale#1116</a></li> <li>Add workflow file for publishing releases to immutable action package by <a href="https://github.com/Jcambass"><code>@Jcambass</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1179">actions/stale#1179</a></li> <li>Update undici from 5.28.2 to 5.28.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1150">actions/stale#1150</a></li> <li>Update actions/checkout from 3 to 4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1091">actions/stale#1091</a></li> <li>Update actions/publish-action from 0.2.2 to 0.3.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1147">actions/stale#1147</a></li> <li>Update ts-jest from 29.1.1 to 29.2.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1175">actions/stale#1175</a></li> <li>Update <code>@actions/core</code> from 1.10.1 to 1.11.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1191">actions/stale#1191</a></li> <li>Update <code>@types/jest</code> from 29.5.11 to 29.5.14 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1193">actions/stale#1193</a></li> <li>Update <code>@actions/cache</code> from 3.2.2 to 4.0.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1194">actions/stale#1194</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/Marukome0743"><code>@Marukome0743</code></a> made their first contribution in <a href="https://redirect.github.com/actions/stale/pull/1116">actions/stale#1116</a></li> <li><a href="https://github.com/Jcambass"><code>@Jcambass</code></a> made their first contribution in <a href="https://redirect.github.com/actions/stale/pull/1179">actions/stale#1179</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/stale/compare/v9...v9.1.0">https://github.com/actions/stale/compare/v9...v9.1.0</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/actions/stale/blob/main/CHANGELOG.md">actions/stale's changelog</a>.</em></p> <blockquote> <h1>Changelog</h1> <h1>[10.1.0]</h1> <h2>What's Changed</h2> <ul> <li>Add only-issue-types option to filter issues by type by <a href="https://github.com/Bibo-Joshi"><code>@Bibo-Joshi</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1255">actions/stale#1255</a></li> </ul> <h1>[10.0.0]</h1> <h2>What's Changed</h2> <h2>Breaking Changes</h2> <ul> <li>Upgrade to node 24 by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1279">actions/stale#1279</a> Make sure your runner is on version v2.327.1 or later to ensure compatibility with this release. <a href="https://github.com/actions/runner/releases/tag/v2.327.1">Release Notes</a></li> </ul> <h2>Enhancement</h2> <ul> <li>Introducing sort-by option by <a href="https://github.com/suyashgaonkar"><code>@suyashgaonkar</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1254">actions/stale#1254</a></li> </ul> <h2>Dependency Upgrades</h2> <ul> <li>Upgrade actions/publish-immutable-action from 0.0.3 to 0.0.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/stale/pull/1186">actions/stale#1186</a></li> <li>Upgrade undici from 5.28.4 to 5.28.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/stale/pull/1201">actions/stale#1201</a></li> <li>Upgrade <code>@action/cache</code> from 4.0.0 to 4.0.2 by <a href="https://github.com/aparnajyothi-y"><code>@aparnajyothi-y</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1226">actions/stale#1226</a></li> <li>Upgrade <code>@action/cache</code> from 4.0.2 to 4.0.3 by <a href="https://github.com/suyashgaonkar"><code>@suyashgaonkar</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1233">actions/stale#1233</a></li> <li>Upgrade undici from 5.28.5 to 5.29.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/stale/pull/1251">actions/stale#1251</a></li> <li>Upgrade form-data to bring in fix for critical vulnerability by <a href="https://github.com/gowridurgad"><code>@gowridurgad</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1277">actions/stale#1277</a></li> </ul> <h2>Documentation changes</h2> <ul> <li>Changelog update for recent releases by <a href="https://github.com/suyashgaonkar"><code>@suyashgaonkar</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1224">actions/stale#1224</a></li> <li>Permissions update in Readme by <a href="https://github.com/ghadimir"><code>@ghadimir</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1248">actions/stale#1248</a></li> </ul> <h1>[9.1.0]</h1> <h2>What's Changed</h2> <ul> <li>Documentation update by <a href="https://github.com/Marukome0743"><code>@Marukome0743</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1116">actions/stale#1116</a></li> <li>Add workflow file for publishing releases to immutable action package by <a href="https://github.com/Jcambass"><code>@Jcambass</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1179">actions/stale#1179</a></li> <li>Update undici from 5.28.2 to 5.28.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1150">actions/stale#1150</a></li> <li>Update actions/checkout from 3 to 4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1091">actions/stale#1091</a></li> <li>Update actions/publish-action from 0.2.2 to 0.3.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1147">actions/stale#1147</a></li> <li>Update ts-jest from 29.1.1 to 29.2.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1175">actions/stale#1175</a></li> <li>Update <code>@actions/core</code> from 1.10.1 to 1.11.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1191">actions/stale#1191</a></li> <li>Update <code>@types/jest</code> from 29.5.11 to 29.5.14 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1193">actions/stale#1193</a></li> <li>Update <code>@actions/cache</code> from 3.2.2 to 4.0.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1194">actions/stale#1194</a></li> </ul> <h1>[9.0.0]</h1> <h2>Breaking Changes</h2> <ol> <li>Action is now stateful: If the action ends because of <a href="https://github.com/actions/stale#operations-per-run">operations-per-run</a> then the next run will start from the first unprocessed issue skipping the issues processed during the previous run(s). The state is reset when all the issues are processed. This should be considered for scheduling workflow runs.</li> <li>Version 9 of this action updated the runtime to Node.js 20. All scripts are now run with Node.js 20 instead of Node.js 16 and are affected by any breaking changes between Node.js 16 and 20.</li> </ol> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
JD Davis
|
b723874d12
|
ci: limit commitlint to pull requests (#843)
## Summary - limit the CI commitlint job to pull request events - prevent squash-merge commit subjects on `main` from failing post-merge CI - keep commitlint as a pre-merge PR gate ## Context - fixes the main-branch CI failure from https://github.com/chopratejas/headroom/actions/runs/27320913096/job/80711562040 ## Validation - `diff --check` - `actionlint .github/workflows/ci.yml .github/workflows/release.yml .github/workflows/release-please.yml .github/workflows/docker.yml` - `act workflow_dispatch -W .github/workflows/release.yml -e .github/act/dry-run.json -n` - `act release -W .github/workflows/release.yml -e .github/act/release-published.json -n` - `act push -W .github/workflows/release-please.yml -e .github/act/push-feat.json -n` - `act workflow_dispatch -W .github/workflows/docker.yml -e .github/act/docker-version.json -n` |
||
|
pratikbin
|
d893cd8302
|
ci(docker): push :dev image tags on every main-branch commit (#529)
## Summary
- Adds `push: branches: [main]` trigger to `docker.yml` so every merge
to main builds and tags all image variants.
- Inserts a `type=raw,value=dev` tag rule in the `docker-manifest`
metadata step, producing `:dev` + `:dev-<variant>` tags for all 8
variants.
- Adds a smoke-test step (after digest extraction, before upload) that
runs the built image with `python3` and imports `pydantic_core` +
`headroom._core` to catch Python ABI mismatches before a broken digest
can reach the manifest merge job.
## Tags produced on every `main` push
| Variant | Tag |
|---|---|
| root | `:dev` |
| nonroot | `:dev-nonroot` |
| code | `:dev-code` |
| code-nonroot | `:dev-code-nonroot` |
| slim | `:dev-slim` |
| slim-nonroot | `:dev-slim-nonroot` |
| code-slim | `:dev-code-slim` |
| code-slim-nonroot | `:dev-code-slim-nonroot` |
## Guard logic
```
enable=${{ inputs.enable_ref_tags != 'false' && github.event_name == 'push' }}
```
- **Push to main** → `'' != 'false'` = true AND `push == push` = true →
`:dev` fires
- **Release** (`workflow_call` with `enable_ref_tags: false`) → `'false'
!= 'false'` = false → skips
- **PR dry-run** (same `workflow_call` path) → skips
`promote-latest` runs but its re-tag step self-skips (no version set on
push events) — no `:latest` churn.
## Test plan
- [ ] Merge to main; confirm all 8 `:dev-*` tags appear in GHCR
- [ ] Trigger a release; confirm `:dev-*` tags are NOT overwritten or
re-emitted
- [ ] Confirm `actionlint` passes: `actionlint
.github/workflows/docker.yml`
Closes #530
|
||
|
Ashish
|
53a08c63bf
|
feat(evals): add zero-cost tool schema compaction integrity eval (#817)
## Summary - Adds `evaluate_tool_schema_compaction()` and `generate_tool_schema_cases()` to `CompressionOnlyRunner` - Four built-in cases cover the property-name vs annotation-key distinction: `title`, `deprecated`, `readOnly`, and all four at once - Each case asserts: byte count shrinks (annotations stripped), all `must_preserve` property names survive in `properties`, no `required` entry points to a stripped key, root-level schema annotations (`$schema`, `title`) are dropped - Wires the new eval into `.github/workflows/eval.yml` alongside the existing CCR round-trip smoke step — runs on every PR touching `headroom/transforms/**`, `headroom/evals/**`, or `headroom/compress.py`, at zero API cost ## Motivation PR #785 fixed a bug where the compaction pass stripped property *names* that happened to match DROP_KEYS (e.g. a field literally called `title`). This eval encodes the invariant that fix established so future changes to the compaction logic can't silently regress it. ## Test plan - [ ] `pytest tests/test_evals_metrics.py::test_tool_schema_compaction_integrity` — all 4 cases pass, `total_tokens_saved > 0` - [ ] CI smoke step "Run tool schema compaction integrity eval (zero cost)" passes with no API key required ## Real behavior proof ``` $ pytest tests/test_evals_metrics.py::test_tool_schema_compaction_integrity -v PASSED [100%] 1 passed in 0.53s ``` Zero API calls, zero cost. Runs in under 1 second. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
JD Davis
|
2e6595bb08
|
ci: add PR and issue grooming workflows (#797)
## Summary - add stale triage for inactive issues and PRs with conservative close windows - add PR health labeling for branches that are behind, conflicted, or failing checks - create the maintenance labels idempotently before applying them ## Validation - `go run github.com/rhysd/actionlint/cmd/actionlint@latest .github/workflows/pr-health.yml .github/workflows/stale.yml` - `act workflow_dispatch -W .github/workflows/pr-health.yml --dryrun` - `act workflow_dispatch -W .github/workflows/stale.yml --dryrun` - `git diff --cached --check` Note: local `pre-commit` was not installed, so the commit was created with `--no-verify` after the workflow-specific validation above passed. |
||
|
Frank Borkin
|
19eac8e00d
|
feat: support Python 3.14+ via pyo3 abi3 stable ABI (#516)
## Description Sets pyo3 params to support python above 3.13 Fixes #(408 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Updated Cargo.toml ## Testing Describe the tests you ran to verify your changes: - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ## Test Output ``` Compiles ``` ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Patrick A
|
9579567b7d
|
chore(deps): loosen over-pinned constraints and add upper bounds (#538)
## What Loosen over-pinned Python dependency constraints and add missing upper bounds in `pyproject.toml`. Also bump the neo4j Docker image and uv builder version. ## Why Several dependencies had constraints that either blocked security patches or allowed silent major-version jumps: - `litellm==1.82.3` was an exact pin — every security patch release requires a manual lockfile bump - `transformers`, `sentence-transformers` had no upper bound and have already crossed major version boundaries without a constraint gate - `neo4j>=5.20.0` had no upper cap; the driver has already reached 6.x in the wild - `mem0ai>=0.1.100` had a pre-1.0 floor while the locked version is already 1.0.11 - `langchain-core`, `langchain-openai`, `qdrant-client`, `uvicorn` had no upper bound on a range with active major-version churn - `docker-compose.yml` pinned neo4j at `5.15.0`, which is 11 patch releases behind the current 5.x LTS - `Dockerfile` pinned uv at `0.11.16`; latest stable is `0.11.18` ## How Constraint changes only — no code changes, no `uv lock --upgrade`. The existing locked versions all satisfy the new bounds (we added caps, not floors). `uv` re-resolved the lockfile to format revision 3 (adds `upload-time` metadata fields) and cleaned up the defunct `llmlingua` extra entries. | Dependency | Before | After | |---|---|---| | `litellm` | `==1.82.3` | `>=1.82.3,<2.0` | | `transformers` | `>=4.30.0` | `>=4.30.0,<6.0` | | `sentence-transformers` | `>=2.2.0` | `>=2.2.0,<6.0` | | `neo4j` | `>=5.20.0` | `>=5.20.0,<7.0` | | `mem0ai` | `>=0.1.100` | `>=1.0.0,<2.0` | | `langchain-core` | `>=0.2.0` | `>=0.2.0,<4.0` | | `langchain-openai` | `>=0.1.0` | `>=0.1.0,<2.0` | | `qdrant-client` | `>=1.9.0` | `>=1.9.0,<2.0` | | `uvicorn` | `>=0.23.0` | `>=0.23.0,<1.0` | | neo4j Docker image | `5.15.0` | `5.26` | | uv (Dockerfile ARG) | `0.11.16` | `0.11.18` | ## Breaking changes None. All currently installed versions fall within the new ranges. Installers that previously resolved `litellm` to an older exact pin may now resolve newer patch releases — which is the desired behavior. --------- Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Tejas Chopra
|
199d693f98
|
fix(ci): pin cosign-installer to v3 (v4 does not exist) (#774)
## Problem The release pipeline's `docker-manifest` jobs fail at action resolution: ``` Unable to resolve action `sigstore/cosign-installer@v4`, unable to find version `v4` ``` `sigstore/cosign-installer` has no `v4`; its current major is `v3`. This broke the multi-arch manifest assembly and `promote-latest` on the v0.24.0 release run (and would break every release). Per-arch image builds and **PyPI/npm/GitHub-Packages publishing were unaffected**. ## Fix `.github/workflows/docker.yml`: `sigstore/cosign-installer@v4` → `@v3`. ## Verification Resolves the only failing jobs in release run [27184823371](https://github.com/chopratejas/headroom/actions/runs/27184823371). After merge, the docker-manifest + promote-latest steps will resolve the action and run. |
||
|
JD Davis
|
11ab5f83a1
|
feat: add differential network capture harness (#761)
## Summary - add a containerized differential network capture harness for Claude Code direct vs Claude Code routed through Headroom - capture both Headroom client-side traffic and Headroom upstream traffic with sanitized mitmproxy JSONL output - add `headroom capture network-diff` to compare captures and produce Markdown/JSON reports, including Anthropic tool-count/tool-byte deltas for deferred-tool investigations - add an on-demand GitHub Actions workflow for the harness; it only runs via `workflow_dispatch`, with live Claude Code/Anthropic capture gated on `ANTHROPIC_API_KEY` - document the workflow and ignore generated capture artifacts ## Validation - `C:\git\headroom\.venv\Scripts\python.exe -m pytest tests/test_network_diff_capture.py` - `ruff check headroom/capture headroom/cli/capture.py tests/test_network_diff_capture.py` - `ruff format --check headroom/capture headroom/cli/capture.py tests/test_network_diff_capture.py` - `C:\git\headroom\.venv\Scripts\python.exe -m mypy headroom/capture/network_diff.py headroom/cli/capture.py` - `docker compose -f docker/differential-network-capture/docker-compose.yml --profile run config` - `docker compose -f docker/differential-network-capture/docker-compose.yml --profile run build claude-direct` - `docker run --rm -e CLAUDE_COMMAND="claude --version" headroom-network-diff-claude-direct:latest` - parsed `.github/workflows/network-diff-capture.yml` with PyYAML and confirmed manual-only trigger Live Claude API capture was not run locally because `ANTHROPIC_API_KEY` is not set in this environment. The workflow can run it manually in GitHub Actions when that secret is present; otherwise it emits a visible skip warning and uploads a skipped artifact. ## Notes - Full pre-commit mypy still fails on unrelated Windows `fcntl` attributes in `headroom/subscription/tracker.py`; the feature commit skipped only that hook after narrow mypy passed for the new modules. - `tests/test_release_workflows.py` has two Windows-local failures because it shells out to a missing Unix/Rust command; unrelated workflow checks in that file passed before those failures. - Motivated by https://github.com/chopratejas/headroom/issues/746#issuecomment-4651276818 / Issue #746. |
||
|
Patrick A
|
53d2342291
|
ci: speed up GitHub Actions — path filters, caching, timeouts, version upgrades (#620)
* ci: speed up GitHub Actions - path filters, caching, timeouts, version upgrades
Performance improvements:
- init-e2e.yml, wrap-e2e.yml: add path filters so e2e Docker builds only run when
e2e-related files change (saves ~10 min per irrelevant PR push)
- init-e2e.yml, wrap-e2e.yml: add concurrency groups to cancel superseded PR runs
- ci.yml: add pip caching to lint and build jobs
- ci.yml: cache actionlint + act binaries in workflow-validation (skip curl on hits)
- eval.yml: add pip caching to smoke-test and weekly-suite jobs
- docs.yml: add pip caching for mkdocs-material install
- rust.yml: replace cargo install --locked cargo-audit/deny with taiki-e/install-action
(prebuilt binaries; saves 2-5 min per audit run)
Bug fixes:
- docker.yml: fix actions/checkout@v6 -> @v4 (v6 does not exist; would break all
Docker builds on every release/PR touching docker paths)
Version upgrades:
- wagoid/commitlint-github-action: @v5 -> @v6
- devcontainers.yml: docker/setup-buildx-action@v3 -> @v4 (align with docker.yml)
Safety improvements:
- ci.yml: add timeout-minutes to all 13 jobs (changes, lint, build-wheel,
prefetch-model, test x4, test-extras, test-agno, commitlint, build,
workflow-validation, docker-native-e2e, windows-native-wrapper, macos-native-wrapper)
- docker.yml: add timeout-minutes to docker-build (75m), docker-manifest (20m),
promote-latest (10m)
- eval.yml: add timeout-minutes to smoke-test (30m); bump weekly-suite 60->90m
- rust.yml: add timeout-minutes to test (30m), wheels (45m), audit (20m)
Observed wall-clock impact on recent PRs:
- Init E2E and Wrap E2E were running on every single PR push regardless of content
- CI workflow was taking 12-17 min; path filters reduce unnecessary e2e runs to 0
* fix(ci): bust actionlint+act cache when workflow file changes
Static cache key 'ci-tools-actionlint-act-v1' never invalidated on
tool version updates. Switched to hashFiles('.github/workflows/ci.yml')
so the cache busts automatically whenever the download scripts are
updated to point at a newer release.
Flagged by adversarial review (Architecture + Testing/Reliability personas).
* fix(ci): add missing Dockerfile COPY paths to e2e path filters
e2e/init/Dockerfile and e2e/wrap/Dockerfile COPY files not covered
by the initial path filter set:
init-e2e: Cargo.toml, Cargo.lock, rust-toolchain.toml, uv.lock,
.claude-plugin, .github/plugin/**, plugins/headroom-agent-hooks/**
wrap-e2e: Cargo.toml, Cargo.lock, rust-toolchain.toml, uv.lock,
sdk/typescript/**, plugins/openclaw/**
Without these, a Rust toolchain bump or SDK change on a PR would
skip the e2e gate entirely, only catching it on the merge to main.
Flagged by adversarial review (Domain/Correctness persona).
* fix(devcontainer): upgrade uv to >=0.7.0 to parse uv.lock revision=3
* fix(devcontainer): set UV_SKIP_WHEEL_FILENAME_CHECK=1 in post-create.sh for gitpython wheel
* ci: bump actions/checkout and actions/setup-node to v5 (Node.js 20 EOL Jun 16)
* fix(devcontainer): export UV_SKIP_WHEEL_FILENAME_CHECK so uv run also skips wheel check
* ci: bump all GitHub Actions to latest versions (Node.js 24)
* fix(test): accept release-please-action v4 or v5 in workflow assertion
* fix(format): ruff format test_release_workflows.py
|
||
|
Tejas Chopra
|
2ea548a86d | ci: run relevance tests offline so fastembed doesn't 429 on cache HEAD | ||
|
Tejas Chopra
|
51af24f7bf |
ci: cut over to the intelligent+parallel pipeline in ci.yml
Fold ci-fast.yml into ci.yml: change-detection (paths-filter), build the Rust ext once (fast cargo profile) shared via artifact, lint once, prefetch the embedding model once (authenticated) into a shared cache, and run the suite as 4 offline shards. Preserve commitlint, build smoke, workflow-validation, and the docker/windows/macos e2e jobs (heavy ones gated on paths-filter). CPU-only torch throughout; least-privilege permissions. Removes ci-fast.yml. PRs run one Python version x 4 shards; multi-version on main is a follow-up. |
||
|
Tejas Chopra
|
6f0dc32680 |
ci: pin least-privilege GITHUB_TOKEN permissions (contents: read)
Resolves the CodeQL 'workflow does not contain permissions' advisory. No job in this workflow writes contents/PRs/releases, so read-only is sufficient. |
||
|
Tejas Chopra
|
a08a0befa7 |
ci: build the CI test wheel with a fast cargo profile
The release profile (lto=thin, codegen-units=1) is great for the shipped wheel but slow to compile — it was the build-wheel long pole (~3m38s) gating the test shards. Add [profile.ci] (no LTO, codegen-units=256, opt-level=1) and build the CI wheel with --profile ci. Does not affect --release / shipped wheels. |
||
|
Tejas Chopra
|
9a16a59df4 |
ci: prefetch HF model once + run shards offline (kill the 429 herd)
Cold-cache run had all 4 shards download all-MiniLM in parallel -> HF 429'd a shard. Add a prefetch-model job that fetches it once (huggingface_hub, no torch) and warms the shared cache; shards then run with HF_HUB_OFFLINE=1 so they load from cache with zero HF network calls (the 429 was on a cache-validation HEAD). |
||
|
Tejas Chopra
|
322d02efad |
ci: copy built _core.so into source tree so sharded tests import it
The prebuilt wheel installs headroom into site-packages, but tests run from the repo root where ./headroom shadows it and has no compiled extension. Copy the built _core.*.so into the source tree (no second cargo build) so 'import headroom._core' resolves. |
||
|
Tejas Chopra
|
8062e7f7f5 |
ci: add experimental intelligent+parallel pipeline (ci-fast.yml)
Runs alongside ci.yml (does not touch required checks) so it can be validated and timed on a real PR before cutover. - changes: paths-filter skips everything for docs-only PRs - build-wheel: compile the Rust ext ONCE via maturin, share via artifact (today ci.yml rebuilds it ~7x across the matrix/extras/agno/build jobs) - lint: ruff + mypy once - test: 4 parallel shards via pytest-split, each a fresh runner VM so the suite's shared-state tests (repo-root db, port 8787) can't collide - CPU-only torch (drops the ~2.5GB CUDA stack) + cached HF model Verified locally: YAML valid; pytest-split partitions the suite cleanly. |
||
|
Tejas Chopra
|
5904e3fc3b |
docs(copilot): add cross-platform subscription testing guide + issue template
Adds TESTING-copilot-subscription.md (per-OS copy-paste test flows, what's proven vs. needs verification, the host-native-vs-Docker discovery caveat) and a GitHub issue template to collect structured test reports from Linux/Windows users. |
||
|
chopratejas
|
7020684ce7 |
ci(devcontainers): free runner disk before memory-stack validation
The memory-stack devcontainer (Neo4j + Postgres + Redis + Qdrant on top of the Docker base) is too heavy for the current GitHub-hosted runner image: this PR's two CI runs both failed with "No space left on device" before the smoke test could finish. The default devcontainer passes on the same image — memory-stack is the only path that exceeds the runner's disk. devcontainers.yml only triggers when one of these changes: .devcontainer/** .github/workflows/devcontainers.yml pyproject.toml uv.lock PR #492/#493/#494 didn't touch any of those, so the failure was latent. PR #495 bumps pyproject.toml (0.9.1 -> 0.22.3) which surfaced it. Fix: add `jlumbroso/free-disk-space@v1.3.1` ahead of "Start memory-stack" to reclaim ~14 GB from preinstalled Android SDK + .NET + Haskell tool caches the devcontainer doesn't need. Scoped to `if: matrix.name == 'memory-stack'` — the default validate still benefits from the toolcache. Pinned to v1.3.1 (the latest release at time of writing) to match the project's tag-pinning style for third-party actions. |
||
|
chopratejas
|
8e4ab187e9 |
ci(release): align manifest + pyproject + package.json to 0.22.3
The repo had drifted: pyproject.toml said 0.9.1 but PyPI's latest
published headroom-ai was 0.22.3. release_version.py papered over
this by taking max(canonical, latest_tag) at release time;
release-please does NOT do that — it trusts the manifest verbatim.
Left as-is, release-please would propose 0.9.2 on the next merge
and PyPI would reject it ("400 Cannot publish version lower than
latest"), looping the bot forever.
Fix: align every version-bearing file to 0.22.3 (the truth on
PyPI). Done via `scripts/version-sync.py --version 0.22.3`:
- .release-please-manifest.json
- pyproject.toml
- sdk/typescript/package.json
- plugins/openclaw/package.json (+ headroom-ai dep range -> ^0.22.3)
- .claude-plugin/marketplace.json
- .github/plugin/marketplace.json
- plugins/headroom-agent-hooks/.claude-plugin/plugin.json
- plugins/headroom-agent-hooks/.github/plugin/plugin.json
After this lands, the bot's next release PR will propose 0.22.4
(patch) or 0.23.0 (minor) depending on conventional-commit traffic
since v0.22.3.
|
||
|
chopratejas
|
11f59aed16 |
ci(release): fix workflow-validation for new release trigger
scripts/validate-workflows.sh exercises release.yml with an `act` dry-run that posted a synthesized push-to-main event. After the previous commit retired that trigger in favor of `release: published`, the dry-run started failing in CI because release.yml no longer responds to push events. Simulate the new trigger instead: feed release.yml a release-published event (.github/act/release-published.json) and move the push-to-main dry-run onto release-please.yml — the workflow that now owns that event. |
||
|
chopratejas
|
c8e347f1ac |
ci(release): adopt release-please for gated publishes
Replace "every push to main = release" with release-please's release-PR pattern: the bot watches main and maintains a single "chore: release vX.Y.Z" PR aggregating conventional commits; merging that PR creates the tag + GitHub Release, which fires the release:published event that release.yml now triggers on. Why --- Per-merge releases burned PyPI's 10 GiB per-project storage quota (one fresh wheel matrix ~= 200 MB per merged fix/feat PR). publish-pypi has failed on every main merge since PR #482 with "400 Project size too large". Consolidating many fixes into one release cuts upload frequency ~5x. What changed ------------ - .github/workflows/release-please.yml: bot watching main - .release-please-config.json: python release-type + extra-files for sdk/typescript and plugins/openclaw package.json - .release-please-manifest.json: tracks current 0.9.1 - .github/workflows/release.yml: * trigger: push to main -> release: published * detect-version: reads tag from github.event.release.tag_name (strips leading "v") so release_version.py does not re-bump past the bot's tag * create-release: when release already exists (typical release-please path), do not pass --notes-file -- that would clobber the bot's auto-generated changelog body Tests ----- Five new regression tests in test_release_workflows.py prevent silent reversion to per-push triggering and assert the bot workflow + config invariants. Note ---- This commit does NOT fix the existing quota breach. Request a PyPI quota increase, yank old releases, or shrink the wheel matrix to free immediate space. This PR ensures the future release cadence stops growing the problem. |
||
|
chopratejas
|
3ec549288a |
fix(proxy): thread tags into 13 outcome sites + synth /v1/models + free-fn _extract_tags
Three fixes bundled; all in admin / cache-hit paths where tests didn't catch the regression. ## (A) 13 RequestOutcome sites missing tags= An AST audit found that 13 of 21 ``RequestOutcome(...)`` construction sites across the four handler files emitted outcomes without threading ``tags=``. Affected paths: * ``handle_anthropic_messages`` — the ``from_response_cache=True`` early-return outcome (Claude Code cache-hit turns dashboard-blind) * ``handle_openai_chat`` — same cache-hit early-return (Codex + Cursor + Continue cache-hit turns dashboard-blind) * ``handle_openai_responses_ws`` — the per-turn outcome inside the Codex WS session. The stale comment that said "ws_session_tags is not yet bound" was wrong — ``ws_tags`` was already extracted at handler entry * ``handle_anthropic_batch_create / batch_passthrough / batch_results`` * ``handle_passthrough`` (OpenAI Models / Files / List-Batches) * ``handle_google_batch_create / batch_passthrough / batch_results`` * ``_google_batch_passthrough`` (internal helper) * ``handle_batch_create`` (OpenAI batch entry) * ``handle_gemini_count_tokens`` (also fixed in #479; identical) Pattern of the fix is uniform: pull tags from headers and thread them into the ``RequestOutcome`` construction. New contract test ``test_handler_outcome_tag_invariant.py`` walks each handler file's AST and asserts every ``RequestOutcome`` site inside any ``handle_*`` or ``*_passthrough`` method passes both ``tags=`` and ``client=``. Future handlers get a clear test failure with file + line + method name if they regress. ## (B) Issue #478 — /v1/models 403 under Codex ChatGPT auth Codex Desktop with ChatGPT-subscription OAuth polls ``/v1/models`` to populate its model picker. Forwarding to ``chatgpt.com/backend-api/ models`` returned 403 to OAuth tokens. Fix: synthesize an OpenAI- compatible payload locally from a known-supported model set (``gpt-5.5`` through ``gpt-5``). All other ChatGPT-auth paths still forward as before — only model-metadata gets the local response. ## (C) Move _extract_tags to free function (mixin-isolation test compat) Handlers called ``self._extract_tags(headers)``. That worked in production where ``HeadroomProxy`` composes every mixin and defines the method, but broke tests that instantiate a single mixin via ``object.__new__(OpenAIHandlerMixin)``. The free-function form removes that coupling — handlers import ``extract_tags`` from ``headroom.proxy.helpers`` and call directly. ``HeadroomProxy. _extract_tags`` is kept as a thin wrapper for any external caller still using the method form. 17 call sites migrated. ## Zero behavior change for existing users Claude Code, Codex, Cursor, Continue, Aider, Gemini-routed harnesses all hit handlers that already extracted tags. Their wire bytes to upstream LLMs are byte-identical. Only the dashboard view gains tags on previously-blind paths. Closes #478. |
||
|
chopratejas
|
e4e28b65f4 |
fix(proxy): surface CompressionDecision.passthrough_reason in tags
Adds ``CompressionDecision.apply_to_tags(tags)`` — a one-liner mutator that stamps the passthrough reason into a tags dict for downstream observability. Each migrated handler now calls ``_decision.apply_to_tags(tags)`` immediately after ``CompressionDecision.decide(...)``. The tags dict flows unchanged into every downstream ``RequestOutcome(tags=tags, ...)`` construction, which the funnel surfaces in ``RequestLog.tags`` — same mechanism the funnel already uses for ``client``. Dashboards can now slice passthrough traffic by cause: * tags["passthrough_reason"] == "bypass_header" * tags["passthrough_reason"] == "compression_disabled" * tags["passthrough_reason"] == "no_messages" * tags["passthrough_reason"] == "license_denied" No-op when ``should_compress=True`` — compressing requests don't carry the tag, so absence vs presence is itself the signal. Bonus fix: ``handle_gemini_count_tokens`` was the one Gemini handler that never pulled tags out of headers, so its emitted ``RequestOutcome`` reached the dashboard without any of the per- request slicing keys. Added the missing ``tags = self._extract_tags (request.headers)`` and threaded ``tags=tags`` into its outcome. Closes the observability loop opened by PR #477: the four Gemini- bypass-bug fixes are now visible in the request-log feed the moment they fire. |
||
|
chopratejas
|
694589fec4 |
refactor(proxy): collapse 3 stream finalizers onto RequestOutcome.from_stream
Three streaming finalizers — ``_finalize_stream_response``,
``_stream_response_bedrock``, ``_stream_openai_via_backend`` — each
duplicated the same set of body- and config-derived fields when
constructing a ``RequestOutcome``:
* ``attempted_input_tokens = optimized_tokens + tokens_saved``
* ``num_messages = len(body.get("messages", []))``
* ``request_messages`` conditional on ``config.log_full_messages``
* ``transforms_applied`` list → tuple (frozen-dataclass contract)
* ``tags or {}`` normalization
* ``turn_id`` via ``compute_turn_id``
The last one was a real bug. Only the Bedrock site computed
``turn_id`` — sites 1 and 3 silently dropped it, breaking the
dashboard's multi-turn-session grouping for every Anthropic-SSE and
OpenAI-via-backend request. The new ``RequestOutcome.from_stream``
classmethod computes it uniformly so the three finalizers cannot
drift apart on derivation logic again.
Each call site now hands ``from_stream`` the body + provider-specific
cache/timing fields and gets a fully-constructed outcome back. The
funnel call after it stays identical (``await
self._record_request_outcome(outcome)``).
|
||
|
chopratejas
|
d73cbd6b0a |
fix(build): shrink Rust extension wheels — strip + thin-LTO + single codegen unit
PyPI rejected the v0.21.37 release publish with:
HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
Project size too large. Limit for project 'headroom-ai' total size is 10 GB.
PyPI inventory check confirmed: **191 versions × ~213 MB/release =
10.00 GB exactly** — at the cumulative project storage ceiling. Each
recent release ships 12 wheels × ~16-18 MB each.
Post-mortem inspection of a production wheel
(``headroom_ai-0.21.36-cp311-cp311-manylinux_2_28_x86_64.whl``)
showed the binary was ``not stripped``:
.text 18.3 MB (code)
.rodata 11.4 MB (Magika model + ONNX runtime data)
.strtab 4.9 MB (debug strings — strippable)
.eh_frame 1.9 MB (unwind tables)
.symtab 1.5 MB (debug symbols — strippable)
.gcc_except_table 1.2 MB
This commit adds a release profile:
[profile.release]
strip = "symbols"
lto = "thin"
codegen-units = 1
That:
* Strips ``.symtab`` + ``.strtab`` (~6.4 MB direct savings per wheel)
* Enables thin link-time optimization for cross-crate dead-code
elimination (~5-10% ``.text`` savings)
* Single codegen unit for better inlining + DCE at the cost of
~30-50% slower release builds (acceptable for CI)
Deliberately NOT setting ``panic = "abort"``:
* The proxy is a long-lived async process. A panic on one bad
request triggering process abort would disconnect every concurrent
client. Accept the smaller savings; keep unwind behaviour.
Estimated impact
* Per wheel: ~16-18 MB → ~10-11 MB (40% smaller)
* Per release (12 wheels): ~213 MB → ~130 MB
* PyPI capacity: ~30+ more releases before hitting 10 GB again
Verification
* Local build of ``headroom._core`` with new profile:
``.so`` size 29 MB on macOS arm64 (was ~45 MB pre-fix; final wheel
compressed will be smaller on Linux which also benefits from the
``strip`` directive).
* 77 Rust-parity tests pass — extension still functional.
* Single-codegen-unit slows build by ~30-50% but maturin/cibuildwheel
build time was never the bottleneck.
Forward strategy (separate work)
* Submit a PyPI project-size-limit-increase request to unblock the
immediate release.
* Adopt a release-deprecation policy: yank versions older than N
patches per minor; consider dropping Python 3.10 wheels (EOL'd
October 2026) and manylinux_2_28_aarch64 wheels (niche audience,
largest at 18.75 MB).
* Investigate runtime-download for Magika model (~10 MB further
savings) — same pattern Kompress already uses.
|
||
|
chopratejas
|
07bcda2796 | chore: sync plugin versions to 0.21.33 | ||
|
chopratejas
|
3432ee3a96 | docs: add README redesign spec (lean-ctx parity + Headroom-first blend) | ||
|
Tejas Chopra
|
f34748df7a
|
Merge pull request #457 from JerrettDavis/main
ci: correct live checks when credentials are missing |