Commit graph

5 commits

Author SHA1 Message Date
Tejas Chopra
cbb950a441
ci(governance): require a Conventional Commit PR title (#3063)
## Description

The repo squash-merges, so the PR title — not the commits inside the PR
— becomes the commit subject on `main`. Nothing validated it.

`commitlint` (`ci.yml:429`) lints a PR's *commits* and therefore cannot
catch this by construction: a PR with clean conventional commits and a
prose title passes CI and then lands a prose subject on `main`.

That is how `31452426` landed:

```
Unify savings attribution across stats, perf, metrics, and dashboard (#2976)
```

release-please cannot parse it — `unexpected token ' ' at 1:6`, because
`Unify` is five characters and position six is a space where the parser
needs `(`, `!` or `:`. The change is silently dropped from the
changelog.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Changes Made

- Added `COMMIT_TYPES` and `TITLE_RE` to `scripts/pr-governance.py`,
matching `.commitlintrc.json`'s `type-enum`.
- Added a title check to `validate_pull_request`, reported through the
existing governance comment.
- Added a test asserting `COMMIT_TYPES` equals `.commitlintrc.json`'s
`type-enum`, so the two gates cannot drift apart.
- Gave the `_event` test helper the `title` field a real `pull_request`
payload always carries.

## Testing

- [x] Unit tests pass
- [x] Linting passes (ruff check + format)
- [ ] Type checking passes — N/A (script + test only)
- [x] New tests added for new functionality

### Test Output

```text
$ .venv/bin/python -m pytest scripts/tests/test_pr_governance.py -q
12 passed in 0.02s
```

Against the parent commit:

```text
FAILED test_validate_pull_request_rejects_non_conventional_title
FAILED test_validate_pull_request_rejects_empty_and_typeless_titles
FAILED test_commit_types_match_commitlint_config
3 failed, 9 passed
```

## Real Behavior Proof

- Environment: macOS 15 (darwin 25.4.0), Python 3.12.13,
`scripts/pr-governance.py` loaded directly.
- Exact command / steps: ran `TITLE_RE` against the titles of **all 117
pull requests opened in this repository between 2026-08-10 and
2026-08-16**, pulled with `gh pr list --json title`.
- Observed result: exactly one title is flagged — `#2976`, `Unify
savings attribution across stats, perf, metrics, and dashboard`, the one
that jammed the release. Zero false positives across the other 116,
including every Dependabot `deps: bump ...` title, `chore: release
main`, and scoped forms like `fix(proxy/anthropic): ...`.
- Not tested: the check running inside a live `pull_request_target`
event on a GitHub runner.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: yes — a PR with a non-conventional
title now gets the `status: needs author action` label and a governance
comment.
- Kill switch / disable path: revert; the check is not independently
configurable.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes

## Additional Notes

The check lives in `pr-governance.py` rather than `ci.yml` for two
reasons:

1. `ci.yml`'s `pull_request` trigger has no `edited` type, so a
corrected title would never be re-checked.
2. Its `paths-ignore` skips docs-only PRs, which still squash-merge a
subject onto `main`.

`pr-health.yml` already triggers on `edited` and reports through the
same governance comment the author is reading anyway.

Bot PRs keep their existing exemption — Dependabot and release-please
titles are already conventional, and the early return for `is_bot_pr` is
untouched.

Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
2026-08-16 19:05:36 -07:00
JD Davis
3077ac81e8
feat: add deterministic runtime rollout controls (#1490)
## Description

Establish one centrally resolved, observable, deterministic, versioned
runtime rollout-control mechanism for Headroom. Runtime rollout controls
which behaviors an already-built artifact may expose; it does not select
or qualify a Headroom release/version.

## Type of Change

- [x] New feature (non-breaking change that adds functionality)
- [x] Bug fix (non-breaking change that fixes rollout enforcement
regressions)
- [x] Documentation update
- [x] Code refactoring (no functional changes)

## Changes Made

- Added `RolloutChannel`, `HEADROOM_ROLLOUT_CHANNEL`,
`--rollout-channel`, and a versioned immutable `RolloutSnapshot` shared
by Python configuration boundaries.
- Added schema/policy versions, canonical registry and snapshot SHA-256
identities, per-feature decision reasons, disable precedence, unsafe
qualification poisoning, strict CLI validation, and fail-closed
environment handling.
- Added `headroom rollout status --json`, Python `/stats.rollout`, and
Rust `/rollout/status` runtime provenance.
- Added equivalent Rust snapshot semantics and shared Python/Rust policy
vectors while retaining language-specific feature registries.
- Enforced rollout policy at alternate Python server composition roots
so `HEADROOM_READ_MATURATION=1` cannot bypass its beta gate.
- Preserved typed rollout snapshots across multi-worker serialization
with schema, policy, registry, snapshot-digest, type, and feature-name
validation.
- Made loopback runtime output-shaper updates replace the immutable
snapshot atomically for request readers, retain explicit request/disable
provenance, preserve channel and kill-switch precedence, invalidate
cached stats, and return the effective rollout decision.
- Made `headroom learn --verbosity --apply` report a channel-blocked
update instead of claiming the shaper is live.
- Made explicit CLI feature flags fail loudly when their current channel
blocks them.
- Made persistent interceptor installation select canary automatically,
or reject an explicitly insufficient channel unless the break-glass
override is set.
- Updated architecture, proxy, rollout, learn, and output-shaper
documentation with required channels and hot-reload semantics.

## Testing

- [x] Unit tests pass
- [x] Linting passes (`ruff check .` and `ruff format --check .`)
- [x] Type checking passes (`mypy headroom --ignore-missing-imports`)
- [x] New regression tests added for every corrected behavior
- [x] Rust tests and production-target Clippy pass
- [x] Documentation build passes

### Test Output

```text
Focused rollout coverage suite
57 passed; headroom.rollout + rollout CLI: 98% coverage

Affected proxy/rollout/transform/governance suites
222 passed; 0 failed

Final changed regression suites
100 passed; 0 failed

Cross-module hot-reload isolation regression
6 passed; 0 failed

cargo test -p headroom-core -p headroom-proxy --quiet
headroom-core: 924 passed; 1 ignored
headroom-proxy and integration suites: all passed

cargo clippy -p headroom-core -p headroom-proxy --lib --bins -- -D warnings
cargo fmt --all -- --check
ruff check .
ruff format --check .
mypy headroom --ignore-missing-imports
git diff --check
All passed

cd docs && npm run build
Compiled successfully; 164 static pages generated
```

The unsharded Windows-only CI selection exposed unrelated baseline
failures, principally the existing `sqlite:///C:\\...` URL parser
producing an invalid `\\C:\\...` path. At commit `8e793a80`, all 52
completed GitHub checks passed; the only other conclusions are expected
skips and superseded governance jobs.

## Real Behavior Proof

- **Environment:** Windows checkout on Python 3.13.3 and the current
Rust workspace, based on upstream `main` at `93f2d7a2`.
- **Exact command / steps:** Exercised canary and beta feature requests
through CLI status, Python `/stats.rollout`, Rust `/rollout/status`,
multi-worker payload round trips, loopback `/admin/runtime-env`, real
proxy request shaping before/after hot reload, installer manifest
generation, and shared Python/Rust policy vectors.
- **Observed result:** Stable blocks unstable requests; disable wins
over explicit/default/legacy/unsafe paths; unsafe state reports
`qualification_eligible=false`; worker handoff rejects tampering;
running output shaping changes only when the effective beta policy
permits it; explicit blocked flags fail with actionable diagnostics.
- **Not tested:** Live production traffic requiring provider
credentials, or future artifact qualification/promotion automation
(intentionally out of scope).

## Runtime Rollout Safety

- **Rollout-managed features:** Python `tool_result_interceptors`,
`proxy_output_shaper`, `read_maturation`; Rust `native_bedrock`,
`openai_responses_streaming`, `canary_probe`.
- **Minimum rollout channel:** Registry-defined per feature; process
default is `stable`.
- **Stable/default behavior changed:** No unstable feature becomes
enabled by default. Explicit blocked CLI flags now fail instead of
silently doing nothing.
- **Kill switch / disable path:**
`HEADROOM_DISABLE_FEATURES=<comma-separated feature names>`; explicit
disable has highest precedence, including over the unsafe override.
- **Unsafe override required:** No.
`HEADROOM_UNSAFE_ALLOW_UNSTABLE_FEATURES=1` is break-glass only and
makes qualification evidence ineligible.
- **Qualification impact:** Adds machine-readable policy/snapshot
identities and eligibility; does not implement qualification itself.
- **Rollback path:** Set the named disable list for operational
rollback, lower the channel, or revert this PR.

## Review Readiness

- [x] I have performed a full diff review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [x] I have made corresponding documentation changes
- [x] My changes generate no new warnings
- [x] I added tests that reproduce and prevent every regression fixed
during review
- [x] New and existing affected tests pass locally
- [x] I did **not** edit `CHANGELOG.md`; release-please generates it
from the Conventional Commit PR title

## Additional Notes

Out of scope: artifact candidates, benchmark orchestration,
qualification manifests/gates, promotion automation, release branches,
publication guards, and release-risk classification. Those workflows can
consume the rollout registry digest, runtime snapshot digest, decision
reasons, and qualification eligibility through supported black-box
interfaces.

---------

Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
Co-authored-by: JD Davis <jd@JDH-AIR-00.local>
2026-08-12 23:16:54 -05:00
JD Davis
adb793bee1
ci: harden PR governance and model cache checks (#1401)
## Description

Hardens two routine PR-review pain points from the recent open-PR sweep:

- PR Governance reruns could keep validating the stale
`pull_request_target` event body even after the live PR description had
been fixed.
- Main CI model-cache misses could surface as dozens of unrelated
memory-test failures instead of one clear cache-preflight failure.

This intentionally avoids PyPI/package-bloat and release/nightly
workflow changes so the PR stays scoped to review and CI stabilization.

## Type of Change

- [x] Bug fix
- [ ] New feature
- [ ] Documentation
- [x] Refactor
- [x] Tests only

## Changes Made

- Added `--body-file` support to `scripts/pr-governance.py` so workflows
can validate the current PR body rather than stale rerun payloads.
- Updated PR Governance to fetch the live PR body via the GitHub API
before validating template fields.
- Added a CI preflight script that loads the default
sentence-transformer model in offline mode and verifies the expected
embedding dimension.
- Wired that preflight into the sharded CI job before pytest starts,
turning missing/corrupt Hugging Face caches into one early, actionable
failure.
- Added workflow/script regression tests for the live-body override and
model-cache preflight placement.

## Testing

- [x] Unit tests
- [x] Lint/static checks
- [ ] Integration tests
- [ ] Manual testing

### Test Output

```text
uv run --with pytest --with pytest-asyncio python -m pytest scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_workflow.py scripts/tests/test_ci_workflow.py -q
9 passed in 0.04s

uv run ruff check scripts/pr-governance.py scripts/ci/verify_hf_model_cache.py scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_workflow.py scripts/tests/test_ci_workflow.py
All checks passed!

python -m py_compile scripts\ci\verify_hf_model_cache.py scripts\pr-governance.py
# passed
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.3, isolated worktree
`C:\git\headroom\.worktrees\stabilization-hardening`.
- Exact command / steps: Ran the focused governance/workflow tests, ruff
on touched Python files, and `py_compile` for the executable scripts.
- Observed result: Governance tests prove a stale event body can be
overridden by the live PR body; workflow tests prove CI validates live
PR body and runs the Hugging Face offline-cache preflight before pytest
shards.
- Not tested: Full GitHub CI before PR creation; that will run on this
PR. The new Hugging Face preflight itself is intentionally not run
locally because it depends on the CI-warmed offline model cache.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review
2026-06-26 21:34:34 -07:00
Parideboy
5194388b66
fix(ci): normalize Windows CRLF line endings in PR governance script (#1012)
## Description

The `CODE_BLOCK_RE` regex in `scripts/pr-governance.py` expects LF after
the opening fenced code block. PR bodies authored on Windows can arrive
with CRLF line endings, which leaves a `\r` before the `\n` and prevents
`has_test_output()` from detecting a valid Test Output block.

This normalizes CRLF to LF once when loading the pull request body,
before section extraction and code-block matching. A regression test now
verifies that a valid PR body with CRLF line endings still passes
governance.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Tests only

## Changes Made

- Normalize Windows CRLF line endings in `scripts/pr-governance.py`
before regex-based validation runs.
- Added `test_validate_pull_request_accepts_crlf_test_output_code_block`
to prevent regressions.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Formatting passes (`ruff format --check`)
- [x] New tests added for new functionality

### Test Output

```text
python -m pytest scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py -q
8 passed in 0.06s

ruff check scripts/pr-governance.py scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py
All checks passed!

ruff format --check scripts/pr-governance.py scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py
4 files already formatted
```

## Real Behavior Proof

- Environment: local Windows 11 checkout, Python 3.13.13.
- Exact command / steps: Converted the known-valid governance test body
to CRLF line endings and passed it through `validate_pull_request` in
the new regression test.
- Observed result: The report is valid with no problems, proving the
fenced Test Output block is recognized after normalization.
- Not tested: GitHub-hosted Windows PR authoring path end to end; the
unit test covers the exact CRLF body shape consumed by the validator.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-06-22 18:45:12 -05:00
Copilot
96a7d7cbbe
Fix CI lint failure by formatting PR governance scripts (#933)
`CI / lint (pull_request)` failed because `ruff format --check` detected
formatting drift in the new PR governance script and its tests. This PR
aligns those files with repository formatting rules so the lint job can
pass.

- **Root cause**
  - `ruff format --check .` reported two files as non-canonical:
    - `scripts/pr-governance.py`
    - `scripts/tests/test_pr_governance.py`

- **Change set**
  - Applied `ruff` formatting to only the two flagged files.
- No behavioral or logic changes; edits are line-wrap/format
normalization only.

- **Representative update**
  ```python
  parser.add_argument(
"--event", type=Path, required=True, help="Path to the GitHub event
payload JSON."
  )
  ```

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-12 17:11:39 -05:00