mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
ci(governance): require a Conventional Commit PR title (#3063)
## Description
The repo squash-merges, so the PR title — not the commits inside the PR
— becomes the commit subject on `main`. Nothing validated it.
`commitlint` (`ci.yml:429`) lints a PR's *commits* and therefore cannot
catch this by construction: a PR with clean conventional commits and a
prose title passes CI and then lands a prose subject on `main`.
That is how `31452426` landed:
```
Unify savings attribution across stats, perf, metrics, and dashboard (#2976)
```
release-please cannot parse it — `unexpected token ' ' at 1:6`, because
`Unify` is five characters and position six is a space where the parser
needs `(`, `!` or `:`. The change is silently dropped from the
changelog.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
## Changes Made
- Added `COMMIT_TYPES` and `TITLE_RE` to `scripts/pr-governance.py`,
matching `.commitlintrc.json`'s `type-enum`.
- Added a title check to `validate_pull_request`, reported through the
existing governance comment.
- Added a test asserting `COMMIT_TYPES` equals `.commitlintrc.json`'s
`type-enum`, so the two gates cannot drift apart.
- Gave the `_event` test helper the `title` field a real `pull_request`
payload always carries.
## Testing
- [x] Unit tests pass
- [x] Linting passes (ruff check + format)
- [ ] Type checking passes — N/A (script + test only)
- [x] New tests added for new functionality
### Test Output
```text
$ .venv/bin/python -m pytest scripts/tests/test_pr_governance.py -q
12 passed in 0.02s
```
Against the parent commit:
```text
FAILED test_validate_pull_request_rejects_non_conventional_title
FAILED test_validate_pull_request_rejects_empty_and_typeless_titles
FAILED test_commit_types_match_commitlint_config
3 failed, 9 passed
```
## Real Behavior Proof
- Environment: macOS 15 (darwin 25.4.0), Python 3.12.13,
`scripts/pr-governance.py` loaded directly.
- Exact command / steps: ran `TITLE_RE` against the titles of **all 117
pull requests opened in this repository between 2026-08-10 and
2026-08-16**, pulled with `gh pr list --json title`.
- Observed result: exactly one title is flagged — `#2976`, `Unify
savings attribution across stats, perf, metrics, and dashboard`, the one
that jammed the release. Zero false positives across the other 116,
including every Dependabot `deps: bump ...` title, `chore: release
main`, and scoped forms like `fix(proxy/anthropic): ...`.
- Not tested: the check running inside a live `pull_request_target`
event on a GitHub runner.
## Runtime Rollout Safety
- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: yes — a PR with a non-conventional
title now gets the `status: needs author action` label and a governance
comment.
- Kill switch / disable path: revert; the check is not independently
configurable.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this commit.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
## Additional Notes
The check lives in `pr-governance.py` rather than `ci.yml` for two
reasons:
1. `ci.yml`'s `pull_request` trigger has no `edited` type, so a
corrected title would never be re-checked.
2. Its `paths-ignore` skips docs-only PRs, which still squash-merge a
subject onto `main`.
`pr-health.yml` already triggers on `edited` and reports through the
same governance comment the author is reading anyway.
Bot PRs keep their existing exemption — Dependabot and release-please
titles are already conventional, and the early return for `is_bot_pr` is
untouched.
Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
This commit is contained in:
parent
ac8646aa3c
commit
cbb950a441
2 changed files with 114 additions and 1 deletions
|
|
@ -41,6 +41,28 @@ ROLLOUT_FIELDS = (
|
|||
"Rollback path",
|
||||
)
|
||||
|
||||
# Conventional-commit types accepted by .commitlintrc.json. Keep the two in
|
||||
# sync: commitlint gates the *commits* on a PR, but the repo squash-merges, so
|
||||
# it is the PR *title* that becomes the subject line on main.
|
||||
COMMIT_TYPES = (
|
||||
"build",
|
||||
"chore",
|
||||
"ci",
|
||||
"deps",
|
||||
"docs",
|
||||
"feat",
|
||||
"fix",
|
||||
"parity",
|
||||
"perf",
|
||||
"refactor",
|
||||
"revert",
|
||||
"style",
|
||||
"test",
|
||||
)
|
||||
|
||||
# type(optional-scope)!: subject
|
||||
TITLE_RE = re.compile(rf"^(?:{'|'.join(COMMIT_TYPES)})(?:\([^)]+\))?!?: .+")
|
||||
|
||||
SECTION_RE = re.compile(r"^##\s+(.+?)\s*$", re.MULTILINE)
|
||||
CHECKBOX_RE = re.compile(r"^- \[(?P<checked>[ xX])\] (?P<label>.+)$", re.MULTILINE)
|
||||
HTML_COMMENT_RE = re.compile(r"<!--.*?-->", re.DOTALL)
|
||||
|
|
@ -172,6 +194,19 @@ def validate_pull_request(event: dict[str, Any]) -> GovernanceReport:
|
|||
sections = extract_sections(body)
|
||||
problems: list[str] = []
|
||||
|
||||
# A squash-merge uses the PR title as the commit subject on main, and
|
||||
# release-please parses those subjects. One unparseable title stops it
|
||||
# building a release PR at all, and the change is silently dropped from the
|
||||
# changelog either way. commitlint cannot catch this: it lints the commits
|
||||
# inside the PR, not the title that replaces them.
|
||||
title = (pull_request.get("title") or "").strip()
|
||||
if not TITLE_RE.match(title):
|
||||
problems.append(
|
||||
f"PR title must be a Conventional Commit — `type(scope): subject` — because "
|
||||
f"squash-merge makes it the commit subject on `main` and release-please parses it. "
|
||||
f"Got: `{title or '(empty)'}`. Valid types: {', '.join(f'`{t}`' for t in COMMIT_TYPES)}."
|
||||
)
|
||||
|
||||
for section_name in REQUIRED_SECTIONS:
|
||||
if section_name not in sections:
|
||||
problems.append(f"Missing required section `{section_name}`.")
|
||||
|
|
|
|||
|
|
@ -19,11 +19,21 @@ def _load_module():
|
|||
return module
|
||||
|
||||
|
||||
def _event(body: str, *, draft: bool = False, login: str = "octocat") -> dict[str, object]:
|
||||
def _event(
|
||||
body: str,
|
||||
*,
|
||||
draft: bool = False,
|
||||
login: str = "octocat",
|
||||
title: str = "feat(governance): add a required PR-governance gate",
|
||||
) -> dict[str, object]:
|
||||
# A real `pull_request` payload always carries a title, and squash-merge
|
||||
# turns it into the commit subject on main, so the default here is a valid
|
||||
# Conventional Commit. Tests that exercise the title rule pass their own.
|
||||
return {
|
||||
"pull_request": {
|
||||
"number": 42,
|
||||
"draft": draft,
|
||||
"title": title,
|
||||
"body": body,
|
||||
"user": {"login": login},
|
||||
}
|
||||
|
|
@ -233,3 +243,71 @@ def test_validate_pull_request_skips_bot_authored_prs() -> None:
|
|||
assert report.is_bot_pr is True
|
||||
assert report.needs_author_action is False
|
||||
assert report.labels_to_add == []
|
||||
|
||||
|
||||
def test_validate_pull_request_rejects_non_conventional_title() -> None:
|
||||
"""The exact title that stalled release-please on v0.35.0 must be caught.
|
||||
|
||||
`Unify savings attribution ...` squash-merged to main as commit 31452426.
|
||||
release-please could not parse it (`unexpected token ' ' at 1:6`), so the
|
||||
change never reached a changelog. commitlint passed the PR because it lints
|
||||
the commits inside it, not the title that replaces them on squash-merge.
|
||||
"""
|
||||
module = _load_module()
|
||||
report = module.validate_pull_request(
|
||||
_event(
|
||||
VALID_BODY, title="Unify savings attribution across stats, perf, metrics, and dashboard"
|
||||
)
|
||||
)
|
||||
|
||||
assert report.valid is False
|
||||
assert any("Conventional Commit" in problem for problem in report.problems)
|
||||
|
||||
|
||||
def test_validate_pull_request_accepts_conventional_titles() -> None:
|
||||
module = _load_module()
|
||||
for title in (
|
||||
"fix(proxy): keep prefixed core tools resident",
|
||||
"deps: bump sha2 from 0.10.9 to 0.11.0",
|
||||
"chore: release main",
|
||||
"feat!: drop python 3.10",
|
||||
"fix(proxy/anthropic): stop replaying the recorded prefix",
|
||||
):
|
||||
report = module.validate_pull_request(_event(VALID_BODY, title=title))
|
||||
assert report.valid is True, (title, report.problems)
|
||||
|
||||
|
||||
def test_validate_pull_request_rejects_empty_and_typeless_titles() -> None:
|
||||
module = _load_module()
|
||||
for title in ("", " ", "WIP", "fix:", "Fix(proxy): capitalised type", "update stuff"):
|
||||
report = module.validate_pull_request(_event(VALID_BODY, title=title))
|
||||
assert report.valid is False, title
|
||||
assert any("Conventional Commit" in problem for problem in report.problems), title
|
||||
|
||||
|
||||
def test_bot_authored_prs_skip_title_enforcement() -> None:
|
||||
"""Dependabot/release-please titles are already conventional; don't gate them."""
|
||||
module = _load_module()
|
||||
report = module.validate_pull_request(
|
||||
_event("", login="dependabot[bot]", title="Bump sha2 from 0.10.9 to 0.11.0")
|
||||
)
|
||||
|
||||
assert report.is_bot_pr is True
|
||||
assert report.valid is True
|
||||
|
||||
|
||||
def test_commit_types_match_commitlint_config() -> None:
|
||||
"""The gate and commitlint must accept the same type vocabulary.
|
||||
|
||||
They enforce the same rule at two points in the lifecycle -- commitlint on
|
||||
the PR's commits, this on the title that squash-merge substitutes for them.
|
||||
Divergence would let a title through that the commit hook rejects, or vice
|
||||
versa.
|
||||
"""
|
||||
module = _load_module()
|
||||
config = json.loads(
|
||||
(Path(__file__).parent.parent.parent / ".commitlintrc.json").read_text(encoding="utf-8")
|
||||
)
|
||||
_level, _applicable, allowed = config["rules"]["type-enum"]
|
||||
|
||||
assert sorted(module.COMMIT_TYPES) == sorted(allowed)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue