akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
headroom/scripts
History
JD Davis 74dff94fb8
fix(ci): make PR governance advisory (#1047) 
## Description

Make the PR Governance workflow advisory for incomplete pull request
bodies. The workflow still validates the template, writes the run
summary, comments on the PR, and syncs governance labels, but it no
longer marks the check red for expected author follow-up.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Replaced the failing incomplete-template step with a reporting step
that exits successfully.
- Added a regression test that guards against reintroducing the hard
failure path.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Manual testing performed

### Test Output

```text
pytest scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py -q
# 7 passed

act pull_request_target -W .github/workflows/pr-health.yml -e .github/act/pr-governance-invalid.json -n
# Job succeeded

act pull_request_target -W .github/workflows/pr-health.yml -e .github/act/pr-governance-valid.json -n
# Job succeeded
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.13, act 0.2.87, Docker Desktop
via npipe.
- Exact command / steps: Ran the focused governance/label tests and
`act` dry-runs for the valid and invalid PR governance payloads.
- Observed result: Tests passed, the invalid payload's reporting step
completed successfully, and both PR Governance dry-runs ended with job
success.
- Not tested: Full non-dry-run `act` execution against GitHub API
side-effect steps, to avoid mutating real labels/comments from a local
run.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review
2026-06-16 12:36:39 -05:00
..
fixtures feat(scripts): add Codex proxy reconnect-storm repro harness 2026-04-20 22:02:02 +07:00
tests fix(ci): make PR governance advisory (#1047) 2026-06-16 12:36:39 -05:00
audit_wheel_glibc_symbols.py fix(crusher): shim __libc_single_threaded for glibc < 2.32 + extend audit 2026-05-05 13:59:21 -07:00
build_rust_extension.sh refactor: single-wheel maturin build backend (fixes #355) 2026-05-03 13:16:41 -07:00
changelog-gen.py chore: renormalize line endings to LF 2026-04-24 15:33:30 +02:00
export_kompress_v2_onnx.py feat: switch Kompress default to kompress-v2-base with weight-only int8 ONNX (#799) 2026-06-09 23:28:40 -07:00
install-git-hooks.sh Fix CI lint failure by formatting PR governance scripts (#933) 2026-06-12 17:11:39 -05:00
install.ps1 feat: add lean-ctx context tool support 2026-05-11 17:54:17 -04:00
install.sh feat: add lean-ctx context tool support 2026-05-11 17:54:17 -04:00
pr-governance.py Fix CI lint failure by formatting PR governance scripts (#933) 2026-06-12 17:11:39 -05:00
README.md feat(scripts): add Codex proxy reconnect-storm repro harness 2026-04-20 22:02:02 +07:00
record_fixtures.py feat(rust): scaffold workspace + parity harness (phase-0) 2026-04-24 13:39:48 -07:00
refresh_model_limits.sh fix(rust): wire ICM compressor into Rust proxy on /v1/messages 2026-05-01 16:44:44 -07:00
replay_codex_ws_load.py fix(tests): ship scripts/replay_codex_ws_load.py so CI can import it 2026-05-14 13:44:41 -07:00
repro_codex_replay.py fix: replace asyncio.timeout with 3.10-compat shim in repro harness 2026-04-20 13:41:02 -05:00
smoke_issue_327.py fix(proxy): remove content-keyed TTL walker that conflated content with positional cache (#327) 2026-05-01 12:04:28 -07:00
sync-plugin-versions.py fix(proxy): lazy-import server to avoid fastapi crash (#442) 2026-06-10 12:44:23 -05:00
validate-workflows.sh Fix CI lint failure by formatting PR governance scripts (#933) 2026-06-12 17:11:39 -05:00
verify-versions.py fix: make proxy upgrades version-aware 2026-05-09 15:58:27 -07:00
version-sync.py fix: make proxy upgrades version-aware 2026-05-09 15:58:27 -07:00

README.md

scripts/

Utility scripts bundled with the Headroom repo. Most are one-off operator tools; a few are runnable as part of development workflows.

Reproducing the reconnect storm

repro_codex_replay.py reproduces the multi-agent Codex reconnect/retry storm against a local Headroom proxy (default http://127.0.0.1:8787), as described in wiki/plans/2026-04-17-codex-proxy-runtime-analysis.md under "Latest Correction". Use it to:

  • Regression-check that /livez stays responsive under a cold-start storm.
  • Empirically tune the Unit 4 pre-upstream semaphore default (HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY).
  • Exercise the Codex WS lifecycle + Anthropic HTTP path simultaneously without needing to replay captured production traffic.

Run

# Default: 8 WS + 4 HTTP clients, 30s storm, p99 /livez must stay <= 500ms.
python scripts/repro_codex_replay.py

# Tighter budget, shorter run:
python scripts/repro_codex_replay.py \
    --url http://127.0.0.1:8787 \
    --ws-clients 16 \
    --anthropic-clients 8 \
    --duration 60 \
    --livez-threshold-ms 100

# Dump the full summary as JSON for downstream tooling:
python scripts/repro_codex_replay.py --json

Exit code:

  • 0 — warmup succeeded (or was skipped), storm ran for the requested duration, and /livez p99 stayed under --livez-threshold-ms.
  • 1 — soft assertion failed, proxy unreachable, or unhandled exception. Proxy-unreachable is detected and reported within ~5 seconds.

Fixtures

The script loads two hand-crafted, fully synthetic JSON fixtures:

  • scripts/fixtures/anthropic_replay_body.json — shape of a large agent reconnect replay /v1/messages?beta=true POST body.
  • scripts/fixtures/codex_response_create_frame.json — first Codex WS frame with the {"type": "response.create", "response": {...}} envelope.

Override via --ws-frame-fixture / --anthropic-body-fixture if you have captured traffic to replay instead.

Interpretation

  • /livez p99 under threshold means the event loop is not starved during the storm. If it rises with the semaphore unbounded (HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY=10000) and drops back under the default, Unit 4's backpressure is working.
  • Codex WS: opened should equal --ws-clients. response.completed typically stays low when upstream auth isn't configured locally — the goal is handshake + relay wiring, not real upstream traffic.
  • Anthropic HTTP: ok_2xx + non_2xx + timed_out + errors should roughly equal attempted. Sustained non-zero timed_out during the storm is the failure signal the plan targets.

A smoke test at tests/test_scripts/test_repro_codex_replay_smoke.py exercises the script against a mock FastAPI server on every PR.

Install scripts

  • install.sh — POSIX installer.
  • install.ps1 — Windows PowerShell installer.

These are generated by the release pipeline; edit with care.