headroom/scripts
Tejas Chopra 5829602fa8 refactor!: remove rtk and lean-ctx CLI context tools
Both third-party CLI context tools are removed, and with them the
context-tool selector itself. Headroom no longer downloads, installs or
configures either one, and there is no replacement.

The previous pass (#2344) gated only three entry points inside
headroom/cli/wrap.py, which left the feature reachable in practice:

* scripts/install.sh and install.ps1 ran `rtk init --global --auto-patch`
  from bash/PowerShell, bypassing the Python gate entirely, so `curl | sh`
  still wrote a Claude Code PreToolUse hook regardless of HEADROOM_RTK.
* `wrap openhands` was broken by default: rtk_required=True met a gate
  returning None and raised SystemExit(1). Every openhands test patched
  _ensure_rtk_binary to a fake path, so it was invisible.
* The proxy shelled out to `rtk gain` from /stats, the dashboard and
  `headroom perf`, and the subscription tracker polled it per contribution.
* Nothing removed artifacts an earlier default had already installed, so a
  machine that once ran the old default kept rtk in the loop forever.

Removed: the headroom/rtk and headroom/lean_ctx packages,
headroom/cli/wrap_rtk_metrics.py, _selected_context_tool /
_setup_context_tool_for_agent / _VALID_CONTEXT_TOOLS, the --rtk, --no-rtk,
--no-project-rtk and --keep-rtk flags, HEADROOM_RTK*, the proxy-side gain
polling and the dashboard CLI-filtering panel.

--context-tool / --no-context-tool / HEADROOM_CONTEXT_TOOL survive only to
fail loudly. They live in shell profiles, aliases and CI jobs, and accepting
them as a no-op would read as Headroom having quietly stopped working. The
installers reject them too: their arg parsers forward the first unknown flag
*and everything after it* to the wrapped tool, so a leftover --no-rtk would
have silently swallowed a following --port and been ignored downstream.

Added headroom/context_tool_cleanup.py. Deleting the code cannot help a
machine that already ran the old default, so purge_context_tool_artifacts()
runs once per wrap/unwrap and removes the registered hook entries, the
generated hook scripts, the Headroom-managed ~/.local/bin symlinks, the
vendored ~/.headroom/bin/{rtk,lean-ctx} binaries, the lean-ctx MCP server
entry and the injected instruction fences. It is idempotent, skips a
malformed config rather than overwriting it, and only unlinks a symlink that
resolves inside Headroom's own bin dir, so a user's own build is untouched.
It reports on stderr, because `wrap/unwrap openclaw --prepare-only` emit
machine-readable JSON on stdout as their entire contract. It is skipped for
`wrap selfheal` (runs from a SessionStart hook, must not race Claude Code's
writer for ~/.claude.json) and for --help, which must stay read-only.

Also hardens the client-config writes this work touched. wrap.py reset a
settings file to {} when an existing file would not parse and then wrote
that back, turning one hand-edited typo or a transient read error into total
loss of a user's permissions/env/hooks on every `wrap claude`; it now
refuses to write. fsutil.write_text is atomic (temp file + fsync +
os.replace), which fixes all 14 non-atomic client-config writes at once,
follows symlinks rather than replacing them, and preserves an existing
file's mode.

rtk stays in the wrapper-peel list in transforms/content_router.py on
purpose: it sits beside sudo/env/timeout as shell-command grammar, so
`rtk cat f` is still classified as a file read for anyone running their own
rtk install, which the purge deliberately leaves alone.
2026-07-30 21:11:10 -07:00
..
ci ci: harden PR governance and model cache checks (#1401) 2026-06-26 21:34:34 -07:00
fixtures feat(scripts): add Codex proxy reconnect-storm repro harness 2026-04-20 22:02:02 +07:00
tests fix(release): sync generated version metadata on the release branch (#2659) 2026-07-29 15:12:04 -07:00
audit_wheel_glibc_symbols.py fix(crusher): shim __libc_single_threaded for glibc < 2.32 + extend audit 2026-05-05 13:59:21 -07:00
bootstrap-windows-dev.ps1 chore(release): harden local artifact smokes (#1824) 2026-07-14 16:07:34 -04:00
build_npm_release_assets.mjs chore(release): harden local artifact smokes (#1824) 2026-07-14 16:07:34 -04:00
build_python_release_smoke.py chore(release): harden local artifact smokes (#1824) 2026-07-14 16:07:34 -04:00
build_rust_extension.sh refactor: single-wheel maturin build backend (fixes #355) 2026-05-03 13:16:41 -07:00
changelog-gen.py chore: renormalize line endings to LF 2026-04-24 15:33:30 +02:00
eval_output_shaper.py feat: output-token reduction — verbosity shaper, per-user learning, counterfactual savings (#965) 2026-06-16 21:06:43 -07:00
export_kompress_v2_onnx.py feat: switch Kompress default to kompress-v2-base with weight-only int8 ONNX (#799) 2026-06-09 23:28:40 -07:00
install-git-hooks.sh Fix CI lint failure by formatting PR governance scripts (#933) 2026-06-12 17:11:39 -05:00
install.ps1 refactor!: remove rtk and lean-ctx CLI context tools 2026-07-30 21:11:10 -07:00
install.sh refactor!: remove rtk and lean-ctx CLI context tools 2026-07-30 21:11:10 -07:00
pr-governance.py ci: harden PR governance and model cache checks (#1401) 2026-06-26 21:34:34 -07:00
README.md chore(release): harden local artifact smokes (#1824) 2026-07-14 16:07:34 -04:00
record_code_compressor_fixtures.py feat(rust): port CodeCompressor AST compressor to Rust (parity-only) (#1154) 2026-07-27 09:21:57 -07:00
record_fixtures.py feat(rust): scaffold workspace + parity harness (phase-0) 2026-04-24 13:39:48 -07:00
record_kompress_fixtures.py feat(rust): port Kompress ML prose compressor to Rust (parity-only) (#1153) 2026-07-27 08:17:53 -07:00
refresh_model_limits.sh fix(rust): wire ICM compressor into Rust proxy on /v1/messages 2026-05-01 16:44:44 -07:00
release_smoke_all.py chore(release): harden local artifact smokes (#1824) 2026-07-14 16:07:34 -04:00
replay_codex_ws_load.py fix(tests): ship scripts/replay_codex_ws_load.py so CI can import it 2026-05-14 13:44:41 -07:00
repro_codex_replay.py fix: replace asyncio.timeout with 3.10-compat shim in repro harness 2026-04-20 13:41:02 -05:00
smoke_issue_327.py fix(proxy): remove content-keyed TTL walker that conflated content with positional cache (#327) 2026-05-01 12:04:28 -07:00
sync-plugin-versions.py fix(proxy): lazy-import server to avoid fastapi crash (#442) 2026-06-10 12:44:23 -05:00
validate-workflows.sh ci: scope PR workflow runs by changed paths (#1067) 2026-06-16 19:11:45 -07:00
verify-ruff-version.py fix(ci): align Ruff tooling versions (#2406) 2026-07-18 20:55:20 -07:00
verify-versions.py fix: make proxy upgrades version-aware 2026-05-09 15:58:27 -07:00
verify_npm_release_assets.mjs chore(release): harden local artifact smokes (#1824) 2026-07-14 16:07:34 -04:00
version-sync.py fix(release): sync generated version metadata on the release branch (#2659) 2026-07-29 15:12:04 -07:00

scripts/

Utility scripts bundled with the Headroom repo. Most are one-off operator tools; a few are runnable as part of development workflows.

Reproducing the reconnect storm

repro_codex_replay.py reproduces the multi-agent Codex reconnect/retry storm against a local Headroom proxy (default http://127.0.0.1:8787). Use it to:

  • Regression-check that /livez stays responsive under a cold-start storm.
  • Empirically tune the Unit 4 pre-upstream semaphore default (HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY).
  • Exercise the Codex WS lifecycle + Anthropic HTTP path simultaneously without needing to replay captured production traffic.

Run

# Default: 8 WS + 4 HTTP clients, 30s storm, p99 /livez must stay <= 500ms.
python scripts/repro_codex_replay.py

# Tighter budget, shorter run:
python scripts/repro_codex_replay.py \
    --url http://127.0.0.1:8787 \
    --ws-clients 16 \
    --anthropic-clients 8 \
    --duration 60 \
    --livez-threshold-ms 100

# Dump the full summary as JSON for downstream tooling:
python scripts/repro_codex_replay.py --json

Exit code:

  • 0 — warmup succeeded (or was skipped), storm ran for the requested duration, and /livez p99 stayed under --livez-threshold-ms.
  • 1 — soft assertion failed, proxy unreachable, or unhandled exception. Proxy-unreachable is detected and reported within ~5 seconds.

Fixtures

The script loads two hand-crafted, fully synthetic JSON fixtures:

  • scripts/fixtures/anthropic_replay_body.json — shape of a large agent reconnect replay /v1/messages?beta=true POST body.
  • scripts/fixtures/codex_response_create_frame.json — first Codex WS frame with the {"type": "response.create", "response": {...}} envelope.

Override via --ws-frame-fixture / --anthropic-body-fixture if you have captured traffic to replay instead.

Interpretation

  • /livez p99 under threshold means the event loop is not starved during the storm. If it rises with the semaphore unbounded (HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY=10000) and drops back under the default, Unit 4's backpressure is working.
  • Codex WS: opened should equal --ws-clients. response.completed typically stays low when upstream auth isn't configured locally — the goal is handshake + relay wiring, not real upstream traffic.
  • Anthropic HTTP: ok_2xx + non_2xx + timed_out + errors should roughly equal attempted. Sustained non-zero timed_out during the storm is the failure signal the plan targets.

A smoke test at tests/test_scripts/test_repro_codex_replay_smoke.py exercises the script against a mock FastAPI server on every PR.

Install scripts

  • install.sh — POSIX installer.
  • install.ps1 — Windows PowerShell installer.

These are generated by the release pipeline; edit with care.

Windows development bootstrap

bootstrap-windows-dev.ps1 prepares a Windows development checkout. It resolves or creates a repo-local Python virtual environment, checks for Rust, installs Python build/test tooling, installs npm dependencies for the TypeScript SDK and OpenClaw plugin, and runs a small smoke set.

powershell -ExecutionPolicy Bypass -File scripts/bootstrap-windows-dev.ps1

Use -CheckOnly to print detected tool versions without installing packages. Use -SkipSmoke, -SkipDocs, -SkipNode, or -SkipRust when intentionally debugging one part of the environment.

npm release asset smoke

build_npm_release_assets.mjs locally reproduces the release workflow's npm asset build. It builds the TypeScript SDK tarball, installs that tarball into OpenClaw, rewrites OpenClaw's release dependency to the same version, regenerates dist/package.json, packs OpenClaw, and then runs verify_npm_release_assets.mjs.

node scripts/build_npm_release_assets.mjs <version>

By default, output goes into a timestamped release-assets-local/<version>-* directory. Pass an explicit empty directory when you want a predictable path:

node scripts/build_npm_release_assets.mjs <version> release-assets-local/smoke

Expected tarballs:

  • headroom-ai-<version>.tgz
  • headroom-openclaw-<version>.tgz

The script restores package metadata after it finishes so the source tree keeps the registry-installable development dependency range.

Python release artifact smoke

build_python_release_smoke.py locally reproduces the Python artifact smoke: it builds a wheel with maturin, builds an sdist, verifies the sdist License-File metadata against tarball contents, installs the wheel into a fresh python -m venv environment, and imports the native headroom._core extension from that installed wheel.

python scripts/build_python_release_smoke.py

By default, the wheel uses the faster Cargo ci profile and output goes into a timestamped release-assets-local/python-<version>-* directory. Use --release when you want the slower shipped-wheel profile:

python scripts/build_python_release_smoke.py --release --out release-assets-local/python-release-smoke

Expected artifacts:

  • headroom_ai-<version>-*.whl
  • headroom_ai-<version>.tar.gz

Full local release smoke

release_smoke_all.py is the one-command local release gate. It first runs scripts/verify-versions.py, then runs the npm release asset smoke and the Python wheel/sdist smoke into sibling output directories.

python scripts/release_smoke_all.py

By default, output goes into release-assets-local/all-<version>-*/npm and release-assets-local/all-<version>-*/python. Pass an explicit empty output directory for a predictable evidence path:

python scripts/release_smoke_all.py --out release-assets-local/full-release-smoke

Use --python-release when the Python smoke should build with maturin's slower release profile. Use --skip-npm or --skip-python only when intentionally debugging one side of the artifact pipeline.