Commit graph

29 commits

Author SHA1 Message Date
chopratejas
f5f465418b feat(rust): retire python diff_compressor, ship rust-only via pyo3
The python `DiffCompressor` is replaced by a thin pyo3-backed shim that
delegates to `headroom._core.DiffCompressor`. There is no python
implementation and no env-var fallback — the wheel is a hard import.

Why now: opt-in defaults don't drive python retirement. Byte-equal
parity was already proven across 27 fixtures (stage 3a); keeping a
shadow python impl behind a flag is a permanent maintenance cost with
no operational benefit. Stage 3b deletes ~700 lines of python parser /
scorer / formatter code; the rust crate has its own coverage.

Surface preserved:
- `headroom.transforms.diff_compressor.DiffCompressor` — same class
  name, same `__init__`, same `compress(content, context)` shape.
  Returns python `DiffCompressionResult` dataclasses so call sites that
  destructure with `asdict()` work unchanged.
- `DiffCompressorConfig` and `DiffCompressionResult` dataclasses kept.
- Sidecar `compress_with_stats(...)` exposes the rust-only
  `DiffCompressorStats` (per-file hunk drops, context lines trimmed,
  file_mode normalizations) for observability.

Removed:
- Python parser / scorer / formatter (~700 lines).
- Internal `DiffHunk` / `DiffFile` parser dataclasses (rust crate has
  parallel coverage).
- 12 tests that probed `_parse_diff` / `_score_hunks` or the deleted
  parser dataclasses. The 29 public-API tests in `test_diff_compressor.py`
  remain and now exercise the rust backend through the same import path.
- `HEADROOM_DIFF_COMPRESSOR_BACKEND` env var — no longer meaningful.

Build:
- `scripts/build_rust_extension.sh` runs `maturin develop` and symlinks
  the built `.so` into `headroom/` so `import headroom._core` resolves
  past the in-tree package shadowing the maturin overlay.
- `.gitignore` excludes the symlinks and allowlists the build script.

Tests:
- 544 transforms pass; 33 rust-vs-fixture parity tests guard the pyo3
  bridge against regressions (`tests/test_transforms/test_diff_compressor_rust_parity.py`).
- Mypy clean.
2026-04-26 09:15:37 -07:00
chopratejas
4429a11166 Merge remote-tracking branch 'origin/main' into rust-rewrite
# Conflicts:
#	headroom/proxy/server.py
2026-04-25 13:01:37 -07:00
chopratejas
0414cb70e4 feat(rust): scaffold workspace + parity harness (phase-0)
Bootstrap the Rust port of Headroom. Additive only — no existing Python
code modified. Ships the workshop, not the widgets.

Layout
  Cargo.toml (workspace) + rust-toolchain.toml
  crates/headroom-core    — transform library, stub only
  crates/headroom-proxy   — axum binary, /healthz only
  crates/headroom-py      — PyO3 cdylib, exposes headroom._core.hello()
  crates/headroom-parity  — Rust-vs-Python oracle harness + parity-run CLI

Tooling
  Makefile: test, test-parity, bench, build-proxy, build-wheel, fmt, lint
  .github/workflows/rust.yml: test, wheels (linux/mac), audit, parity-nightly
  deny.toml for cargo-deny

Parity corpus
  tests/parity/recorder.py + scripts/record_fixtures.py
  125 recorded fixtures across 5 leaf transforms (ccr, tokenizer,
  log_compressor, diff_compressor, cache_aligner)

Docs
  RUST_DEV.md — developer setup and workspace reference
  docs/spec/022-rust-migration.md — migration plan and stage breakdown

.gitignore: whitelist scripts/record_fixtures.py; ignore target/

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 13:39:48 -07:00
Garm
efd2ac1ca4 chore: renormalize line endings to LF
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but
74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings,
violating that contract. Every macOS/Linux clone reports these files as
"modified" on fresh checkout because git's diff engine sees the stored
bytes don't match the attribute contract, even though the working tree
and index match byte-for-byte.

Running `git add --renormalize .` rewrites each affected blob so the
stored form matches the attribute declaration. No semantic changes —
every affected file's diff is "N insertions, N deletions" with inserts
and deletes being the same lines modulo line endings.

Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub
blame skip this mechanical commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 15:33:30 +02:00
JerrettDavis
297f499e37 ci: retry workflow validation dry-runs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-23 13:20:13 -05:00
JerrettDavis
56f6307665 fix: support py310 version sync scripts
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-21 20:42:56 -05:00
JerrettDavis
b852460af9 chore: normalize line endings in init diffs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-21 20:17:14 -05:00
JerrettDavis
a278a7b0ba test: cover init install flows end to end
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-21 20:15:11 -05:00
JerrettDavis
c5d795c2af build: sync agent hook manifests to repo semver
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-21 20:03:14 -05:00
JerrettDavis
913435e03a fix: replace asyncio.timeout with 3.10-compat shim in repro harness
asyncio.timeout was added in Python 3.11, but the project supports >=3.10.
The test_repro_codex_replay_smoke test was crashing on Python 3.10 CI with
"AttributeError: module 'asyncio' has no attribute 'timeout'".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 13:41:02 -05:00
Tejas Chopra
cd11acc3c4
Merge pull request #197 from adryanev/fix/responses-retries-keep-compression
fix(proxy): Codex reconnect-storm resilience — bounded pre-upstream + WS session tracking + stage timings
2026-04-20 09:50:37 -07:00
Adryan Eka Vandra
b71b659e1b
fix(ci): restore repro harness test in dev installs
Add websockets to the dev extra so the repro harness smoke test can import
its websocket client dependency in the CI test matrix. Also apply ruff
formatting to the files the formatter check was rejecting so the 3.12 lint
job passes.
2026-04-20 22:12:01 +07:00
Adryan Eka Vandra
c438268d8a
feat(scripts): improve repro harness CLI contract and backoff
- Route --json output cleanly: JSON goes to stdout (machine-readable),
  human-readable summary goes to stderr. Without --json, the human
  summary stays on stdout as before. Makes piping the harness into
  jq / other tools trivial without losing operator visibility.
- Add distinct exit codes so CI and shell wrappers can branch on the
  failure class: 0 success, 1 crash, 2 proxy_unreachable, 3 livez
  threshold exceeded, 4 warmup failed, 130 SIGINT (already correct).
  Smoke test updated to assert EXIT_PROXY_UNREACHABLE instead of 1.
- Replace flat 50-250ms jitter in the Anthropic-client retry loop with
  exponential backoff + 50-150% jitter
  (base=250ms, max=5000ms, attempt counter). Matches the proxy's own
  jitter_delay_ms helper; inlined to keep the script free of proxy
  package imports.
2026-04-20 22:09:23 +07:00
Adryan Eka Vandra
c482d75a73
feat(scripts): add Codex proxy reconnect-storm repro harness
Reproducibly exercises the multi-agent Codex reconnect/retry storm
(origin §"Latest Correction") against a local proxy: concurrent Codex WS
sessions + parallel Anthropic /v1/messages replays, with /livez probed
every 250ms to detect event-loop starvation. Exits non-zero if /livez
p99 exceeds the configured threshold.

- scripts/repro_codex_replay.py — CLI harness (asyncio + websockets +
  httpx, no new pip deps).
- scripts/fixtures/{anthropic_replay_body,codex_response_create_frame}.json
  — hand-crafted, fully synthetic fixtures shaped like real traffic.
- scripts/README.md — "Reproducing the reconnect storm" section.
- tests/test_scripts/test_repro_codex_replay_smoke.py — FastAPI/uvicorn
  mock server; runs the harness end-to-end in < 3s.
- .gitignore — allowlist the new checked-in scripts (existing rule
  previously ignored everything under scripts/ except install.sh/.ps1).
2026-04-20 22:02:02 +07:00
JerrettDavis
d8c2ae88cd ci: validate release workflows with act
Add a workflow-validation CI job that installs actionlint and act,
checks the release and Docker workflows against checked-in event
fixtures, and shares the same validation script developers can run
locally.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-18 15:38:10 -05:00
Tejas Chopra
cb7bc8c815
Merge pull request #191 from JerrettDavis/feat/filesystem-contract
feat: canonical HEADROOM_CONFIG_DIR and HEADROOM_WORKSPACE_DIR filesystem contract
2026-04-16 20:49:24 -07:00
JerrettDavis
4a87753713 feat(docker): forward HEADROOM_WORKSPACE_DIR and HEADROOM_CONFIG_DIR into containers
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 19:19:25 -05:00
JerrettDavis
c24b1fa46f fix: harden release notes and semver
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-16 18:50:36 -05:00
JerrettDavis
bc49e6da16 fix: build TypeScript packages, add OIDC permission, sync SDK dep, scope GPR publish
- publish-pypi: add permissions: id-token: write for OIDC trusted publishing
- publish-npm: add npm run build before npm publish for both packages
- publish-github-packages: add npm run build, use --registry for GPR
- version-sync: add update_openclaw_package_json to sync headroom-ai dep range

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 22:09:05 -05:00
JerrettDavis
3d891264dd feat: add version alignment verification script 2026-04-15 19:37:55 -05:00
JerrettDavis
6299644b6d feat: add changelog generator from conventional commits
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 19:34:50 -05:00
JerrettDavis
93af60ac31 feat: add version synchronization script for multi-package releases 2026-04-15 19:29:15 -05:00
JerrettDavis
865bef2216 fix: harden persistent install wrappers and review gaps
Align Docker-native wrapper help and runtime behavior with the Python install contract, including persistent deployment metadata, baked install-image defaults, and explicit unsupported wrap targets.

Harden the Python persistent-install path with profile validation, safer provider-scope handling, Windows environment restoration, runtime parity improvements, and rollback-safe apply/update behavior.

Update README, Docker install docs, CI, and focused regressions to cover the Windows BOM failure, wrapper parity, compose coverage, and Docker-native wrap behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 18:03:21 -05:00
JerrettDavis
b325a06aae feat: harden persistent install wrappers
Tighten Docker-native bash and PowerShell wrapper validation for wrap and proxy flows, pin the bash wrapper to the install-time interpreter, clean up failed persistent container starts, and extend docs, CI, e2e, and native installer coverage for persistent Docker installs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 15:56:18 -05:00
JerrettDavis
4d9c281749 fix(cli): align native wrapper help and version flags
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 00:40:16 -05:00
JerrettDavis
a1dcda6bc4 feat(cli): support OpenClaw in Docker-native installs
Add host-managed OpenClaw wrap and unwrap flows to the Docker-native wrappers so the installed headroom script can configure the OpenClaw plugin on the host while keeping Headroom itself in Docker. Reuse hidden prepare-only hooks for OpenClaw config payloads, preserve existing plugin metadata on unwrap, and update the Docker-native and integration docs to reflect the supported flow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 00:04:15 -05:00
JerrettDavis
38b1483a76 feat(cli): add Docker-native install flow and parity docs
Add system-native install scripts and host wrappers for running Headroom from Docker while keeping wrapped tools on the host. Document the Docker-native path, add a complete CLI reference with help output and parity details, and add support for root help/version aliases and proxy env-based binding behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-10 23:27:24 -05:00
chopratejas
859a1a49ef Remove dedupe_telemetry.py from repo, add scripts/ to .gitignore
Private scripts with credentials should not be tracked in git.
2026-03-30 20:21:37 -07:00
chopratejas
8228f0edfb Fix streaming tool calls, compressed request bodies, beacon field names; bump to 0.5.13
- Fix LiteLLM stream_message: emit tool_use blocks from delta.tool_calls,
  set stop_reason from finish_reason (fixes silent MCP tool call failures)
- Fix _convert_messages_for_litellm: convert Anthropic tool_result/tool_use
  to OpenAI role=tool/tool_calls format (fixes 500 on tool round-trips)
- Fix proxy request body parsing: decompress zstd/gzip/deflate/brotli
  Content-Encoding before JSON decode (fixes Codex UnicodeDecodeError crash)
- Fix telemetry beacon field names to match /stats endpoint
  (tokens.total_before_compression, not tokens.original)
- Add dedupe_telemetry.py script for hourly Supabase row deduplication
2026-03-30 14:48:48 -07:00