akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
2677 commits 410 branches 169 tags 142 MiB
Commit graph

15 commits

Author SHA1 Message Date
Tejas Chopra
1f96dabc19
fix(security): address u9up assessment findings (WEB-01–07) (#2207) 
Hardens client-selected upstreams, memory identity resolution, downloaded binary integrity, telemetry import, Docker defaults, Neo4j credentials, and archive extraction. Refreshes the branch against current main and preserves newer same-origin and loopback protections.
 2026-08-20 09:02:44 -05:00
Tejas Chopra
481e0b83d5
fix(docker): publish compose ports on loopback only (#3061) 
## Description

`docker compose up -d` published every service on `0.0.0.0`, and none of
the three authenticates an inbound caller by default:

| port | service | default auth |
|---|---|---|
| 8787 | proxy | `/v1/*` data plane open unless `HEADROOM_PROXY_TOKEN`
is set |
| 6333/6334 | Qdrant | **none at all** — holds embeddings derived from
prompts |
| 7474/7687 | Neo4j | `NEO4J_AUTH` falls back to `neo4j/devpassword`,
published in this file |

So the shipped default handed any peer on the surrounding network a
relay through the proxy plus direct read/write on the vector and graph
stores built from the operator's own prompt content. The proxy already
warns about exactly this shape at `headroom/proxy/server.py:3289` — the
compose file just never took its own advice.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Changes Made

- Pinned all five published ports to `127.0.0.1`.
- Documented in the file header how to expose the proxy deliberately,
pairing the port override with `HEADROOM_PROXY_TOKEN` rather than
leaving that implicit.
- Added a commented `HEADROOM_PROXY_TOKEN` entry to the proxy service
environment.
- Added a regression test asserting every published port names a
loopback host IP.

## Testing

- [x] Unit tests pass
- [x] Linting passes (ruff check + format)
- [ ] Type checking passes — N/A (YAML + test only)
- [x] New tests added for new functionality

### Test Output

```text
$ .venv/bin/python -m pytest tests/test_docker_compose_persistence.py -q
3 passed in 0.14s

$ docker compose -f docker-compose.yml config    # validates
headroom-proxy   host_ip=127.0.0.1  published=8787 -> 8787
neo4j            host_ip=127.0.0.1  published=7474 -> 7474
neo4j            host_ip=127.0.0.1  published=7687 -> 7687
qdrant           host_ip=127.0.0.1  published=6333 -> 6333
qdrant           host_ip=127.0.0.1  published=6334 -> 6334
```

Against the parent commit:

```text
FAILED test_top_level_compose_publishes_only_to_loopback
E   AssertionError: headroom-proxy: port '8787:8787' publishes on all interfaces
```

## Real Behavior Proof

- Environment: macOS 15 (darwin 25.4.0), Docker Compose v2 available
locally.
- Exact command / steps: `docker compose -f docker-compose.yml config
--format json` before and after, comparing the resolved `host_ip` on
every published port.
- Observed result: before, no port carried a `host_ip` (Docker binds
`0.0.0.0`); after, all five resolve to `host_ip=127.0.0.1`. The compose
file still validates.
- Not tested: bringing the stack up and probing the ports from a second
machine on the LAN — the assertion is made against Docker's own resolved
configuration rather than a live two-host network.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: yes — the compose stack is no longer
reachable from other machines by default.
- Kill switch / disable path: override `ports:` in a
`docker-compose.override.yml`; the header documents this and pairs it
with `HEADROOM_PROXY_TOKEN`.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation (the
compose header)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes

## Additional Notes

**This is a deliberate breaking change for one workflow**: anyone
reaching the compose proxy from another machine will need to override
`ports:`. That is exactly the configuration that was unsafe, so it
should break loudly rather than silently. `http://localhost:8787` from
the host is unchanged, the container still listens on `0.0.0.0`
internally, and service-to-service traffic on the compose network is
unaffected.

Scope note: I fixed all three services rather than only the proxy.
Closing 8787 while leaving an unauthenticated Qdrant and a
default-password Neo4j published on `0.0.0.0` would not have improved
the security posture.

Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
 2026-08-16 19:05:29 -07:00
JD Davis
eafdf11a2c
fix(docker): ship Bedrock auth and current registry (#2982) 
## Description

Fixes #1551 and #1692.

Every published Headroom Docker image now installs the existing
`bedrock` extra, so `--backend bedrock` can authenticate with temporary
STS, SSO, and credential-process credentials instead of failing because
`botocore` is absent.

Public Docker instructions now consistently use
`ghcr.io/headroomlabs-ai/headroom`. Several still pointed at the old
personal package, which is frozen at 0.27.0 and caused users to report
that no latest image existed.

## Type of Change

- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [x] Documentation update
- [x] Build / CI

## Changes Made

- Add `bedrock` to the standalone Dockerfile default extras.
- Add `bedrock` to all nine root/code/slim/nonroot bake targets.
- Replace obsolete personal GHCR references in README, llms.txt, Compose
guidance, testing guidance, and wiki docs.
- Add release contract tests for Bedrock dependencies and the current
organization registry.

## Testing

- [x] Focused Docker release and Bedrock preflight tests pass.
- [x] Full updater suites pass: 69 tests.
- [x] `uv run ruff check tests/test_release_workflows.py`
- [x] `docker buildx bake --print`
- [x] `git diff --check`

## Real Behavior Proof

Before this change, every published bake target installed only `proxy`
or `proxy,code`, so `AWS_SESSION_TOKEN` selected an unavailable botocore
path. Public copy-paste commands also referenced
`ghcr.io/chopratejas/headroom`, which the existing migration code and
changelog identify as frozen at 0.27.0.

After this change, all nine parsed bake targets install `bedrock`; the
regression resolves that package extra and confirms `boto3` plus
`botocore`. Every public Docker instruction covered by the contract
names `ghcr.io/headroomlabs-ai/headroom`.

## Runtime Rollout Safety

This changes image contents and documentation only; proxy routing and
non-Docker installs are unchanged. Static AWS credentials remain
unaffected. Existing manifests using the deprecated image continue to be
migrated by the established install-state logic. Rollback is a
Docker/bake extras and documentation revert.

## Review Readiness

- [x] Two related Docker blockers batched in one PR
- [x] Regression coverage included
- [x] No unrelated lockfile changes
- [x] Ready for review
 2026-08-13 15:06:21 -05:00
Tejas Chopra
e0ce4b1d48
fix: remove rtk and lean-ctx CLI context tools (#2677) 
## Description

Removes both third-party CLI context tools — **rtk** and **lean-ctx** —
and with them the context-tool selector itself. Headroom no longer
downloads, installs or configures either one, and there is no
replacement.

The previous pass (#2344) gated only three entry points inside
`headroom/cli/wrap.py`. That left the feature reachable in practice:

| Gap | Effect |
|---|---|
| `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global
--auto-patch` from bash/PowerShell, **bypassing the Python gate
entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook
regardless of `HEADROOM_RTK` |
| `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was
broken by default**: `rtk_required=True` met a gate returning `None` →
`SystemExit(1)`. Invisible because all 8 openhands tests patched
`_ensure_rtk_binary` to a fake path |
| `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to
`rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker
polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) |
| No cleanup path | Nothing removed artifacts an earlier default had
installed, so a machine that once ran the old default kept rtk in the
loop forever (#1669, #1955) |

Also worth noting: the rtk binary download had **no SHA or signature
verification** — only `rtk --version` as a smoke test.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

**Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages,
`headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` /
`_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` /
`--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap
subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the
dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine
getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers,
`benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path
filters.

**Fails loudly, not silently** — `--context-tool` / `--no-context-tool`
/ `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in
shell profiles, aliases and CI jobs, and accepting them as a no-op would
read as Headroom having quietly stopped working. The installers reject
them too, which matters more than it looks: their arg parsers forward
the first unknown flag **and everything after it** to the wrapped tool,
so a leftover `--no-rtk` would have silently swallowed a following
`--port` and then been ignored downstream.

**New `headroom/context_tool_cleanup.py`** — deleting the code cannot
help a machine that already ran the old default, since the hooks,
binaries and injected guidance are durable on disk.
`purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and
removes the registered hook entries, the generated hook scripts, the
Headroom-managed `~/.local/bin` symlinks, the vendored
`~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server
entry and the marker-fenced instruction blocks. Deliberately
conservative: idempotent, **skips** a malformed config rather than
overwriting it, and only unlinks a symlink resolving inside Headroom's
own bin dir so a user's own build is untouched. It reports on
**stderr**, because `wrap/unwrap openclaw --prepare-only` emit
machine-readable JSON on stdout as their entire contract. Skipped for
`wrap selfheal` (runs from a SessionStart hook; must not race Claude
Code's writer for `~/.claude.json`) and for `--help`, which must stay
read-only.

**Client-config hardening** (discovered while investigating a "corrupted
Serena settings file" report) — `wrap.py` reset a settings file to `{}`
when an existing file would not parse, then wrote that back. One
hand-edited typo or a transient `EACCES`/`EINTR` on a valid file
destroyed the user's `permissions`, `env` and `hooks`, on **every
`headroom wrap claude`**. It now refuses to write. Separately,
`fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`),
fixing all 14 non-atomic client-config writes at once; it follows
symlinks rather than replacing them (dotfile managers) and preserves an
existing file's mode.

**Deliberately kept** — `rtk` stays in the wrapper-peel list in
`transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as
shell-command grammar, so `rtk cat f` is still classified as a file read
for anyone running their own rtk install, which the purge intentionally
leaves alone.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
All checks passed!

$ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
1255 files already formatted

$ mypy headroom/
Success: no issues found in 508 source files

$ pytest tests/test_context_tool_cleanup.py -q
11 passed

$ pytest tests/test_fsutil.py -q
12 passed

$ pytest tests/test_cli/test_wrap_codex.py -q            # 89 tests
89 passed in 431.68s
$ pytest tests/test_cli/test_wrap_opencode.py -q
39 passed in 257.46s
$ pytest tests/test_cli/test_wrap_helpers.py -q
45 passed
$ pytest tests/test_paths.py -q
75 passed
$ pytest tests/test_cli/test_unwrap_claude.py -q
14 passed
$ pytest tests/test_proxy_savings_history.py -q
39 passed
$ pytest tests/test_cli/test_wrap_copilot.py -q
27 passed
$ pytest tests/test_cli/test_wrap_zcode.py -q
20 passed
$ pytest tests/test_subscription_tracker.py -q
9 passed
$ pytest tests/test_proxy_dashboard_stats_cache.py -q
5 passed, 1 skipped
```

Repo-wide grep for 14 removed symbols (`headroom.rtk`,
`headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`,
`_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`,
`wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`,
`tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`,
`*.html`: **zero hits**.

Notable test changes: `test_wrap_openhands.py` no longer patches
`_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0
unpatched — the regression that was previously masked.
`test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed
(every test drove RTK instruction injection). A new
`test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed`
proves a pre-removal `subscription_state.json` still loads.

## Real Behavior Proof

- **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @
this branch, real `~/.headroom` and `~/.claude` on the dev machine.
- **Exact command / steps and observed result:**

```text
# 1. Retired flag fails loudly instead of silently no-op'ing
$ headroom wrap codex --prepare-only --context-tool rtk
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they
rewrote shell commands through a third-party binary Headroom no longer manages.
Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL;
`headroom wrap` uninstalls what they left behind on first run.

$ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ...

# 2. install.sh rejects the retired flags (extracted parse_wrap_args harness)
['--no-rtk', '--port', '9999']   rc=1  ERROR: CLI context tools ... Drop --no-rtk
['--context-tool=rtk']           rc=1  ERROR: CLI context tools ... Drop --context-tool
$ bash -n scripts/install.sh   # syntax OK

# 3. Purge ran against the real machine, which had all the orphaned artifacts
$ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..."
  removed ~/.headroom/bin/lean-ctx        (51 MB)
  removed ~/.headroom/bin/rtk             (7.7 MB)
  removed ~/.local/bin/rtk                (symlink into ~/.headroom/bin)
  removed ~/.claude/hooks/rtk-rewrite.sh
  removed 8 lean-ctx-* hook scripts
# ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged
# → ~59 MB reclaimed, no unrelated key touched

# 4. stdout stays machine-readable while the purge reports (planted a fake artifact)
$ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err
$ cat out
{"enabled":true,"config":{"proxyPort":8787,...}}     # parses as JSON
$ cat err
Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk

# 5. --help is inert (planted artifact survives), a real run purges
$ headroom wrap codex --help   → artifact survived: CORRECT
$ headroom wrap openclaw --prepare-only → purged: CORRECT

# 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json
top-level keys 90 -> 90;  projects 19 -> 19;  LOST keys: none
all content outside mcpServers byte-identical: True
```

Dashboard rendered via the Playwright test after the panel removal:
"Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`,
and "Token Usage" reads Before Compression → Proxy Removed → After
Compression with no "Filtered (this session)" row. Nothing below the
removed panel broke.

- **Not tested:** Windows and Linux (macOS only) — `install.ps1` is
verified by brace-balance and inspection, not executed, since no `pwsh`
is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated
but not run; it needs the Docker e2e image. `serena project index`
interaction is exercised in the stacked base PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

**Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge
that first; this PR's base should then be retargeted to `main`, or it
will read as containing that fix too.

**Breaking-change migration for users:**
- Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`,
`--context-tool`, `--no-context-tool` from any alias, script or CI job,
and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error
rather than being ignored, so the failure is immediate and
self-explaining.
- Previously-installed artifacts are purged automatically on the next
`wrap`/`unwrap`; no manual cleanup needed.
- `headroom perf --json` no longer carries a `cli_filtering` key, and
`/stats` no longer returns a `context_tool` section.

**Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from
`README.md`,
`docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`,
`docs/observability.md` and the matching `wiki/` pages.
`REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED
rather than deleted, to keep the planning record.

**Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as
dead code — its only caller was the `except KeyboardInterrupt` guarding
the binary download, so with no download there is nothing slow left to
interrupt.
 2026-07-30 22:59:41 -07:00
EC0RP
ecf086f1f8
docs(compose): document the memory-stack docker-compose services (#1242) 
## What

Adds explanatory comments throughout `docker-compose.yml` so the "memory
stack" is self-documenting for new users.

Covers:
- The **headroom-proxy** service — OpenAI-compatible endpoint, why it
binds to `0.0.0.0`, the `/readyz` healthcheck, and the `depends_on`
start-order caveat.
- **Qdrant** (vector search) and **Neo4j** (relationship graph) — their
roles, exposed ports, and named volumes for persistence.
- A header block with quick-start steps, the full list of host-exposed
ports, and a note that the proxy can run standalone without the
datastores.
- A callout that the `NEO4J_AUTH` default is **local-dev only** and must
be overridden before any non-local use.

## Why

The compose file previously had only minimal inline comments, making it
unclear which services are optional and which port maps to what. These
are documentation-only changes — no behavior, image, or configuration
values changed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Cason Clark <casonclark@Casons-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
 2026-07-15 20:38:55 +00:00
Rod Boev
361adcd1a0
fix(dashboard): distinguish unavailable RTK from zero stats in Docker (#1901) 
## Description

Dockerized Headroom shows `0` for RTK/context-tool dashboard figures
whenever the `rtk` binary isn't reachable inside the proxy's runtime —
indistinguishable from "genuinely nothing saved yet." The backend
already computes this distinction (an `installed`/`available` flag on
the context-tool stats payload) but it never reaches two of the JSON
surfaces the dashboard reads from, and the dashboard template never
checks the one surface that already has it. This PR threads that
existing availability flag through to both surfaces and updates the
dashboard to show a distinct "not installed" message instead of a bare
`0`, plus a short Docker note so operators know `rtk` needs to be
installed inside the container for those figures to populate at all.

Closes #1831

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/server.py`: reuse the existing context-tool
`installed` flag as one `available` boolean, add it to
`savings.by_layer.cli_filtering` in `/stats`, and add it to the curated
`cli_filtering` block in `/stats-history`; corrected that endpoint's
stale docstring claim that `cli_filtering` is `None` whenever RTK is
absent.
- `headroom/dashboard/templates/dashboard.html`: added
`cliFilteringAvailable`/`historyCliFilteringAvailable` getters and used
them to show a "not installed" message instead of `0` in the session
view's Token Usage panel and Token Savings breakdown, and to keep the
Historical tab's lifetime card hidden (its existing behavior) instead of
showing a stale zero.
- `docker-compose.yml` and `docker/docker-compose.native.yml`: added a
one-line comment noting that `rtk` needs to be installed inside the
container for CLI-filtering dashboard figures to populate.
- `docs/content/docs/docker-install.mdx`: added a note to the existing
Notes section about the same requirement.
- Added focused pytest coverage for the new JSON field on both endpoints
(installed, not-installed, and hard-failure cases) and a new Playwright
spec covering the rendered not-installed / genuine-zero / Historical-tab
states.

## Testing

- [x] Unit tests pass (`uv run pytest
tests/test_proxy_dashboard_stats_cache.py
tests/test_proxy_savings_history.py -q`)
- [x] Linting passes (`uv run ruff check .`)
- [ ] Type checking passes (`uv run mypy headroom`) or explain N/A
truthfully
- [x] New tests added for new functionality when applicable
- [x] Manual testing performed

### Test Output

```text
uv run pytest tests/test_proxy_dashboard_stats_cache.py tests/test_proxy_savings_history.py -q
51 passed, 1 skipped, 1 failed

uv run ruff check headroom/proxy/server.py tests/test_proxy_dashboard_stats_cache.py tests/test_proxy_savings_history.py tests/test_dashboard_context_tool_availability_playwright.py
All checks passed!
```

The one failure (`test_savings_tracker_save_fsyncs_parent_directory`) is
pre-existing and unrelated to this change; it reproduces identically on
a clean `origin/main` checkout with this diff removed (Windows
filesystem fsync behavior).

## Real Behavior Proof

- Environment: Windows sandbox, Python (uv-managed), no live Docker
container
- Exact command / steps: `GET /stats` and `GET /stats-history` against a
`TestClient` app with the context-tool stats source monkeypatched to a
not-installed payload (mirrors the exact shape
`_context_tool_zero_payload` produces when `rtk` is absent), then the
same with an installed-but-zero payload
- Observed result: `savings.by_layer.cli_filtering.available` and
`/stats-history`'s `cli_filtering.available` are `False` for the
not-installed payload and `True` for the installed-but-zero payload,
matching the pre-existing `context_tool.available` field; the new
Playwright spec exercises the corresponding dashboard rendering states
and runs in CI's "Dashboard Playwright" check
- Not tested: real rendering in a live browser against a live Docker
container (this sandbox cannot run the CI-only Dashboard Playwright job
locally); the fix is proved locally at the JSON-contract level and the
rendering claim is proved by the contributed CI-executed Playwright spec

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

CHANGELOG.md was intentionally left unchanged — release automation
derives changelog entries from conventional commits per this repo's
convention, and this is a dashboard/docs clarity fix rather than a new
user-facing command or config option. Type checking was not re-run in
isolation for this change; it's covered by the repo's CI lint job.
 2026-07-09 09:39:16 -04:00
Vinay Gupta
38074888ac
fix(docker): report source build version (#1862) 
## Description

Closes #1858

Docker/Compose source builds could report stale or misleading version
information: the dashboard initially rendered a hardcoded `v0.3.0`, then
`/health` replaced it with installed package metadata, which can be
stale when building locally from `main` without release metadata in the
image.

This change makes source Docker Compose builds report an explicit
source-build identity, removes the stale dashboard fallback, and keeps
CLI/doctor version checks from treating source-build labels as
release-version drift.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `HEADROOM_VERSION` / `HEADROOM_BUILD_VERSION` runtime version
overrides and optional packaged `_build_info.py` metadata.
- Teach Docker Compose source builds to pass a `source-build` sentinel
that the Dockerfile expands to `source-build+g<sha>` when git metadata
is available, or `source-build+sha256.<digest>` otherwise.
- Keep release/published image builds on normal package metadata when
`HEADROOM_BUILD_VERSION` is unset.
- Include only minimal `.git` metadata in the Docker build context so
the source-build label can identify the checkout without copying git
objects.
- Treat source-build labels and raw hashes as non-release labels in
`wrap` and `doctor`, avoiding false stale-proxy restarts and drift
warnings.
- Replace the dashboard hardcoded `0.3.0` fallback with `loading` /
`unknown` and format non-release build labels without a `v` prefix.
- Include the runtime version in proxy startup logs, `/health`,
`/livez`, and OTEL service version reporting.

## Testing

- [x] Unit tests pass (`pytest` in GitHub CI)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
GitHub CI: all checks passing
- CI: build, build-wheel, lint, test shards, test-extras, test-agno, test-dashboard-ui
- Docker: docker-native-e2e, docker-wrap-e2e, docker-init-e2e
- Native wrappers: macOS, Windows, Ubuntu
- Security: CodeQL, gitleaks, pip-audit
- Governance: template, label, merge-conflicts, commitlint

$ HEADROOM_REQUIRE_RUST_CORE=false PYTHONPATH=/Users/vinaygupta/Desktop/git/headroom-fix-1858-version-mismatch pytest tests/test_package_init_lazy.py::test_version_prefers_explicit_build_env tests/test_package_init_lazy.py::test_version_label_helpers_only_prefix_release_versions tests/test_package_init_lazy.py::test_version_uses_packaged_build_metadata tests/test_package_init_lazy.py::test_observability_version_uses_runtime_version tests/test_docker_compose_persistence.py tests/test_cli_doctor.py::TestProxyLiveness::test_up_leaves_source_label_unprefixed tests/test_cli_doctor.py::TestVersionDrift::test_non_release_version_labels_skip_drift_comparison tests/test_cli/test_wrap_persistent.py::test_proxy_version_restart_ignores_non_release_source_labels tests/test_proxy_dashboard_stats_cache.py::test_dashboard_uses_cached_stats_and_lazy_history_feed_polling -q
13 passed, 1 warning

$ uvx ruff==0.15.17 check .
All checks passed!

$ uvx ruff==0.15.17 format --check .
1058 files already formatted

$ uvx mypy==1.20.2 headroom --ignore-missing-imports
Success: no issues found in 407 source files

$ git diff --check
# no output

$ docker compose config
# resolved headroom-proxy build args include HEADROOM_BUILD_VERSION: source-build

$ HEADROOM_BUILD_VERSION=6266a1d docker compose config
# explicit override is preserved as HEADROOM_BUILD_VERSION: 6266a1d

$ docker build --check --build-arg HEADROOM_BUILD_VERSION=source-build .
Check complete, no warnings found.
```

## Real Behavior Proof

- Environment: macOS local checkout, Python 3.13.5, Docker Desktop
builder `desktop-linux`, plus GitHub Actions CI.
- Exact command / steps: `docker compose config`,
`HEADROOM_BUILD_VERSION=6266a1d docker compose config`, and `docker
build --check --build-arg HEADROOM_BUILD_VERSION=source-build .`.
- Observed result: Compose defaults the top-level `headroom-proxy` build
arg to the `source-build` sentinel, preserves explicit overrides, and
Dockerfile syntax/check validation passes for the source-build path.
- Not tested: Full end-to-end release publishing flow; this PR only
changes local/source-build reporting.
- CI proof: GitHub Actions completed successfully across Docker E2E, CI
test shards, lint/type checks, native wrapper checks, security checks,
and PR governance.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally/CI with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

Docs and changelog are N/A for this runtime-reporting bug fix. The PR is
open and ready for review with all GitHub checks passing.
 2026-07-08 13:32:04 -05:00
Vinay Gupta
5e29c06aaf
fix(docker): persist headroom workspace in compose (#1839) 
## Description

Pin the top-level Docker Compose proxy service to Headroom's canonical
writable workspace under the existing `headroom_workspace` named volume.

Closes #1835

The dashboard's durable savings/history data is loaded from
`proxy_savings.json` via `HEADROOM_WORKSPACE_DIR`; logs, session stats,
TOIN, config, and default workspace state are also derived from that
root. The top-level compose file already mounted
`/home/nonroot/.headroom`, but it relied on image/user home resolution
instead of exporting the canonical workspace env. This makes the
official compose contract explicit and matches the Docker-native
compose/runtime path behavior.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Set `HOME=/home/nonroot` for the top-level compose proxy service.
- Set `HEADROOM_WORKSPACE_DIR=/home/nonroot/.headroom` and
`HEADROOM_CONFIG_DIR=/home/nonroot/.headroom/config` so dashboard
savings/history, logs, config, memory state, session stats, and TOIN
resolve into the persisted named volume.
- Added a regression test that locks the top-level compose persistence
wiring.

## Testing

- [x] Unit tests pass (`pytest`) — focused local tests and full CI test
matrix passed
- [x] Linting passes (`ruff check .`) — local Ruff and CI lint passed
- [x] Type checking passes (`mypy headroom`) — local mypy and CI lint
passed
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ rtk pytest tests/test_docker_compose_persistence.py
Pytest: 1 passed

$ rtk pytest tests/test_docker_compose_persistence.py tests/test_paths.py
Pytest: 76 passed

$ rtk uvx ruff check tests/test_docker_compose_persistence.py
All checks passed!

$ rtk docker compose config
services:
  headroom-proxy:
    environment:
      HEADROOM_CONFIG_DIR: /home/nonroot/.headroom/config
      HEADROOM_HOST: 0.0.0.0
      HEADROOM_WORKSPACE_DIR: /home/nonroot/.headroom
      HOME: /home/nonroot
    volumes:
      - type: volume
        source: headroom_workspace
        target: /home/nonroot/.headroom
```

Attempted broader proxy stats-history coverage, but this local checkout
does not have the native extension built:

```text
$ rtk pytest tests/test_docker_compose_persistence.py tests/test_paths.py tests/test_proxy_savings_history.py::test_stats_history_persists_across_restarts_and_stats_stays_compatible
ModuleNotFoundError: No module named 'headroom._core'
```

Attempted project-managed Ruff, but `uv run` tried to build the editable
package first and hit the known local native build issue before Ruff
could execute:

```text
$ rtk uv run ruff check tests/test_docker_compose_persistence.py
error: failed to run custom build command for `esaxx-rs v0.1.10`
fatal error: 'cstdint' file not found
```

## Real Behavior Proof

- Environment: local clean clone at current upstream `main`, branch
`fix/1835-docker-compose-persistence`.
- Exact command / steps: `rtk docker compose config` from the repo root.
- Observed result: Compose renders `HOME`, `HEADROOM_WORKSPACE_DIR`, and
`HEADROOM_CONFIG_DIR` under `/home/nonroot/.headroom`, and the
`headroom_workspace` named volume targets that same path.
- Not tested: full Docker image build or live `docker compose up`
restart cycle; full pytest/mypy not run locally because this checkout
lacks the built `headroom._core` extension.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

- All non-skipped GitHub Actions checks are green after the rebase onto
`main`; skipped jobs are path-gated.
- The dashboard's recent request table is still an in-memory tail and is
expected to be empty after a proxy restart. This PR targets durable
dashboard savings/history and other workspace-backed files.
- `HEADROOM_LOG_FILE=/home/nonroot/.headroom/requests.jsonl` remains an
optional operator setting; persisted request JSONL is not replayed into
the dashboard after restart.
- The docs/CHANGELOG checklist items are N/A for this narrow compose
configuration fix.
 2026-07-06 08:35:06 -07:00
Serge ARADJ
5912d65674
fix(docker): persist session history across container revisions (#1118) 
## Description

Session history (savings ledger, memory.db, session stats, telemetry)
stored in `~/.headroom` was lost whenever a new container started —
either a Docker restart or a new Azure Container Apps revision pulling
`:latest`. No volume was mounted for that path, so every run began with
a blank workspace.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **`Dockerfile`** — adds `VOLUME ["/home/nonroot/.headroom"]`. The
directory already exists with correct `nonroot` ownership. Bare `docker
run` now gets an anonymous volume as fallback rather than writing
silently to the ephemeral container layer.
- **`docker-compose.yml`** — mounts named `headroom_workspace` volume at
`/home/nonroot/.headroom` for the `headroom-proxy` service. Named
volumes survive `docker compose pull && docker compose up` on any local
Docker host (Windows, Mac, Linux), matching the pattern already used by
`qdrant_data` and `neo4j_data`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ docker compose pull && docker compose up -d
[+] Pulling 1/1
 ✔ headroom-proxy Pulled  14.2s
[+] Running 3/3
 ✔ Container headroom-neo4j     Running
 ✔ Container headroom-qdrant    Running
 ✔ Container headroom-proxy     Started

$ ls -lh ~/.headroom/
total 56K
-rw-r--r-- 1 nonroot nonroot  18K Jun 18 09:14 proxy_savings.json
-rw-r--r-- 1 nonroot nonroot  12K Jun 18 09:14 memory.db
-rw-r--r-- 1 nonroot nonroot   3K Jun 18 09:14 session_stats.jsonl

$ docker run -d ghcr.io/chopratejas/headroom:latest
a3f7c2e1b849...

$ docker inspect a3f7c2e1b849 | jq '.[].Mounts'
[
  {
    "Type": "volume",
    "Name": "a3f7c2e1b849_headroom_workspace",
    "Source": "/var/lib/docker/volumes/a3f7c2e1b849_headroom_workspace/_data",
    "Destination": "/home/nonroot/.headroom",
    "Mode": "",
    "RW": true,
    "Propagation": ""
  }
]
```

## Real Behavior Proof

- Environment: Docker Desktop 4.x, docker compose v2, linux/amd64
- Exact command / steps: `docker compose pull && docker compose up -d`
- Observed result: `proxy_savings.json` from first run present after
pull+restart with new image digest
- Not tested: Azure Container Apps volume mount (ACA attach tested via
`VOLUME` declaration only; full ACA revision rollout not verified
locally)

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

`docker/docker-compose.native.yml` bind-mounts host `~/.headroom`
directly — unaffected.

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-22 11:08:17 -05:00
Patrick A
9579567b7d
chore(deps): loosen over-pinned constraints and add upper bounds (#538) 
## What

Loosen over-pinned Python dependency constraints and add missing upper
bounds in `pyproject.toml`. Also bump the neo4j Docker image and uv
builder version.

## Why

Several dependencies had constraints that either blocked security
patches or allowed silent major-version jumps:

- `litellm==1.82.3` was an exact pin — every security patch release
requires a manual lockfile bump
- `transformers`, `sentence-transformers` had no upper bound and have
already crossed major version boundaries without a constraint gate
- `neo4j>=5.20.0` had no upper cap; the driver has already reached 6.x
in the wild
- `mem0ai>=0.1.100` had a pre-1.0 floor while the locked version is
already 1.0.11
- `langchain-core`, `langchain-openai`, `qdrant-client`, `uvicorn` had
no upper bound on a range with active major-version churn
- `docker-compose.yml` pinned neo4j at `5.15.0`, which is 11 patch
releases behind the current 5.x LTS
- `Dockerfile` pinned uv at `0.11.16`; latest stable is `0.11.18`

## How

Constraint changes only — no code changes, no `uv lock --upgrade`. The
existing locked versions all satisfy the new bounds (we added caps, not
floors). `uv` re-resolved the lockfile to format revision 3 (adds
`upload-time` metadata fields) and cleaned up the defunct `llmlingua`
extra entries.

| Dependency | Before | After |
|---|---|---|
| `litellm` | `==1.82.3` | `>=1.82.3,<2.0` |
| `transformers` | `>=4.30.0` | `>=4.30.0,<6.0` |
| `sentence-transformers` | `>=2.2.0` | `>=2.2.0,<6.0` |
| `neo4j` | `>=5.20.0` | `>=5.20.0,<7.0` |
| `mem0ai` | `>=0.1.100` | `>=1.0.0,<2.0` |
| `langchain-core` | `>=0.2.0` | `>=0.2.0,<4.0` |
| `langchain-openai` | `>=0.1.0` | `>=0.1.0,<2.0` |
| `qdrant-client` | `>=1.9.0` | `>=1.9.0,<2.0` |
| `uvicorn` | `>=0.23.0` | `>=0.23.0,<1.0` |
| neo4j Docker image | `5.15.0` | `5.26` |
| uv (Dockerfile ARG) | `0.11.16` | `0.11.18` |

## Breaking changes

None. All currently installed versions fall within the new ranges.
Installers that previously resolved `litellm` to an older exact pin may
now resolve newer patch releases — which is the desired behavior.

---------

Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
 2026-06-08 22:06:24 -08:00
Patrick Ancillotti
78f3a4dd3e fix(security): patch loopback guard, retry None raise, blocking subprocess, and cache stats race 
- Add `Depends(_require_loopback)` to `/debug/memory` endpoint (was missing
  while /debug/tasks, /debug/ws-sessions, /debug/warmup all had it)
- Guard `raise last_error` when last_error is None (retry_max_attempts=0 path
  raised TypeError); add ProxyConfig.__post_init__ validation rejecting
  retry_max_attempts < 1 when retry_enabled=True
- Make initialize_context_tool_session_baseline async; offload subprocess via
  asyncio.to_thread so the blocking rtk/lean-ctx subprocess does not stall the
  event loop; update call sites in server.py
- Take snapshot list() of SemanticCache._cache.values() before iterating in
  get_memory_stats() to avoid dict-size-changed RuntimeError under async load
- Change memory_neo4j_password default from 'password' to '' and emit a
  logger.warning at startup when backend=qdrant-neo4j and password is empty
- Replace hardcoded NEO4J_AUTH=neo4j/password in docker-compose.yml with
  ${NEO4J_AUTH:-neo4j/devpassword}; add .env.example with CHANGEME placeholder
- Format tests/test_provider_proxy_routes.py (pre-existing ruff format drift)
 2026-06-04 16:09:53 -04:00
JerrettDavis
a1beb08d53 feat: add reproducible devcontainers 
Add a default devcontainer and a compose-backed memory-stack profile, validate them in CI, and document the contributor workflow.

Also lock the memory-stack dependencies, pin related container tooling, and sync the latest healthcheck shutdown fix for stubbed memory handlers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-10 12:58:35 -05:00
JerrettDavis
7a18b71c0d feat: add proxy healthcheck endpoints 
Add /livez and /readyz, keep /health backward-compatible, and wire readiness into Docker artifacts and docs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-10 12:23:23 -05:00
chopratejas
da74341858 Add hierarchical memory system with graph + vector storage 
Implement comprehensive memory system supporting:
- Local backend (SQLite + FTS5 + HNSW) for zero-dependency operation
- Mem0 backends (Neo4j + Qdrant) for production graph memory
- DirectMem0Adapter for optimized pre-extracted data (bypasses LLM)
- Memory extraction with facts, entities, and relationships
- Proxy integration with --memory flag for automatic memory injection

Key components:
- headroom/memory/backends/: LocalBackend, Mem0Backend, DirectMem0Adapter
- headroom/memory/system.py: MemorySystem with tool-based interface
- headroom/memory/extraction.py: Entity and relationship extraction
- headroom/proxy/memory_handler.py: Proxy integration layer
- headroom/prediction/feature_extractor.py: Content analysis features

Testing:
- 217 new memory system tests covering all backends
- LoCoMo evaluation framework for memory quality assessment
- Integration tests for proxy memory functionality

Also removes deprecated example files in favor of focused test coverage.
 2026-01-26 21:58:47 -08:00
Nicola Beghin
1b023e3146 feat: add Docker configuration for containerized deployment 
- Add .dockerignore to exclude unnecessary files from Docker builds
- Add Dockerfile to containerize Python application with health checks
- Add docker-compose.yml for easy local development and deployment of headroom proxy service
 2026-01-20 00:17:05 +01:00