Commit graph

9 commits

Author SHA1 Message Date
Vinay Gupta
38074888ac
fix(docker): report source build version (#1862)
## Description

Closes #1858

Docker/Compose source builds could report stale or misleading version
information: the dashboard initially rendered a hardcoded `v0.3.0`, then
`/health` replaced it with installed package metadata, which can be
stale when building locally from `main` without release metadata in the
image.

This change makes source Docker Compose builds report an explicit
source-build identity, removes the stale dashboard fallback, and keeps
CLI/doctor version checks from treating source-build labels as
release-version drift.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `HEADROOM_VERSION` / `HEADROOM_BUILD_VERSION` runtime version
overrides and optional packaged `_build_info.py` metadata.
- Teach Docker Compose source builds to pass a `source-build` sentinel
that the Dockerfile expands to `source-build+g<sha>` when git metadata
is available, or `source-build+sha256.<digest>` otherwise.
- Keep release/published image builds on normal package metadata when
`HEADROOM_BUILD_VERSION` is unset.
- Include only minimal `.git` metadata in the Docker build context so
the source-build label can identify the checkout without copying git
objects.
- Treat source-build labels and raw hashes as non-release labels in
`wrap` and `doctor`, avoiding false stale-proxy restarts and drift
warnings.
- Replace the dashboard hardcoded `0.3.0` fallback with `loading` /
`unknown` and format non-release build labels without a `v` prefix.
- Include the runtime version in proxy startup logs, `/health`,
`/livez`, and OTEL service version reporting.

## Testing

- [x] Unit tests pass (`pytest` in GitHub CI)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
GitHub CI: all checks passing
- CI: build, build-wheel, lint, test shards, test-extras, test-agno, test-dashboard-ui
- Docker: docker-native-e2e, docker-wrap-e2e, docker-init-e2e
- Native wrappers: macOS, Windows, Ubuntu
- Security: CodeQL, gitleaks, pip-audit
- Governance: template, label, merge-conflicts, commitlint

$ HEADROOM_REQUIRE_RUST_CORE=false PYTHONPATH=/Users/vinaygupta/Desktop/git/headroom-fix-1858-version-mismatch pytest tests/test_package_init_lazy.py::test_version_prefers_explicit_build_env tests/test_package_init_lazy.py::test_version_label_helpers_only_prefix_release_versions tests/test_package_init_lazy.py::test_version_uses_packaged_build_metadata tests/test_package_init_lazy.py::test_observability_version_uses_runtime_version tests/test_docker_compose_persistence.py tests/test_cli_doctor.py::TestProxyLiveness::test_up_leaves_source_label_unprefixed tests/test_cli_doctor.py::TestVersionDrift::test_non_release_version_labels_skip_drift_comparison tests/test_cli/test_wrap_persistent.py::test_proxy_version_restart_ignores_non_release_source_labels tests/test_proxy_dashboard_stats_cache.py::test_dashboard_uses_cached_stats_and_lazy_history_feed_polling -q
13 passed, 1 warning

$ uvx ruff==0.15.17 check .
All checks passed!

$ uvx ruff==0.15.17 format --check .
1058 files already formatted

$ uvx mypy==1.20.2 headroom --ignore-missing-imports
Success: no issues found in 407 source files

$ git diff --check
# no output

$ docker compose config
# resolved headroom-proxy build args include HEADROOM_BUILD_VERSION: source-build

$ HEADROOM_BUILD_VERSION=6266a1d docker compose config
# explicit override is preserved as HEADROOM_BUILD_VERSION: 6266a1d

$ docker build --check --build-arg HEADROOM_BUILD_VERSION=source-build .
Check complete, no warnings found.
```

## Real Behavior Proof

- Environment: macOS local checkout, Python 3.13.5, Docker Desktop
builder `desktop-linux`, plus GitHub Actions CI.
- Exact command / steps: `docker compose config`,
`HEADROOM_BUILD_VERSION=6266a1d docker compose config`, and `docker
build --check --build-arg HEADROOM_BUILD_VERSION=source-build .`.
- Observed result: Compose defaults the top-level `headroom-proxy` build
arg to the `source-build` sentinel, preserves explicit overrides, and
Dockerfile syntax/check validation passes for the source-build path.
- Not tested: Full end-to-end release publishing flow; this PR only
changes local/source-build reporting.
- CI proof: GitHub Actions completed successfully across Docker E2E, CI
test shards, lint/type checks, native wrapper checks, security checks,
and PR governance.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally/CI with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

Docs and changelog are N/A for this runtime-reporting bug fix. The PR is
open and ready for review with all GitHub checks passing.
2026-07-08 13:32:04 -05:00
Vinay Gupta
5e29c06aaf
fix(docker): persist headroom workspace in compose (#1839)
## Description

Pin the top-level Docker Compose proxy service to Headroom's canonical
writable workspace under the existing `headroom_workspace` named volume.

Closes #1835

The dashboard's durable savings/history data is loaded from
`proxy_savings.json` via `HEADROOM_WORKSPACE_DIR`; logs, session stats,
TOIN, config, and default workspace state are also derived from that
root. The top-level compose file already mounted
`/home/nonroot/.headroom`, but it relied on image/user home resolution
instead of exporting the canonical workspace env. This makes the
official compose contract explicit and matches the Docker-native
compose/runtime path behavior.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Set `HOME=/home/nonroot` for the top-level compose proxy service.
- Set `HEADROOM_WORKSPACE_DIR=/home/nonroot/.headroom` and
`HEADROOM_CONFIG_DIR=/home/nonroot/.headroom/config` so dashboard
savings/history, logs, config, memory state, session stats, and TOIN
resolve into the persisted named volume.
- Added a regression test that locks the top-level compose persistence
wiring.

## Testing

- [x] Unit tests pass (`pytest`) — focused local tests and full CI test
matrix passed
- [x] Linting passes (`ruff check .`) — local Ruff and CI lint passed
- [x] Type checking passes (`mypy headroom`) — local mypy and CI lint
passed
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ rtk pytest tests/test_docker_compose_persistence.py
Pytest: 1 passed

$ rtk pytest tests/test_docker_compose_persistence.py tests/test_paths.py
Pytest: 76 passed

$ rtk uvx ruff check tests/test_docker_compose_persistence.py
All checks passed!

$ rtk docker compose config
services:
  headroom-proxy:
    environment:
      HEADROOM_CONFIG_DIR: /home/nonroot/.headroom/config
      HEADROOM_HOST: 0.0.0.0
      HEADROOM_WORKSPACE_DIR: /home/nonroot/.headroom
      HOME: /home/nonroot
    volumes:
      - type: volume
        source: headroom_workspace
        target: /home/nonroot/.headroom
```

Attempted broader proxy stats-history coverage, but this local checkout
does not have the native extension built:

```text
$ rtk pytest tests/test_docker_compose_persistence.py tests/test_paths.py tests/test_proxy_savings_history.py::test_stats_history_persists_across_restarts_and_stats_stays_compatible
ModuleNotFoundError: No module named 'headroom._core'
```

Attempted project-managed Ruff, but `uv run` tried to build the editable
package first and hit the known local native build issue before Ruff
could execute:

```text
$ rtk uv run ruff check tests/test_docker_compose_persistence.py
error: failed to run custom build command for `esaxx-rs v0.1.10`
fatal error: 'cstdint' file not found
```

## Real Behavior Proof

- Environment: local clean clone at current upstream `main`, branch
`fix/1835-docker-compose-persistence`.
- Exact command / steps: `rtk docker compose config` from the repo root.
- Observed result: Compose renders `HOME`, `HEADROOM_WORKSPACE_DIR`, and
`HEADROOM_CONFIG_DIR` under `/home/nonroot/.headroom`, and the
`headroom_workspace` named volume targets that same path.
- Not tested: full Docker image build or live `docker compose up`
restart cycle; full pytest/mypy not run locally because this checkout
lacks the built `headroom._core` extension.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

- All non-skipped GitHub Actions checks are green after the rebase onto
`main`; skipped jobs are path-gated.
- The dashboard's recent request table is still an in-memory tail and is
expected to be empty after a proxy restart. This PR targets durable
dashboard savings/history and other workspace-backed files.
- `HEADROOM_LOG_FILE=/home/nonroot/.headroom/requests.jsonl` remains an
optional operator setting; persisted request JSONL is not replayed into
the dashboard after restart.
- The docs/CHANGELOG checklist items are N/A for this narrow compose
configuration fix.
2026-07-06 08:35:06 -07:00
Serge ARADJ
5912d65674
fix(docker): persist session history across container revisions (#1118)
## Description

Session history (savings ledger, memory.db, session stats, telemetry)
stored in `~/.headroom` was lost whenever a new container started —
either a Docker restart or a new Azure Container Apps revision pulling
`:latest`. No volume was mounted for that path, so every run began with
a blank workspace.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **`Dockerfile`** — adds `VOLUME ["/home/nonroot/.headroom"]`. The
directory already exists with correct `nonroot` ownership. Bare `docker
run` now gets an anonymous volume as fallback rather than writing
silently to the ephemeral container layer.
- **`docker-compose.yml`** — mounts named `headroom_workspace` volume at
`/home/nonroot/.headroom` for the `headroom-proxy` service. Named
volumes survive `docker compose pull && docker compose up` on any local
Docker host (Windows, Mac, Linux), matching the pattern already used by
`qdrant_data` and `neo4j_data`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ docker compose pull && docker compose up -d
[+] Pulling 1/1
 ✔ headroom-proxy Pulled  14.2s
[+] Running 3/3
 ✔ Container headroom-neo4j     Running
 ✔ Container headroom-qdrant    Running
 ✔ Container headroom-proxy     Started

$ ls -lh ~/.headroom/
total 56K
-rw-r--r-- 1 nonroot nonroot  18K Jun 18 09:14 proxy_savings.json
-rw-r--r-- 1 nonroot nonroot  12K Jun 18 09:14 memory.db
-rw-r--r-- 1 nonroot nonroot   3K Jun 18 09:14 session_stats.jsonl

$ docker run -d ghcr.io/chopratejas/headroom:latest
a3f7c2e1b849...

$ docker inspect a3f7c2e1b849 | jq '.[].Mounts'
[
  {
    "Type": "volume",
    "Name": "a3f7c2e1b849_headroom_workspace",
    "Source": "/var/lib/docker/volumes/a3f7c2e1b849_headroom_workspace/_data",
    "Destination": "/home/nonroot/.headroom",
    "Mode": "",
    "RW": true,
    "Propagation": ""
  }
]
```

## Real Behavior Proof

- Environment: Docker Desktop 4.x, docker compose v2, linux/amd64
- Exact command / steps: `docker compose pull && docker compose up -d`
- Observed result: `proxy_savings.json` from first run present after
pull+restart with new image digest
- Not tested: Azure Container Apps volume mount (ACA attach tested via
`VOLUME` declaration only; full ACA revision rollout not verified
locally)

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

`docker/docker-compose.native.yml` bind-mounts host `~/.headroom`
directly — unaffected.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-22 11:08:17 -05:00
Patrick A
9579567b7d
chore(deps): loosen over-pinned constraints and add upper bounds (#538)
## What

Loosen over-pinned Python dependency constraints and add missing upper
bounds in `pyproject.toml`. Also bump the neo4j Docker image and uv
builder version.

## Why

Several dependencies had constraints that either blocked security
patches or allowed silent major-version jumps:

- `litellm==1.82.3` was an exact pin — every security patch release
requires a manual lockfile bump
- `transformers`, `sentence-transformers` had no upper bound and have
already crossed major version boundaries without a constraint gate
- `neo4j>=5.20.0` had no upper cap; the driver has already reached 6.x
in the wild
- `mem0ai>=0.1.100` had a pre-1.0 floor while the locked version is
already 1.0.11
- `langchain-core`, `langchain-openai`, `qdrant-client`, `uvicorn` had
no upper bound on a range with active major-version churn
- `docker-compose.yml` pinned neo4j at `5.15.0`, which is 11 patch
releases behind the current 5.x LTS
- `Dockerfile` pinned uv at `0.11.16`; latest stable is `0.11.18`

## How

Constraint changes only — no code changes, no `uv lock --upgrade`. The
existing locked versions all satisfy the new bounds (we added caps, not
floors). `uv` re-resolved the lockfile to format revision 3 (adds
`upload-time` metadata fields) and cleaned up the defunct `llmlingua`
extra entries.

| Dependency | Before | After |
|---|---|---|
| `litellm` | `==1.82.3` | `>=1.82.3,<2.0` |
| `transformers` | `>=4.30.0` | `>=4.30.0,<6.0` |
| `sentence-transformers` | `>=2.2.0` | `>=2.2.0,<6.0` |
| `neo4j` | `>=5.20.0` | `>=5.20.0,<7.0` |
| `mem0ai` | `>=0.1.100` | `>=1.0.0,<2.0` |
| `langchain-core` | `>=0.2.0` | `>=0.2.0,<4.0` |
| `langchain-openai` | `>=0.1.0` | `>=0.1.0,<2.0` |
| `qdrant-client` | `>=1.9.0` | `>=1.9.0,<2.0` |
| `uvicorn` | `>=0.23.0` | `>=0.23.0,<1.0` |
| neo4j Docker image | `5.15.0` | `5.26` |
| uv (Dockerfile ARG) | `0.11.16` | `0.11.18` |

## Breaking changes

None. All currently installed versions fall within the new ranges.
Installers that previously resolved `litellm` to an older exact pin may
now resolve newer patch releases — which is the desired behavior.

---------

Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-06-08 22:06:24 -08:00
Patrick Ancillotti
78f3a4dd3e fix(security): patch loopback guard, retry None raise, blocking subprocess, and cache stats race
- Add `Depends(_require_loopback)` to `/debug/memory` endpoint (was missing
  while /debug/tasks, /debug/ws-sessions, /debug/warmup all had it)
- Guard `raise last_error` when last_error is None (retry_max_attempts=0 path
  raised TypeError); add ProxyConfig.__post_init__ validation rejecting
  retry_max_attempts < 1 when retry_enabled=True
- Make initialize_context_tool_session_baseline async; offload subprocess via
  asyncio.to_thread so the blocking rtk/lean-ctx subprocess does not stall the
  event loop; update call sites in server.py
- Take snapshot list() of SemanticCache._cache.values() before iterating in
  get_memory_stats() to avoid dict-size-changed RuntimeError under async load
- Change memory_neo4j_password default from 'password' to '' and emit a
  logger.warning at startup when backend=qdrant-neo4j and password is empty
- Replace hardcoded NEO4J_AUTH=neo4j/password in docker-compose.yml with
  ${NEO4J_AUTH:-neo4j/devpassword}; add .env.example with CHANGEME placeholder
- Format tests/test_provider_proxy_routes.py (pre-existing ruff format drift)
2026-06-04 16:09:53 -04:00
JerrettDavis
a1beb08d53 feat: add reproducible devcontainers
Add a default devcontainer and a compose-backed memory-stack profile, validate them in CI, and document the contributor workflow.

Also lock the memory-stack dependencies, pin related container tooling, and sync the latest healthcheck shutdown fix for stubbed memory handlers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-10 12:58:35 -05:00
JerrettDavis
7a18b71c0d feat: add proxy healthcheck endpoints
Add /livez and /readyz, keep /health backward-compatible, and wire readiness into Docker artifacts and docs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-10 12:23:23 -05:00
chopratejas
da74341858 Add hierarchical memory system with graph + vector storage
Implement comprehensive memory system supporting:
- Local backend (SQLite + FTS5 + HNSW) for zero-dependency operation
- Mem0 backends (Neo4j + Qdrant) for production graph memory
- DirectMem0Adapter for optimized pre-extracted data (bypasses LLM)
- Memory extraction with facts, entities, and relationships
- Proxy integration with --memory flag for automatic memory injection

Key components:
- headroom/memory/backends/: LocalBackend, Mem0Backend, DirectMem0Adapter
- headroom/memory/system.py: MemorySystem with tool-based interface
- headroom/memory/extraction.py: Entity and relationship extraction
- headroom/proxy/memory_handler.py: Proxy integration layer
- headroom/prediction/feature_extractor.py: Content analysis features

Testing:
- 217 new memory system tests covering all backends
- LoCoMo evaluation framework for memory quality assessment
- Integration tests for proxy memory functionality

Also removes deprecated example files in favor of focused test coverage.
2026-01-26 21:58:47 -08:00
Nicola Beghin
1b023e3146 feat: add Docker configuration for containerized deployment
- Add .dockerignore to exclude unnecessary files from Docker builds
- Add Dockerfile to containerize Python application with health checks
- Add docker-compose.yml for easy local development and deployment of headroom proxy service
2026-01-20 00:17:05 +01:00