diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d8c0ffaa6..cb3692a08 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -13,7 +13,7 @@ By participating, you agree to our [Code of Conduct](CODE_OF_CONDUCT.md). | ๐Ÿงน Refactor-only | **Don't.** Only if a maintainer asked, as part of a concrete fix. | | ๐Ÿงช Test/CI-only PR chasing a known `main` failure | **Don't.** We're tracking it. | | ๐Ÿ“ฆ New dep or version bump | **PR with written justification.** | -| โ“ Question | **Discord `#help` | +| โ“ Question | Ask in **Discord `#help`** | **Open PR cap: 10 per author.** Get existing ones merged before opening more. diff --git a/README.md b/README.md index ff376b4f2..542b1745f 100644 --- a/README.md +++ b/README.md @@ -48,10 +48,10 @@ Headroom compresses everything your AI agent reads โ€” tool outputs, logs, RAG c - **Library** โ€” `compress(messages)` in Python or TypeScript, inline in any app - **Proxy** โ€” `headroom proxy --port 8787`, zero code changes, any language -- **Agent wrap** โ€” `headroom wrap claude|codex|aider|copilot|opencode` in one command; Cursor prints manual proxy settings to paste into the app +- **Agent wrap** โ€” `headroom wrap claude|codex|copilot|cursor|aider|opencode|cline|continue|goose|openhands|openclaw|vibe` in one command; undo with `headroom unwrap ` - **MCP server** โ€” `headroom_compress`, `headroom_retrieve`, `headroom_stats` for any MCP client - **Cross-agent memory** โ€” shared store across Claude, Codex, Gemini, auto-dedup -- **`headroom learn`** โ€” mines failed sessions, writes corrections to `CLAUDE.md` / `AGENTS.md` +- **`headroom learn`** โ€” mines failed sessions, writes corrections to `CLAUDE.local.md` (default, gitignored) or `CLAUDE.md` / `AGENTS.md` / `GEMINI.md` - **Output token reduction** โ€” trims what the model *writes back* (not just what you send): drops ceremony/restated code and skips deep "thinking" on routine steps. See [Output token reduction](#output-token-reduction-cut-what-the-model-writes-back). - **Reversible (CCR)** โ€” originals are cached for retrieval on demand @@ -96,7 +96,8 @@ headroom wrap claude # wrap a coding agent headroom proxy --port 8787 # drop-in proxy, zero code changes # or: from headroom import compress # inline library -# 3 โ€” See the savings +# 3 โ€” Verify setup and see the savings +headroom doctor # health check โ€” confirms routing is working headroom perf headroom dashboard # live savings dashboard (proxy must be running) ``` @@ -192,16 +193,22 @@ shows an **Output Tokens Saved** card next to input compression, labelled | Agent | `headroom wrap` | Notes | |--------------|:---------------:|----------------------------------| -| Claude Code | โœ… | `--memory` ยท `--code-graph` ยท `--1m` | +| Claude Code | โœ… | `--memory` ยท `--code-graph` ยท `--1m` ยท `--tool-search` | | Codex | โœ… | shares memory with Claude | | Cursor | Manual setup | starts proxy and prints base URLs for Cursor settings | | Aider | โœ… | starts proxy + launches | | Copilot CLI | โœ… | starts proxy + launches | | OpenClaw | โœ… | installs as ContextEngine plugin | | OpenCode | โœ… | injects config ยท starts proxy + launches | +| Cline | โœ… | starts proxy + injects config | +| Continue | โœ… | starts proxy + injects config | +| Goose | โœ… | starts proxy + launches | +| OpenHands | โœ… | starts proxy + launches | +| Mistral Vibe | โœ… | starts proxy + launches | | Cortex Code | โœ… | 60โ€“65% savings ยท library mode | Any OpenAI-compatible client works via `headroom proxy`. MCP-native: `headroom mcp install`. +Undo durable wrapping with `headroom unwrap ` (supports: `claude`, `copilot`, `codex`, `opencode`, `openclaw`). ### GitHub Copilot CLI subscription mode @@ -309,6 +316,8 @@ docker pull ghcr.io/chopratejas/headroom:latest Granular extras: `[proxy]`, `[mcp]`, `[ml]` (Kompress-base), `[code]`, `[memory]`, `[relevance]`, `[image]`, `[agno]`, `[langchain]`, `[evals]`, `[pytorch-mps]` (Apple-GPU memory-embedder offload โ€” set `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps`). Requires **Python 3.10+**. +> **Note**: `[all]` covers the core stack but excludes framework adapters. Install them separately: `pip install "headroom-ai[langchain]"` (also `[agno]`, `[strands]`, `[anyllm]`, `[bedrock]`). + Using `pipx`? Choose a supported interpreter explicitly: ```bash @@ -401,7 +410,7 @@ download entirely. headroom learn in action

-`headroom learn` โ€” mines failed sessions, writes corrections to `CLAUDE.md` / `AGENTS.md` / `GEMINI.md`. +`headroom learn` โ€” mines failed sessions, writes corrections to `CLAUDE.local.md` (default, gitignored; use `--target CLAUDE.md` for the shared team file) / `AGENTS.md` / `GEMINI.md`. ## Documentation @@ -413,6 +422,7 @@ download entirely. | [Memory](https://headroom-docs.vercel.app/docs/memory) | [Cache optimization](https://headroom-docs.vercel.app/docs/cache-optimization) | | [Failure learning](https://headroom-docs.vercel.app/docs/failure-learning) | [Benchmarks](https://headroom-docs.vercel.app/docs/benchmarks) | | [Configuration](https://headroom-docs.vercel.app/docs/configuration) | [Limitations](https://headroom-docs.vercel.app/docs/limitations) | +| [Persistent installs](https://headroom-docs.vercel.app/docs/persistent-installs) (`headroom init` / `headroom install apply`) | [Savings analytics](https://headroom-docs.vercel.app/docs/savings) (`headroom savings` / `headroom perf` / `headroom doctor`) | ## Compared to diff --git a/SECURITY.md b/SECURITY.md index c308f5cf1..237341f77 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -4,8 +4,8 @@ | Version | Supported | | ------- | ------------------ | -| 0.2.x | :white_check_mark: | -| 0.1.x | :x: | +| 0.27.x (latest) | :white_check_mark: | +| < 0.27.x | :x: | ## Reporting a Vulnerability @@ -46,7 +46,7 @@ When using Headroom: The following are in scope for security reports: - Headroom Python package (`pip install headroom-ai`) - Headroom proxy server -- Official integrations (LangChain, MCP) +- Official integrations (LangChain, Agno, Strands, LiteLLM, Vercel AI SDK, Anthropic/OpenAI SDK wrappers, MCP) The following are out of scope: - Third-party integrations not maintained by us diff --git a/docs/content/docs/api-reference.mdx b/docs/content/docs/api-reference.mdx index 4f5835e96..f5a0d654f 100644 --- a/docs/content/docs/api-reference.mdx +++ b/docs/content/docs/api-reference.mdx @@ -281,7 +281,7 @@ config = SmartCrusherConfig( **Proxy `--mode` is a separate axis**: `headroom proxy --mode token` (maximize compression) or `--mode cache` (freeze prior turns for prefix-cache stability). The proxy does not accept `audit`, `optimize`, or `simulate`. + ## SDK Configuration @@ -354,7 +358,7 @@ provider = AnthropicProvider( ```python -from headroom import GoogleProvider +from headroom.providers import GoogleProvider provider = GoogleProvider( enable_context_caching=True, diff --git a/docs/content/docs/meta.json b/docs/content/docs/meta.json index e1572e4e1..210e37d92 100644 --- a/docs/content/docs/meta.json +++ b/docs/content/docs/meta.json @@ -33,6 +33,7 @@ "strands", "litellm", "claude-code-vertex", + "claude-code-azure-foundry", "opencode", "mcp", "---Configuration---", diff --git a/docs/content/docs/opencode.mdx b/docs/content/docs/opencode.mdx index 9f98cc700..8617a03cf 100644 --- a/docs/content/docs/opencode.mdx +++ b/docs/content/docs/opencode.mdx @@ -37,7 +37,7 @@ headroom unwrap opencode ```bash headroom wrap opencode \ - --port 8787 \ # Proxy port. Defaults to a random available port + --port 8787 \ # Proxy port (default: 8787) --no-rtk \ # Skip RTK context tool injection --no-mcp \ # Skip Headroom MCP registration --no-serena \ # Skip Serena code graph MCP @@ -45,7 +45,7 @@ headroom wrap opencode \ --no-proxy \ # Use an existing proxy instead of starting one --learn \ # Enable memory and live learning --memory \ # Enable persistent memory - --backend anthropic \ # Backend: anthropic, openai, anyllm + --backend anthropic \ # Backend: anthropic, anyllm, litellm- --anyllm-provider ... \ # AnyLLM provider selection --region ... \ # Provider region -- # Arguments passed to the opencode binary diff --git a/docs/content/docs/shared-context.mdx b/docs/content/docs/shared-context.mdx index 0891a88c3..9555dca0c 100644 --- a/docs/content/docs/shared-context.mdx +++ b/docs/content/docs/shared-context.mdx @@ -10,7 +10,7 @@ When agents hand off to each other, context gets replayed in full. SharedContext ```ts twoslash -import { SharedContext } from "headroom"; +import { SharedContext } from "headroom-ai"; const ctx = new SharedContext(); @@ -53,7 +53,7 @@ Store content under a key. Compresses automatically using Headroom's full pipeli ```ts twoslash -import { SharedContext } from "headroom"; +import { SharedContext } from "headroom-ai"; const ctx = new SharedContext(); // ---cut--- const entry = await ctx.put("findings", bigJsonOutput, { @@ -85,7 +85,7 @@ Retrieve content. Returns the compressed version by default, or the original wit ```ts twoslash -import { SharedContext } from "headroom"; +import { SharedContext } from "headroom-ai"; const ctx = new SharedContext(); // ---cut--- const compressed = ctx.get("findings"); // 4K tokens @@ -109,7 +109,7 @@ Aggregated statistics across all entries. ```ts twoslash -import { SharedContext } from "headroom"; +import { SharedContext } from "headroom-ai"; const ctx = new SharedContext(); // ---cut--- const stats = ctx.stats(); @@ -141,7 +141,7 @@ stats.savings_percent # 80.0 ```ts twoslash -import { SharedContext } from "headroom"; +import { SharedContext } from "headroom-ai"; // ---cut--- const ctx = new SharedContext({ model: "claude-sonnet-4-5-20250929", // For token counting diff --git a/docs/content/docs/simulation.mdx b/docs/content/docs/simulation.mdx index 6ca494778..6f40a435c 100644 --- a/docs/content/docs/simulation.mdx +++ b/docs/content/docs/simulation.mdx @@ -29,8 +29,8 @@ plan = client.chat.completions.simulate( print(f"Tokens before: {plan.tokens_before}") print(f"Tokens after: {plan.tokens_after}") -print(f"Would save: {plan.tokens_saved} tokens ({plan.savings_percent:.1f}%)") -print(f"Transforms: {plan.transforms_applied}") +print(f"Would save: {plan.tokens_saved} tokens ({plan.tokens_saved/plan.tokens_before*100:.1f}%)") +print(f"Transforms: {plan.transforms}") ``` @@ -114,7 +114,7 @@ if plan.tokens_saved == 0: print("- No tool outputs with compressible JSON arrays") print("- Content is already compact (code, grep results)") else: - print(f"Transforms applied: {plan.transforms_applied}") + print(f"Transforms applied: {plan.transforms}") # See the optimized messages print(json.dumps(plan.messages_optimized, indent=2)) ``` @@ -141,7 +141,7 @@ for config in configs: ) plan = client.chat.completions.simulate(model="gpt-4o", messages=messages) print(f"max_items={config.max_items_after_crush}: " - f"{plan.tokens_saved} tokens saved ({plan.savings_percent:.1f}%)") + f"{plan.tokens_saved} tokens saved ({plan.tokens_saved/plan.tokens_before*100:.1f}%)") ``` diff --git a/docs/content/docs/troubleshooting.mdx b/docs/content/docs/troubleshooting.mdx index 537d86973..6fd2cf865 100644 --- a/docs/content/docs/troubleshooting.mdx +++ b/docs/content/docs/troubleshooting.mdx @@ -22,7 +22,7 @@ headroom proxy --port 8788 pip install "headroom-ai[proxy]" # Run with debug logging -headroom proxy --log-level debug +headroom proxy --log-file ~/.headroom/logs/proxy.jsonl --log-messages ``` ### Connection refused when calling proxy @@ -43,7 +43,7 @@ ps aux | grep headroom ```bash # Check proxy logs for the actual error -headroom proxy --log-level debug +headroom proxy --log-file ~/.headroom/logs/proxy.jsonl --log-messages # Verify API key is set echo $OPENAI_API_KEY # or ANTHROPIC_API_KEY diff --git a/headroom/capture/network_diff.py b/headroom/capture/network_diff.py index 29b31f4c8..82342dfa8 100644 --- a/headroom/capture/network_diff.py +++ b/headroom/capture/network_diff.py @@ -10,12 +10,15 @@ from __future__ import annotations import base64 import hashlib import json +import logging from dataclasses import dataclass, field from datetime import datetime, timezone from pathlib import Path from typing import Any from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit +logger = logging.getLogger(__name__) + SENSITIVE_HEADER_PARTS = ("authorization", "api-key", "apikey", "token", "secret", "cookie") SENSITIVE_QUERY_PARTS = ("key", "token", "secret", "signature", "code") MAX_BODY_PREVIEW_CHARS = 1200 @@ -168,13 +171,22 @@ def load_capture_file(path: str | Path, *, fallback_lane: str) -> list[CapturedE exchanges: list[CapturedExchange] = [] capture_path = Path(path) + skipped = 0 for line_number, line in enumerate(capture_path.read_text(encoding="utf-8").splitlines(), 1): if not line.strip(): continue - record = json.loads(line) + # mitmproxy captures can be truncated mid-write; skip a corrupt line + # rather than aborting the whole diff with a raw JSONDecodeError. + try: + record = json.loads(line) + except json.JSONDecodeError: + skipped += 1 + continue exchanges.append( exchange_from_record(record, fallback_lane=fallback_lane, sequence=line_number) ) + if skipped: + logger.warning("Skipped %d malformed line(s) in capture file %s", skipped, capture_path) return exchanges diff --git a/headroom/cli/agent_savings.py b/headroom/cli/agent_savings.py index daf4de996..7bf2fac28 100644 --- a/headroom/cli/agent_savings.py +++ b/headroom/cli/agent_savings.py @@ -35,10 +35,10 @@ from .main import main ) @click.option( "--hours", - type=float, + type=click.FloatRange(min=0), default=24.0, show_default=True, - help="Hours of proxy logs to inspect with --check-perf.", + help="Hours of proxy logs to inspect with --check-perf (0 = all data).", ) @click.option( "--accuracy-report", @@ -76,7 +76,10 @@ def agent_savings( ) -> None: """Render or verify Codex/Claude/Cursor token-savings settings.""" - savings_profile = get_agent_savings_profile(profile) + try: + savings_profile = get_agent_savings_profile(profile) + except ValueError as exc: + raise click.BadParameter(str(exc), param_hint="--profile") from None if write_smoke_fixture is not None: eval_path = _write_smoke_fixture(write_smoke_fixture) click.echo(f"Wrote agent-90 smoke fixture to {write_smoke_fixture}") diff --git a/headroom/cli/doctor.py b/headroom/cli/doctor.py index 08d6c9064..491e7cd0c 100644 --- a/headroom/cli/doctor.py +++ b/headroom/cli/doctor.py @@ -356,7 +356,7 @@ def _render(checks: list[CheckResult], port: int, installed: str) -> None: "--port", "-p", default=8787, - type=int, + type=click.IntRange(1, 65535), envvar="HEADROOM_PORT", help="Proxy port to check (default: 8787, env: HEADROOM_PORT)", ) diff --git a/headroom/cli/evals.py b/headroom/cli/evals.py index bbb8ed015..754f9aa45 100644 --- a/headroom/cli/evals.py +++ b/headroom/cli/evals.py @@ -10,14 +10,45 @@ import click from .main import main +def _parse_categories(categories: str | None) -> list[int] | None: + """Parse a comma-separated ``--categories`` value into validated ints. + + Raises ``click.BadParameter`` (clean usage error, exit 2) instead of + letting a non-numeric or out-of-range token surface as a raw traceback. + """ + if not categories: + return None + parsed: list[int] = [] + for token in categories.split(","): + token = token.strip() + if not token: + continue + try: + value = int(token) + except ValueError: + raise click.BadParameter( + f"{token!r} is not an integer; expected comma-separated values 1-5, e.g. 1,2,3", + param_hint="--categories", + ) from None + if not 1 <= value <= 5: + raise click.BadParameter( + f"{value} is out of range; categories must be 1-5", + param_hint="--categories", + ) + parsed.append(value) + return parsed or None + + @main.group() def evals() -> None: - """Memory evaluation commands. + """Evaluation commands (memory, compression robustness, retention). \b Examples: headroom evals memory Run LoCoMo memory evaluation headroom evals memory-v2 Run V2 evaluation with LLM-controlled tools + headroom evals adversarial Compression-robustness adversarial grid + headroom evals probes Retention probes over recorded sessions """ pass @@ -425,9 +456,7 @@ def _run_memory_eval( import asyncio # Build configuration - parsed_categories = None - if categories: - parsed_categories = [int(c) for c in categories.split(",")] + parsed_categories = _parse_categories(categories) memory_config = MemoryConfig() @@ -599,9 +628,7 @@ def _run_memory_eval_v2( import asyncio # Build configuration - parsed_categories = None - if categories: - parsed_categories = [int(c) for c in categories.split(",")] + parsed_categories = _parse_categories(categories) eval_config = MemoryEvalConfigV2( n_conversations=n_conversations, diff --git a/headroom/cli/init.py b/headroom/cli/init.py index 2c68f1faf..13c1d1315 100644 --- a/headroom/cli/init.py +++ b/headroom/cli/init.py @@ -38,7 +38,7 @@ from headroom.install.runtime import ( stop_runtime, wait_ready, ) -from headroom.install.state import load_manifest, save_manifest +from headroom.install.state import ManifestError, load_manifest, save_manifest from headroom.install.supervisors import start_supervisor from headroom.providers.claude import TOOL_SEARCH_DEFAULT, TOOL_SEARCH_ENV from headroom.providers.codex.install import codex_uses_chatgpt_auth @@ -520,7 +520,12 @@ def _ensure_runtime_manifest( memory: bool, ) -> str: profile = _runtime_profile(global_scope) - existing = load_manifest(profile) + try: + existing = load_manifest(profile) + except ManifestError as e: + # Recover from a corrupt manifest by overwriting it rather than crashing. + click.echo(f"Warning: {e}; overwriting.") + existing = None merged_targets = sorted(set(existing.targets if existing else []).union(targets)) manifest = build_manifest( profile=profile, @@ -688,7 +693,11 @@ def _suppress_hook_output() -> Iterator[None]: def _ensure_profile_running(profile: str) -> None: - manifest = load_manifest(profile) + # Best-effort hook path: a corrupt manifest must not crash the session. + try: + manifest = load_manifest(profile) + except ManifestError: + return if manifest is None: return with _suppress_hook_output(): @@ -901,7 +910,13 @@ def _install_headroom_mcp_for_targets(*, targets: list[str], port: int) -> None: @main.group(invoke_without_command=True) @click.option("-g", "--global", "global_scope", is_flag=True, help="Install for the current user.") -@click.option("--port", default=8787, type=int, show_default=True, help="Headroom proxy port.") +@click.option( + "--port", + default=8787, + type=click.IntRange(1, 65535), + show_default=True, + help="Headroom proxy port.", +) @click.option("--backend", default="anthropic", show_default=True, help="Proxy backend.") @click.option("--anyllm-provider", default=None, help="Provider for any-llm backends.") @click.option("--region", default=None, help="Cloud region for Bedrock / Vertex style backends.") @@ -938,6 +953,11 @@ def init( memory, ctx.invoked_subcommand, ) + if anyllm_provider and backend != "anyllm": + click.echo( + f"Warning: --anyllm-provider is ignored unless --backend anyllm " + f"(got --backend {backend})." + ) if ctx.invoked_subcommand is not None: ctx.obj = { "global_scope": global_scope, @@ -1041,14 +1061,22 @@ def init_hook() -> None: def init_hook_ensure(profile: str | None, marker: str | None) -> None: """Best-effort ensure used by installed agent hooks.""" del marker + + def _has_manifest(name: str) -> bool: + # Best-effort: a corrupt manifest must not crash the session-start hook. + try: + return load_manifest(name) is not None + except ManifestError: + return False + profiles: list[str] = [] if profile: profiles.append(profile) else: local_profile = _local_profile() - if load_manifest(local_profile) is not None: + if _has_manifest(local_profile): profiles.append(local_profile) - elif load_manifest(_GLOBAL_PROFILE) is not None: + elif _has_manifest(_GLOBAL_PROFILE): profiles.append(_GLOBAL_PROFILE) for name in profiles: _ensure_profile_running(name) diff --git a/headroom/cli/install.py b/headroom/cli/install.py index 916a849e4..40da76f52 100644 --- a/headroom/cli/install.py +++ b/headroom/cli/install.py @@ -2,6 +2,8 @@ from __future__ import annotations +import shutil +import subprocess from copy import deepcopy import click @@ -26,7 +28,12 @@ from headroom.install.runtime import ( stop_runtime, wait_ready, ) -from headroom.install.state import delete_manifest, load_manifest, save_manifest +from headroom.install.state import ( + ManifestError, + delete_manifest, + load_manifest, + save_manifest, +) from headroom.install.supervisors import ( install_supervisor, remove_supervisor, @@ -43,19 +50,35 @@ def install() -> None: def _require_manifest(profile: str) -> DeploymentManifest: - manifest = load_manifest(profile) + try: + manifest = load_manifest(profile) + except ManifestError as e: + raise click.ClickException(str(e)) from None if manifest is None: raise click.ClickException(f"No deployment profile named '{profile}' is installed.") return manifest def _start_deployment(manifest: DeploymentManifest) -> None: - if manifest.preset == InstallPreset.PERSISTENT_DOCKER.value: - start_persistent_docker(manifest) - elif manifest.supervisor_kind == SupervisorKind.SERVICE.value: - start_supervisor(manifest) - else: - start_detached_agent(manifest.profile) + if manifest.preset == InstallPreset.PERSISTENT_DOCKER.value and shutil.which("docker") is None: + raise click.ClickException( + "Docker is required for this deployment but 'docker' was not found on PATH." + ) + try: + if manifest.preset == InstallPreset.PERSISTENT_DOCKER.value: + start_persistent_docker(manifest) + elif manifest.supervisor_kind == SupervisorKind.SERVICE.value: + start_supervisor(manifest) + else: + start_detached_agent(manifest.profile) + except FileNotFoundError as e: + # A required external binary (docker, launchctl, systemctl) is missing. + raise click.ClickException(f"Cannot start deployment '{manifest.profile}': {e}") from None + except subprocess.CalledProcessError as e: + raise click.ClickException( + f"Cannot start deployment '{manifest.profile}': command failed " + f"({' '.join(map(str, e.cmd)) if isinstance(e.cmd, (list, tuple)) else e.cmd})" + ) from None if not wait_ready(manifest, timeout_seconds=45): raise click.ClickException( @@ -140,7 +163,12 @@ def _reject_task_lifecycle(manifest: DeploymentManifest, action: str) -> None: ) @click.option("--profile", default="default", show_default=True, help="Deployment profile name.") @click.option( - "--port", "-p", default=8787, type=int, show_default=True, help="Persistent proxy port." + "--port", + "-p", + default=8787, + type=click.IntRange(1, 65535), + show_default=True, + help="Persistent proxy port.", ) @click.option( "--backend", @@ -193,6 +221,12 @@ def install_apply( ) -> None: """Install a persistent Headroom deployment.""" + if anyllm_provider and backend != "anyllm": + click.echo( + f"Warning: --anyllm-provider is ignored unless --backend anyllm " + f"(got --backend {backend})." + ) + if preset == InstallPreset.PERSISTENT_DOCKER.value: runtime = RuntimeKind.DOCKER.value @@ -213,7 +247,12 @@ def install_apply( image=image, ) - existing = load_manifest(profile) + try: + existing = load_manifest(profile) + except ManifestError as e: + # A corrupt existing manifest shouldn't block a fresh apply; overwrite it. + click.echo(f"Warning: {e}; overwriting.") + existing = None if existing is not None: click.echo(f"Updating existing deployment profile '{profile}'...") _remove_deployment(existing) @@ -223,12 +262,16 @@ def install_apply( manifest.artifacts = install_supervisor(manifest) save_manifest(manifest) _start_deployment(manifest) - except Exception: + except Exception as exc: _remove_deployment(manifest) if existing is not None: click.echo(f"Restoring previous deployment '{profile}'...") _restore_deployment(existing) - raise + # Surface non-Click errors (OSError, CalledProcessError, โ€ฆ) as a clean + # message rather than a raw traceback; Click errors pass through as-is. + if isinstance(exc, (click.ClickException, click.Abort)): + raise + raise click.ClickException(f"Failed to install deployment '{profile}': {exc}") from exc click.echo( f"Installed persistent deployment '{profile}' " diff --git a/headroom/cli/learn.py b/headroom/cli/learn.py index d1e707ca7..b8a20c405 100644 --- a/headroom/cli/learn.py +++ b/headroom/cli/learn.py @@ -101,7 +101,7 @@ Use 'auto' (default) to scan all detected agents.""" @click.option( "--workers", "-j", - type=int, + type=click.IntRange(min=1), default=None, help="Parallel workers for session scanning. " "Default: auto (min of CPU count, 8). Use 1 for serial.", @@ -164,11 +164,37 @@ def learn( from ..learn.analyzer import SessionAnalyzer, _detect_default_model from ..learn.registry import auto_detect_plugins, get_plugin + # Flag-combination validation โ€” reject contradictory/no-op combinations up + # front rather than letting one flag silently win or be ignored. + if analyze_all and project is not None: + raise click.UsageError("--all and --project are mutually exclusive.") + if llm_judge and not verbosity_mode: + raise click.UsageError("--llm-judge only applies with --verbosity.") + if verbosity_mode and analyze_all and apply: + raise click.UsageError( + "--verbosity persists a single global level, so --all --apply would keep " + "only the last project's level. Re-run with one --project (or drop --apply " + "to preview every project)." + ) + max_workers = workers if workers is not None else min(os.cpu_count() or 4, 8) # Verbosity learning is a distinct flow: it mines behavioral signals (no # failure analysis) and needs no LLM unless --llm-judge is set. if verbosity_mode: + ignored = [ + flag + for flag, is_set in ( + ("--target", target is not None), + ("--main-only", main_only), + ("--workers", workers is not None), + ("--model", model is not None and not llm_judge), + ) + if is_set + ] + if ignored: + verb = "is" if len(ignored) == 1 else "are" + click.echo(f"Note: {', '.join(ignored)} {verb} ignored with --verbosity.") _run_verbosity( project=project, analyze_all=analyze_all, @@ -217,6 +243,10 @@ def learn( click.echo(f"Note: --target is not supported for {agent_name}; ignoring.") all_projects = plugin.discover_projects() if not all_projects: + # An explicitly-selected agent with no data should say so rather than + # exiting silently (the auto path aggregates across agents instead). + if agent != "auto": + click.echo(f"No {plugin.display_name} project data found.") continue available_projects.extend((agent_name, proj.project_path) for proj in all_projects) @@ -367,6 +397,41 @@ def _make_llm_judge(model: str) -> Any: return judge +def _activate_output_shaper(port: int | None = None) -> tuple[str, int]: + """Best-effort: turn the output shaper ON for a running local proxy. + + Writing ``verbosity.json`` is inert on its own โ€” the shaper is a live, + off-by-default knob, so the learned level does nothing until + ``HEADROOM_OUTPUT_SHAPER`` is enabled in the proxy that serves traffic. + When a proxy is already running locally we hot-enable it via + ``/admin/runtime-env`` (no restart, the same channel ``wrap`` uses), so + ``--apply`` actually takes effect. Returns ``(status, port)`` where status is + ``"live"`` (enabled on a running proxy), ``"absent"`` (no reachable proxy), + or ``"error"``. + """ + import json as _json + import os as _os + import urllib.error + import urllib.request + + resolved_port = port if port is not None else int(_os.environ.get("HEADROOM_PORT", "8787")) + request = urllib.request.Request( + f"http://127.0.0.1:{resolved_port}/admin/runtime-env", + data=_json.dumps({"HEADROOM_OUTPUT_SHAPER": "1"}).encode("utf-8"), + method="POST", + headers={"Content-Type": "application/json"}, + ) + try: + with urllib.request.urlopen(request, timeout=2) as response: + response.read() + return "live", resolved_port + except (urllib.error.URLError, OSError): + # ConnectionRefused (no proxy) or 404 (proxy predates the endpoint). + return "absent", resolved_port + except ValueError: + return "error", resolved_port + + def _run_verbosity( *, project: Path | None, @@ -468,9 +533,28 @@ def _run_verbosity( f" [WROTE] {ledger_path} (baseline: {baseline.total_samples} samples, " f"{len(baseline.strata)} strata)" ) - click.echo( - "\n The output shaper now uses this level when " - "HEADROOM_OUTPUT_SHAPER=1 and HEADROOM_VERBOSITY_LEVEL is unset." - ) + # Writing the level is not enough โ€” the shaper is off by default. + # Make --apply actually take effect: hot-enable a running proxy, and + # otherwise tell the user exactly how to turn it on. + status, shaper_port = _activate_output_shaper() + if status == "live": + click.echo( + f"\n โœ“ Output shaper enabled on the running proxy (port {shaper_port}); " + f"level {profile.level} is live now (while HEADROOM_VERBOSITY_LEVEL is unset)." + ) + click.echo( + " To keep it on across restarts: export HEADROOM_OUTPUT_SHAPER=1 " + "before `headroom wrap ...` (wrap pushes it to the proxy)." + ) + else: + click.echo( + "\n โš  Level written, but the output shaper is OFF by default โ€” it is " + "NOT shaping output yet." + ) + click.echo( + " Enable it: export HEADROOM_OUTPUT_SHAPER=1 then `headroom wrap ...` " + "(or start `headroom proxy` with it set). The learned level is then used " + "automatically while HEADROOM_VERBOSITY_LEVEL is unset." + ) else: click.echo("\n Dry run โ€” use --apply to persist the level and baseline.") diff --git a/headroom/cli/mcp.py b/headroom/cli/mcp.py index 27584824b..2f83aea96 100644 --- a/headroom/cli/mcp.py +++ b/headroom/cli/mcp.py @@ -287,6 +287,10 @@ def mcp_status() -> None: click.echo(" Run: headroom proxy") except httpx.TimeoutException: click.echo("Proxy Status: โœ— Timeout") + except httpx.HTTPError as e: + # Catch the rest (InvalidURL, UnsupportedProtocol, ProtocolError, โ€ฆ) + # so a malformed configured HEADROOM_PROXY_URL can't crash `status`. + click.echo(f"Proxy Status: โœ— Unreachable ({proxy_url}: {e})") except ImportError: click.echo("Proxy Status: ? (httpx not installed)") diff --git a/headroom/cli/memory.py b/headroom/cli/memory.py index 1c033033d..c02798c18 100644 --- a/headroom/cli/memory.py +++ b/headroom/cli/memory.py @@ -30,14 +30,33 @@ from ._utils.parsers import parse_duration from .main import main +def _default_db_path() -> str: + """Resolve the memory DB the proxy/install actually use. + + Prefer the project-scoped store the proxy writes when run inside a project + (``./.headroom/memory.db``); otherwise fall back to the global workspace + store (``~/.headroom/memory.db``). The previous bare ``headroom_memory.db`` + default pointed at neither, so ``headroom memory list`` silently read an + empty/legacy DB. + """ + project_db = Path.cwd() / ".headroom" / "memory.db" + if project_db.exists(): + return str(project_db) + from ..paths import memory_db_path + + return str(memory_db_path()) + + def db_path_option(fn: Any) -> Any: """Shared --db-path option for memory commands.""" return click.option( "--db-path", type=click.Path(), - default="headroom_memory.db", - help="Path to the memory database file.", - show_default=True, + default=_default_db_path, + help="Path to the memory database file. Defaults to the project store " + "(./.headroom/memory.db) if present, else the global store " + "(~/.headroom/memory.db).", + show_default="project store if present, else global store", )(fn) @@ -245,7 +264,13 @@ def memory(ctx: click.Context) -> None: @memory.command("list") @db_path_option -@click.option("--limit", "-n", type=int, default=50, help="Maximum number of memories to show.") +@click.option( + "--limit", + "-n", + type=click.IntRange(min=1), + default=50, + help="Maximum number of memories to show.", +) @click.option("--session", "-s", "session_id", type=str, help="Filter by session ID.") @click.option( "--scope", @@ -283,8 +308,29 @@ def list_memories( try: if search_query: - # Use text search - memories = _search_content(store, search_query, limit=limit) + # Content search is a SQL LIKE that ignores structured filters; apply + # --scope/--session/--since to the results so combining them with + # --search isn't a silent no-op. Fetch a generous set before + # filtering (so the limit isn't consumed by rows we'll drop), then + # truncate to the requested limit. + has_filters = bool(scope or session_id or since_duration) + memories = _search_content( + store, search_query, limit=max(limit, 1000) if has_filters else limit + ) + if scope: + memories = [m for m in memories if get_scope_label(m) == scope.upper()] + if session_id: + memories = [m for m in memories if m.session_id == session_id] + if since_duration: + duration = parse_duration(since_duration) + cutoff = datetime.now(timezone.utc).replace(tzinfo=None) - duration + memories = [ + m + for m in memories + if (m.created_at.replace(tzinfo=None) if m.created_at.tzinfo else m.created_at) + >= cutoff + ] + memories = memories[:limit] else: # Build filter filter_kwargs: dict[str, Any] = { diff --git a/headroom/cli/perf.py b/headroom/cli/perf.py index 992ae9b56..5c512a073 100644 --- a/headroom/cli/perf.py +++ b/headroom/cli/perf.py @@ -12,9 +12,9 @@ from .main import main @main.command() @click.option( "--hours", - type=float, + type=click.FloatRange(min=0), default=168.0, - help="Analyze logs from the last N hours (default: 168 = 7 days)", + help="Analyze logs from the last N hours (default: 168 = 7 days; 0 = all data)", ) @click.option("--raw", is_flag=True, help="Show raw PERF records instead of report") @click.option( diff --git a/headroom/cli/proxy.py b/headroom/cli/proxy.py index 18a802892..12f2aac7f 100644 --- a/headroom/cli/proxy.py +++ b/headroom/cli/proxy.py @@ -83,12 +83,22 @@ def _get_env_bool_optional(name: str) -> bool | None: def _get_env_int_optional(name: str) -> int | None: val = os.environ.get(name) - return int(val) if val is not None and val != "" else None + if val is None or val == "": + return None + try: + return int(val) + except ValueError: + raise click.ClickException(f"{name} must be an integer, got {val!r}") from None def _get_env_float_optional(name: str) -> float | None: val = os.environ.get(name) - return float(val) if val is not None and val != "" else None + if val is None or val == "": + return None + try: + return float(val) + except ValueError: + raise click.ClickException(f"{name} must be a number, got {val!r}") from None def _selected_context_tool() -> str: @@ -109,7 +119,7 @@ def _selected_context_tool() -> str: "--port", "-p", default=8787, - type=int, + type=click.IntRange(1, 65535), envvar="HEADROOM_PORT", help="Proxy port (default: 8787, env: HEADROOM_PORT)", ) @@ -141,7 +151,7 @@ def dashboard(port: int, no_open: bool) -> None: "--port", "-p", default=8787, - type=int, + type=click.IntRange(1, 65535), envvar="HEADROOM_PORT", help="Port to bind to (default: 8787, env: HEADROOM_PORT)", ) @@ -547,7 +557,7 @@ def dashboard(port: int, no_open: bool) -> None: ) @click.option( "--read-maturation-quiesce-turns", - type=int, + type=click.IntRange(min=1), default=5, show_default=True, envvar="HEADROOM_READ_MATURATION_QUIESCE_TURNS", @@ -555,7 +565,7 @@ def dashboard(port: int, no_open: bool) -> None: ) @click.option( "--read-maturation-max-hold-turns", - type=int, + type=click.IntRange(min=1), default=25, show_default=True, envvar="HEADROOM_READ_MATURATION_MAX_HOLD_TURNS", @@ -563,7 +573,7 @@ def dashboard(port: int, no_open: bool) -> None: ) @click.option( "--read-maturation-min-size-bytes", - type=int, + type=click.IntRange(min=0), default=2048, show_default=True, envvar="HEADROOM_READ_MATURATION_MIN_SIZE_BYTES", @@ -662,7 +672,7 @@ def dashboard(port: int, no_open: bool) -> None: ) @click.option( "--memory-qdrant-port", - type=int, + type=click.IntRange(1, 65535), default=None, help=( "Qdrant port for the qdrant-neo4j backend (default: 6333, also reads HEADROOM_QDRANT_PORT)" @@ -688,7 +698,7 @@ def dashboard(port: int, no_open: bool) -> None: ) @click.option( "--min-evidence", - type=int, + type=click.IntRange(min=1), default=None, envvar="HEADROOM_MIN_EVIDENCE", help=( @@ -924,6 +934,28 @@ def proxy( err=True, ) + # Warn on contradictory / no-op flag combinations. The resolved value still + # applies; the warning just prevents a silently-ignored flag. + if no_rate_limit and (rpm is not None or tpm is not None): + click.secho( + "Warning: --rpm/--tpm have no effect because --no-rate-limit disables rate limiting.", + fg="yellow", + err=True, + ) + if no_optimize and target_ratio is not None: + click.secho( + "Warning: --target-ratio has no effect because --no-optimize disables compression.", + fg="yellow", + err=True, + ) + if telemetry and no_telemetry: + click.secho( + "Warning: both --telemetry and --no-telemetry were specified; --no-telemetry " + "takes precedence and telemetry will be disabled.", + fg="yellow", + err=True, + ) + # Opt-in: turn on tool_result interceptors (ast-grep Read outline, etc.). # Only fetch the bundled CLI tool binaries when the feature is enabled โ€” # otherwise we'd pay a network round-trip and risk a readonly-FS failure @@ -1258,8 +1290,8 @@ Memory (Multi-Provider): context_tool_line = f" Context Tool: {_selected_context_tool()}" # Performance tuning section โ€” only shown when at least one tuning var is active. - _stable_turn = int(os.environ.get("HEADROOM_COMPRESSION_STABLE_AFTER_TURN", "0")) - _stale_turns = int(os.environ.get("HEADROOM_STALE_READ_COMPRESS_AFTER_TURNS", "0")) + _stable_turn = _get_env_int_optional("HEADROOM_COMPRESSION_STABLE_AFTER_TURN") or 0 + _stale_turns = _get_env_int_optional("HEADROOM_STALE_READ_COMPRESS_AFTER_TURNS") or 0 _embed_socket = os.environ.get("HEADROOM_EMBEDDING_SERVER_SOCKET") or ( embedding_server and (embedding_server_socket or f"/tmp/headroom-embed-{port}.sock") ) diff --git a/headroom/cli/savings.py b/headroom/cli/savings.py index beb5881ba..302bde4ec 100644 --- a/headroom/cli/savings.py +++ b/headroom/cli/savings.py @@ -49,7 +49,7 @@ def _window_line(label: str, window: dict[str, Any]) -> str: @click.option("--json", "as_json", is_flag=True, help="Emit the raw report as JSON.") @click.option( "--days", - type=int, + type=click.IntRange(min=1), default=savings_ledger.DEFAULT_RETENTION_DAYS, show_default=True, help="Retention/lookback window for the ledger, in days.", diff --git a/headroom/cli/wrap.py b/headroom/cli/wrap.py index 3dba737f1..51b280faf 100644 --- a/headroom/cli/wrap.py +++ b/headroom/cli/wrap.py @@ -2569,6 +2569,13 @@ def _ensure_proxy( ) -> subprocess.Popen | None: """Start or verify proxy. Returns process handle if we started it.""" helpers = _live_wrap_module() + # --no-proxy reuses an already-running proxy, so backend/region/provider + # flags (which only apply when we start one) would be silently dropped. + if no_proxy and (backend or anyllm_provider or region): + click.echo( + " Warning: --backend/--region/--anyllm-provider have no effect with --no-proxy " + "(reusing the existing proxy)." + ) if not no_proxy: manifest = helpers._find_persistent_manifest(port) if manifest is not None: @@ -3244,7 +3251,9 @@ def unwrap() -> None: @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -3588,7 +3597,9 @@ def claude( @unwrap.command("claude") -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option("--no-stop-proxy", is_flag=True, help="Do not stop the local Headroom proxy") @click.option("--keep-mcp", is_flag=True, help="Keep Headroom MCP registrations") @click.option("--keep-rtk", is_flag=True, help="Keep rtk Claude hooks") @@ -3659,7 +3670,9 @@ def unwrap_claude( @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -3946,7 +3959,9 @@ def copilot( @unwrap.command("copilot") -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option("--no-stop-proxy", is_flag=True, help="Do not stop the local Headroom proxy") def unwrap_copilot(port: int, no_stop_proxy: bool) -> None: """Undo durable setup from ``headroom wrap copilot``.""" @@ -3966,7 +3981,9 @@ def unwrap_copilot(port: int, no_stop_proxy: bool) -> None: @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -4197,7 +4214,9 @@ def codex( @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -4299,7 +4318,9 @@ def aider( @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -4376,7 +4397,9 @@ def vibe( @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -4460,7 +4483,9 @@ def cursor( @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -4560,7 +4585,9 @@ def cline( @wrap.command("continue", context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -4682,7 +4709,9 @@ def continue_dev( @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -4805,7 +4834,9 @@ def goose( @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -4968,7 +4999,9 @@ def openhands( is_flag=True, help="Install by copying plugin path instead of using --link", ) -@click.option("--proxy-port", default=8787, type=int, help="Headroom proxy port") +@click.option( + "--proxy-port", default=8787, type=click.IntRange(1, 65535), help="Headroom proxy port" +) @click.option("--startup-timeout-ms", default=20000, type=int, help="Proxy startup timeout") @click.option( "--gateway-provider-id", @@ -5182,7 +5215,9 @@ def openclaw( @wrap.command(context_settings={"ignore_unknown_options": True}) -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option( "--no-context-tool", "--no-rtk", @@ -5344,7 +5379,9 @@ def _opencode_home_dir() -> Path: @unwrap.command("opencode") -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option("--no-stop-proxy", is_flag=True, help="Do not stop the local Headroom proxy") def unwrap_opencode(port: int, no_stop_proxy: bool) -> None: """Undo ``headroom wrap opencode`` edits to the active OpenCode config file. @@ -5420,7 +5457,9 @@ def unwrap_opencode(port: int, no_stop_proxy: bool) -> None: @unwrap.command("openclaw") -@click.option("--proxy-port", default=8787, type=int, help="Headroom proxy port") +@click.option( + "--proxy-port", default=8787, type=click.IntRange(1, 65535), help="Headroom proxy port" +) @click.option("--no-stop-proxy", is_flag=True, help="Do not stop the local Headroom proxy") @click.option("--no-restart", is_flag=True, help="Do not restart OpenClaw gateway at the end") @click.option("--verbose", "-v", is_flag=True, help="Verbose output") @@ -5499,7 +5538,9 @@ def unwrap_openclaw( @unwrap.command("codex") -@click.option("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)") +@click.option( + "--port", "-p", default=8787, type=click.IntRange(1, 65535), help="Proxy port (default: 8787)" +) @click.option("--no-stop-proxy", is_flag=True, help="Do not stop the local Headroom proxy") def unwrap_codex(port: int, no_stop_proxy: bool) -> None: """Undo ``headroom wrap codex`` edits to the active Codex config file. diff --git a/headroom/install/state.py b/headroom/install/state.py index 5ef9f0f35..03070a329 100644 --- a/headroom/install/state.py +++ b/headroom/install/state.py @@ -13,6 +13,10 @@ from .paths import deploy_root, manifest_path, profile_root logger = logging.getLogger(__name__) +class ManifestError(Exception): + """A deployment manifest exists on disk but could not be parsed.""" + + def save_manifest(manifest: DeploymentManifest) -> None: """Persist a deployment manifest to disk. @@ -35,10 +39,17 @@ def load_manifest(profile: str = "default") -> DeploymentManifest | None: path = manifest_path(profile) if not path.exists(): return None - payload = json.loads(path.read_text(encoding="utf-8")) - payload["mutations"] = [ManagedMutation(**item) for item in payload.get("mutations", [])] - payload["artifacts"] = [ArtifactRecord(**item) for item in payload.get("artifacts", [])] - return DeploymentManifest(**payload) + # A present-but-corrupt manifest (partial write, hand-edit, schema drift) + # must not crash callers with a raw traceback โ€” every install lifecycle + # command and the auto-run `init hook ensure` route through here. Raise a + # typed error so callers can report cleanly or degrade gracefully. + try: + payload = json.loads(path.read_text(encoding="utf-8")) + payload["mutations"] = [ManagedMutation(**item) for item in payload.get("mutations", [])] + payload["artifacts"] = [ArtifactRecord(**item) for item in payload.get("artifacts", [])] + return DeploymentManifest(**payload) + except (json.JSONDecodeError, ValueError, TypeError, OSError) as e: + raise ManifestError(f"deployment profile '{profile}' is corrupt ({path}): {e}") from e def list_manifests() -> list[DeploymentManifest]: diff --git a/headroom/perf/analyzer.py b/headroom/perf/analyzer.py index 660e82c1c..76c42d3b3 100644 --- a/headroom/perf/analyzer.py +++ b/headroom/perf/analyzer.py @@ -210,6 +210,10 @@ class PerfReport: oldest_kept_ts: str | None = None newest_kept_ts: str | None = None records_filtered_out: int = 0 + # True when no time cutoff was applied (--hours 0, or a value so large it + # overflows datetime arithmetic). The header says "all data" instead of a + # misleading "last 0h". + window_all_data: bool = False # Log timestamps are emitted by Python's `logging` formatter as @@ -249,7 +253,17 @@ def parse_log_files(last_n_hours: float = 168.0) -> PerfReport: if not log_dir.exists(): return report - cutoff = datetime.now() - timedelta(hours=last_n_hours) if last_n_hours > 0 else None + # A huge --hours value (e.g. 1e9) overflows datetime arithmetic. Since + # "look back a billion hours" is effectively "all data", treat overflow as + # no cutoff rather than crashing with a raw OverflowError traceback. + if last_n_hours > 0: + try: + cutoff: datetime | None = datetime.now() - timedelta(hours=last_n_hours) + except OverflowError: + cutoff = None + else: + cutoff = None + report.window_all_data = cutoff is None def _within_window(ts_str: str | None) -> bool: # Fail-open: records without a parseable timestamp are kept. The @@ -498,14 +512,15 @@ def format_report(report: PerfReport) -> str: lines.append("Headroom Performance Report") lines.append("=" * 60) if report.requested_hours is not None: + window_label = "all data" if report.window_all_data else f"last {report.requested_hours:g}h" if report.oldest_kept_ts and report.newest_kept_ts: window_str = ( - f"Window: last {report.requested_hours:g}h " + f"Window: {window_label} " f"(actual data: {report.oldest_kept_ts[:19]} โ†’ " f"{report.newest_kept_ts[:19]})" ) else: - window_str = f"Window: last {report.requested_hours:g}h (no records found in window)" + window_str = f"Window: {window_label} (no records found in window)" lines.append(window_str) if report.records_filtered_out > 0: lines.append( diff --git a/llms.txt b/llms.txt index b3e632d5a..ffc120031 100644 --- a/llms.txt +++ b/llms.txt @@ -21,7 +21,7 @@ The canonical, always-current documentation index lives at the docs site below. - TypeScript / Node: `npm install headroom-ai` (or `pnpm add headroom-ai`, `bun add headroom-ai`) - Docker: `docker run -p 8787:8787 ghcr.io/chopratejas/headroom:latest` - Run the proxy: `headroom proxy --port 8787` then point any client at `http://127.0.0.1:8787` -- Wrap an agent in one command: `headroom wrap claude` (also: `codex`, `cursor`, `aider`, `copilot`, `opencode`) +- Wrap an agent in one command: `headroom wrap claude` (also: `codex`, `copilot`, `cursor`, `aider`, `opencode`, `cline`, `continue`, `goose`, `openhands`, `openclaw`, `vibe`) ## Entry points @@ -53,7 +53,7 @@ The canonical, always-current documentation index lives at the docs site below. - [Persistent memory](https://headroom-docs.vercel.app/docs/memory): Per-project SQLite + HNSW vector store. No cross-project bleed (GH #462). - [SharedContext](https://headroom-docs.vercel.app/docs/shared-context): Compressed inter-agent context handoffs. -- [Failure learning](https://headroom-docs.vercel.app/docs/failure-learning): Offline analysis writes corrections to `CLAUDE.md` / `AGENTS.md`. +- [Failure learning](https://headroom-docs.vercel.app/docs/failure-learning): Offline analysis writes corrections to `CLAUDE.local.md` (default, gitignored) or `CLAUDE.md` (shared) / `AGENTS.md` / `GEMINI.md`. ## Operations @@ -64,4 +64,4 @@ The canonical, always-current documentation index lives at the docs site below. ## Licensing -Apache 2.0. Use commercially, modify, redistribute. Data stays on the user's machine when running the library, proxy, or MCP server locally. Anonymous telemetry enabled by default; set `HEADROOM_TELEMETRY=off` to disable. +Apache 2.0. Use commercially, modify, redistribute. Data stays on the user's machine when running the library, proxy, or MCP server locally. Anonymous telemetry is **off by default** (opt-in); enable with `HEADROOM_TELEMETRY=on` or `headroom proxy --telemetry`. diff --git a/tests/test_cli/test_install_cli.py b/tests/test_cli/test_install_cli.py index 770ec5b6d..07ffe3e0f 100644 --- a/tests/test_cli/test_install_cli.py +++ b/tests/test_cli/test_install_cli.py @@ -319,6 +319,15 @@ def test_install_apply_uses_docker_runtime_for_persistent_docker(monkeypatch) -> monkeypatch.setattr( "headroom.cli.install.wait_ready", lambda deployment, timeout_seconds=45: True ) + # _start_deployment guards the persistent-docker preset with + # `shutil.which("docker")`. Fake docker as present so the test exercises the + # runtime-selection path itself rather than the host's docker install โ€” + # otherwise it passes on dev machines with Docker but fails on CI runners + # (e.g. macos-latest) that have no docker on PATH. + monkeypatch.setattr( + "headroom.cli.install.shutil.which", + lambda name, *args, **kwargs: "/usr/local/bin/docker" if name == "docker" else None, + ) result = runner.invoke(main, ["install", "apply", "--preset", "persistent-docker"]) diff --git a/wiki/cli.md b/wiki/cli.md index 9eb3f56e2..c6fcbe201 100644 --- a/wiki/cli.md +++ b/wiki/cli.md @@ -255,9 +255,9 @@ headroom proxy --mode cache | `--no-code-aware` | off | Disable AST-aware code compression | | `--code-aware` | off | Enable code-aware compression in the proxy (env: HEADROOM_CODE_AWARE_ENABLED) | | `--no-read-lifecycle` | off | Disable stale/superseded read compression | -| `--no-intelligent-context` | off | Disable intelligent context manager | -| `--no-intelligent-scoring` | off | Disable multi-factor importance scoring | -| `--no-compress-first` | off | Disable deep compression before dropping messages | +| `--no-ccr-inject-tool` | off | Disable injecting the `headroom_retrieve` tool | +| `--no-ccr-marker` | off | Disable adding retrieval markers to compressed output | +| `--no-ccr-proactive-expansion` | off | Disable proactive CCR context expansion | | `--memory` | off | Enable persistent user memory | | `--memory-db-path` | `""` | Override memory DB path (help text: `{cwd}/.headroom/memory.db`) | | `--no-memory-tools` | off | Disable automatic memory tool injection | @@ -406,7 +406,7 @@ headroom memory list -q "budget" | Option | Default | Meaning | |---|---|---| -| `--db-path` | `headroom_memory.db` | Memory database path | +| `--db-path` | `./.headroom/memory.db` if present, else `~/.headroom/memory.db` | Memory database path | | `--limit`, `-n` | `50` | Maximum memories to show | | `--session`, `-s` | unset | Filter by session ID | | `--scope` | unset | `USER`, `SESSION`, `AGENT`, or `TURN` | @@ -423,7 +423,7 @@ headroom memory show 1234abcd --json | Argument / option | Default | Meaning | |---|---|---| | `memory_id` | required | Full or partial memory ID | -| `--db-path` | `headroom_memory.db` | Memory database path | +| `--db-path` | `./.headroom/memory.db` if present, else `~/.headroom/memory.db` | Memory database path | | `--json` | off | Emit raw JSON | ### `headroom memory stats` @@ -434,7 +434,7 @@ headroom memory stats | Option | Default | Meaning | |---|---|---| -| `--db-path` | `headroom_memory.db` | Memory database path | +| `--db-path` | `./.headroom/memory.db` if present, else `~/.headroom/memory.db` | Memory database path | ### `headroom memory edit ` @@ -446,7 +446,7 @@ headroom memory edit 1234abcd --importance 0.9 | Argument / option | Default | Meaning | |---|---|---| | `memory_id` | required | Full or partial memory ID | -| `--db-path` | `headroom_memory.db` | Memory database path | +| `--db-path` | `./.headroom/memory.db` if present, else `~/.headroom/memory.db` | Memory database path | | `--content`, `-c` | unset | New memory content | | `--importance`, `-i` | unset | New importance score (`0.0` to `1.0`) | @@ -462,7 +462,7 @@ headroom memory delete 1234abcd --force | Argument / option | Default | Meaning | |---|---|---| | `memory_ids...` | required | One or more memory IDs | -| `--db-path` | `headroom_memory.db` | Memory database path | +| `--db-path` | `./.headroom/memory.db` if present, else `~/.headroom/memory.db` | Memory database path | | `--force`, `-f` | off | Skip confirmation | ### `headroom memory prune` @@ -474,7 +474,7 @@ headroom memory prune --scope SESSION --force | Option | Default | Meaning | |---|---|---| -| `--db-path` | `headroom_memory.db` | Memory database path | +| `--db-path` | `./.headroom/memory.db` if present, else `~/.headroom/memory.db` | Memory database path | | `--older-than` | unset | Age threshold | | `--scope` | unset | Scope filter: `USER`, `SESSION`, `AGENT`, `TURN` | | `--low-importance` | unset | Importance cutoff | @@ -492,7 +492,7 @@ headroom memory purge --confirm | Option | Default | Meaning | |---|---|---| -| `--db-path` | `headroom_memory.db` | Memory database path | +| `--db-path` | `./.headroom/memory.db` if present, else `~/.headroom/memory.db` | Memory database path | | `--confirm` | off | Required confirmation flag | ### `headroom memory export` @@ -504,7 +504,7 @@ headroom memory export --output export.json | Option | Default | Meaning | |---|---|---| -| `--db-path` | `headroom_memory.db` | Memory database path | +| `--db-path` | `./.headroom/memory.db` if present, else `~/.headroom/memory.db` | Memory database path | | `--output`, `-o` | stdout | Output path | ### `headroom memory import ` @@ -517,7 +517,7 @@ headroom memory import export.json --force | Argument / option | Default | Meaning | |---|---|---| | `file` | required | JSON file containing exported memories | -| `--db-path` | `headroom_memory.db` | Memory database path | +| `--db-path` | `./.headroom/memory.db` if present, else `~/.headroom/memory.db` | Memory database path | | `--force`, `-f` | off | Skip confirmation | The import expects a JSON array. Malformed entries are skipped. diff --git a/wiki/configuration.md b/wiki/configuration.md index 90985911f..6f6b8b6c5 100644 --- a/wiki/configuration.md +++ b/wiki/configuration.md @@ -53,11 +53,14 @@ headroom proxy --no-optimize # Disable semantic caching headroom proxy --no-cache -# Disable CCR response handling -headroom proxy --no-ccr-responses +# Disable CCR tool injection +headroom proxy --no-ccr-inject-tool -# Disable proactive expansion -headroom proxy --no-ccr-expansion +# Disable CCR retrieval markers +headroom proxy --no-ccr-marker + +# Disable proactive CCR expansion +headroom proxy --no-ccr-proactive-expansion # (The earlier --llmlingua flag was retired in 0.9.x and replaced by # Kompress (ModernBERT). See `wiki/transforms.md` for the current diff --git a/wiki/learn.md b/wiki/learn.md index db531b569..eb4eb87d6 100644 --- a/wiki/learn.md +++ b/wiki/learn.md @@ -8,9 +8,12 @@ Offline failure learning for coding agents. Analyzes past conversations, finds w # See recommendations for current project (dry-run, no changes) headroom learn -# Write recommendations to CLAUDE.md and MEMORY.md +# Write recommendations to CLAUDE.local.md (gitignored, personal default) headroom learn --apply +# Write to the shared team file instead +headroom learn --apply --target CLAUDE.md + # Analyze a specific project headroom learn --project ~/my-project --apply @@ -99,10 +102,10 @@ Commands repeatedly rejected โ€” model should suggest them to the user instead. | Pattern | Claude Code | Codex | Gemini CLI | |---------|-------------|-------|-----------| -| Environment, paths, commands | **CLAUDE.md** | **AGENTS.md** | **GEMINI.md** | +| Environment, paths, commands | **CLAUDE.local.md** (default) or `CLAUDE.md` (with `--target CLAUDE.md`) | **AGENTS.md** | **GEMINI.md** | | Retry patterns, permissions | **MEMORY.md** | **instructions.md** | **GEMINI.md** | -Output files are agent-native: Claude Code uses CLAUDE.md/MEMORY.md, Codex uses AGENTS.md, Gemini uses GEMINI.md. The same learnings, written to the format each agent reads. +Output files are agent-native: Claude Code writes to `CLAUDE.local.md` by default (gitignored, personal); pass `--target CLAUDE.md` for the shared team file. Codex uses `AGENTS.md`, Gemini uses `GEMINI.md`. The same learnings, written to the format each agent reads. ## Marker-Based Updates @@ -150,13 +153,33 @@ headroom learn [OPTIONS] Options: --project PATH Project directory (default: current directory) - --all Analyze all discovered projects + --all Analyze all discovered projects (mutually exclusive with --project) --apply Write recommendations (default: dry-run) + --target TEXT Context file to write (default: CLAUDE.local.md for Claude Code) + --main-only Write only to the main context file, skip MEMORY.md --agent [auto|claude|codex|gemini] Which agent to analyze (default: auto-detect) --model TEXT LLM for analysis (default: auto from API keys or CLI) + --workers / -j INTEGER Parallel analysis workers (min 1, default: auto) + --verbosity Analyze verbosity level instead of failure patterns + --llm-judge Use an LLM to score verbosity quality (requires --verbosity) ``` +### Verbosity learning (`--verbosity`) + +`headroom learn --verbosity` analyzes past sessions to infer the ideal output verbosity level for your project and writes a `verbosity.json` profile. + +**Important**: the output shaper is **off by default**. Running `--verbosity --apply` will either: +- Hot-enable the output shaper on a running proxy (`POST /admin/runtime-env`), OR +- Print instructions to set `HEADROOM_OUTPUT_SHAPER=1` before `headroom wrap ...` + +To keep the shaper on across proxy restarts, add `export HEADROOM_OUTPUT_SHAPER=1` to your shell profile before starting the proxy. + +**Flag interactions**: +- `--all` and `--project` are mutually exclusive +- `--llm-judge` requires `--verbosity` +- `--verbosity --all --apply` is rejected (verbosity persists a single global level) + ### Supported Agents | Agent | Scanner | Writer | Output Files | diff --git a/wiki/metrics.md b/wiki/metrics.md index ddde65cf8..01c09612e 100644 --- a/wiki/metrics.md +++ b/wiki/metrics.md @@ -410,8 +410,8 @@ DEBUG:headroom.transforms.smart_crusher:Kept items: [0,1,2,42,77,97,98,99] (erro # Log to file headroom proxy --log-file headroom.jsonl -# Increase verbosity -headroom proxy --log-level debug +# Enable request logging +headroom proxy --log-messages ``` ## Grafana Dashboard diff --git a/wiki/proxy.md b/wiki/proxy.md index d8f6994c4..9384abd33 100644 --- a/wiki/proxy.md +++ b/wiki/proxy.md @@ -102,23 +102,15 @@ Legacy values (`token_headroom`, `cost_savings`) are still accepted as aliases. ### Context Management Options -| Option | Default | Description | -|--------|---------|-------------| -| `--no-intelligent-context` | `false` | Disable IntelligentContextManager (fall back to RollingWindow) | -| `--no-intelligent-scoring` | `false` | Disable multi-factor importance scoring (use position-based) | -| `--no-compress-first` | `false` | Disable trying deeper compression before dropping messages | +Context management in the proxy is handled automatically by the compression pipeline. CCR (Compress-Cache-Retrieve) ensures that when content is compressed or messages are dropped, the original data remains accessible for the LLM to retrieve on demand. See [CCR documentation](ccr.md) for details. -By default, the proxy uses **IntelligentContextManager** which scores messages by multiple factors (recency, semantic similarity, TOIN-learned patterns, error indicators, forward references) and drops lowest-scored messages first. This is smarter than simple age-based truncation. +Key CCR-related proxy flags: -**CCR Integration:** When messages are dropped, they're stored in CCR so the LLM can retrieve them if needed. The inserted marker includes the CCR reference. Drops are also recorded to TOIN, so the system learns which message patterns are important across all users. - -```bash -# Use legacy RollingWindow (drops oldest first) -headroom proxy --no-intelligent-context - -# Disable semantic scoring (faster, but less intelligent) -headroom proxy --no-intelligent-scoring -``` +| Option | Description | +|--------|-------------| +| `--no-ccr-inject-tool` | Do not inject the `headroom_retrieve` tool into the LLM's available tools | +| `--no-ccr-marker` | Do not add retrieval markers to compressed output | +| `--no-ccr-proactive-expansion` | Disable proactive context expansion before the LLM asks | ### ML Compression โ€” RETIRED `--llmlingua` flag diff --git a/wiki/quickstart.md b/wiki/quickstart.md index 5ce5e9a98..ab823b714 100644 --- a/wiki/quickstart.md +++ b/wiki/quickstart.md @@ -319,7 +319,7 @@ response = client.chat.completions.create( ## Next Steps -- **[Configuration Reference](api.md)** - All configuration options +- **[Configuration Reference](configuration.md)** - All configuration options - **[Transform Reference](transforms.md)** - How each transform works - **[Troubleshooting](troubleshooting.md)** - Common issues and solutions - **[Examples](../examples/)** - More complete examples diff --git a/wiki/troubleshooting.md b/wiki/troubleshooting.md index 109b61c8c..c91583aff 100644 --- a/wiki/troubleshooting.md +++ b/wiki/troubleshooting.md @@ -23,8 +23,8 @@ headroom proxy --port 8788 # 3. Check for missing dependencies pip install "headroom-ai[proxy]" -# 4. Run with debug logging -headroom proxy --log-level debug +# 4. Run with request logging +headroom proxy --log-file ~/.headroom/logs/proxy.jsonl --log-messages ``` ### "Connection refused" when calling proxy @@ -69,7 +69,7 @@ Alternatively, restarting the proxy process clears the in-memory tracker. See [S ```bash # 1. Check proxy logs for the actual error -headroom proxy --log-level debug +headroom proxy --log-file ~/.headroom/logs/proxy.jsonl --log-messages # 2. Verify API key is set echo $OPENAI_API_KEY # or ANTHROPIC_API_KEY