headroom/docs/cortex-code.md

Cortex Code + Headroom — Integration Guide

Headroom compresses the context Cortex Code (CoCo) sends to claude-sonnet-4-6 before it reaches the Snowflake Cortex inference endpoint. The result is 60–65% fewer prompt tokens billed, with the same answers.

Benchmark (measured, not estimated)

Token counts are from usage.prompt_tokens in the actual Snowflake Cortex API response — not headroom's local estimate.

Payload	Before	After	Saved
Full CoCo session (tables + dbt + search)	17,827	6,781	62%
INFORMATION_SCHEMA tables (79 rows)	10,161	3,979	61%
dbt run-results (40 models)	4,968	1,927	61%
Cortex Search results (15 docs)	2,764	956	65%

At 1,000 calls/day: ~$16/day saved, ~$6,000/year saved.

Numbers above are per-call averages across the four benchmark payloads. The full-session payload alone saves ~$33/1,000 calls/day.

How it works

CoCo (cortex CLI)
  │  OPENAI_BASE_URL=http://127.0.0.1:8787/v1
  ▼
Headroom proxy  (local, your data never leaves your machine)
  │  SmartCrusher compresses JSON context
  │  CacheAligner stabilises KV-cache prefixes
  ▼
Snowflake Cortex  /api/v2/cortex/inference:complete
  │  claude-sonnet-4-6
  ▼
Response (same answer, fewer billed tokens)

Headroom's SmartCrusher targets the large JSON blobs that CoCo produces: INFORMATION_SCHEMA query results, dbt run-results, Cortex Search payloads, and schema introspection output. These are highly repetitive structures that compress 60–99% without any loss of information.

Quick start

pip install "headroom-ai[all]"
headroom wrap cortex-code          # starts proxy + prints the env var to set

headroom wrap cortex-code starts the local proxy and prints:

  Headroom proxy is running. Configure Cortex Code (CoCo):

  Set the following environment variable before launching cortex:
    OPENAI_BASE_URL=http://127.0.0.1:8787/v1

Then in a new shell:

OPENAI_BASE_URL=http://127.0.0.1:8787/v1 cortex

Or add it to your shell profile so it applies to every CoCo session:

# ~/.zshrc or ~/.bashrc
export OPENAI_BASE_URL=http://127.0.0.1:8787/v1

Manual proxy startup

If you prefer to manage the proxy lifecycle yourself:

# Terminal 1 — start the proxy
headroom proxy --port 8787

# Terminal 2 — launch CoCo through the proxy
OPENAI_BASE_URL=http://127.0.0.1:8787/v1 cortex

Point the proxy at your Snowflake Cortex endpoint explicitly with --openai-api-url:

headroom proxy \
  --port 8787 \
  --openai-api-url https://<account>.snowflakecomputing.com

Library mode (inline, no proxy)

If you are building an application on top of the Snowflake Cortex REST API and want to compress context before every call:

from headroom import compress
import json, urllib.request

# Build your messages (large JSON tool results, search results, etc.)
messages = [
    {"role": "system", "content": json.dumps(cortex_search_results, indent=2)},
    {"role": "assistant", "content": "I have reviewed the context."},
    {"role": "user", "content": "What is failing and how do I fix it?"},
]

# Compress before sending  — local, no API call, no data leaves your machine
result = compress(messages, model="claude-sonnet-4-6")
print(f"Saved {result.tokens_saved} tokens ({result.tokens_saved / result.tokens_before:.0%})")

# Send compressed messages to Snowflake Cortex REST API
response = call_cortex(result.messages, token=sf_token)

What to put in the system message

The Snowflake Cortex REST API supports system, user, and assistant roles. For maximum compression, inject large retrieved context into system:

# Query results, search results, schema — these compress 60–99%
system_context = {
    "tables":         json.loads(show_tables_result),
    "search_results": cortex_search_results,
    "schema":         describe_table_result,
    "dbt_results":    dbt_run_results_json,
}
messages = [
    {"role": "system", "content": json.dumps(system_context, indent=2)},
    {"role": "assistant", "content": "Context loaded."},
    {"role": "user",      "content": user_question},
]
result = compress(messages, model="claude-sonnet-4-6")

Authentication

Cortex Code authenticates using your Snowflake connection. Headroom sits between CoCo and the Cortex endpoint and forwards auth headers unchanged — it never reads or stores your credentials.

If you use snowflake-connector-python directly, keep the connection open while making API calls; closing it invalidates the OAuth session token:

import snowflake.connector, sys, io

# Suppress connector's browser-auth console output
_s = sys.stdout; sys.stdout = io.StringIO()
conn = snowflake.connector.connect(connection_name="my_connection")
token = conn.rest.token
sys.stdout = _s

# Make all API calls while conn is open, then:
conn.close()

Per-project savings attribution

Use headroom wrap cortex-code --project <name> to attribute savings to a specific project in the headroom dashboard:

headroom wrap cortex-code --project my-dbt-project

The dashboard at http://127.0.0.1:8787 shows per-project token and cost savings across all your CoCo sessions.

Verifying savings

After a CoCo session, check what headroom saved:

headroom perf          # token savings for the last session
headroom perf --hours 24  # last 24 hours

Or run the included end-to-end benchmark against your own Snowflake account:

# Measures real usage.prompt_tokens from claude-sonnet-4-6
python3 tests/e2e_cortex_savings.py

Testing

Unit tests for the provider slice:

uv run --with pytest pytest tests/test_provider_cortex_code.py -v

Compression benchmark (no API key needed — local only):

uv run --with pytest pytest tests/test_cortex_code_compression.py -v -s

Real E2E test against Snowflake Cortex (requires Snowflake connection):

python3 tests/e2e_cortex_savings.py

How the provider is implemented

Cortex Code routes through headroom's OpenAI-compatible pipeline. The provider slice lives in headroom/providers/cortex_code/:

File	Purpose
runtime.py	proxy_base_url(port) → http://127.0.0.1:{port}/v1; default_api_url() reads SNOWFLAKE_HOST / SNOWFLAKE_ACCOUNT
install.py	build_install_env() → {"OPENAI_BASE_URL": ...}; render_setup_lines()
__init__.py	Public exports

Registered in headroom/providers/install_registry.py under the key "cortex-code", which is what headroom wrap cortex-code resolves to.

Limitations

• The Snowflake Cortex REST API at /api/v2/cortex/inference:complete does not support role: "tool" messages or OpenAI-style tool_calls. Use the system message to inject large retrieved context (where SmartCrusher achieves the highest compression ratios).

• The headroom proxy cannot rewrite the Cortex inference path (/api/v2/cortex/inference:complete ≠ /v1/chat/completions), so library mode (from headroom import compress) is required when calling the Cortex REST API directly. The proxy mode works for any OpenAI-compatible client that points at Cortex via a gateway that exposes /v1/chat/completions.

• Output-token reduction (HEADROOM_OUTPUT_SHAPER=1) is supported in proxy mode. In library mode only input compression applies.

See also

• Architecture
• Proxy configuration
• CCR — reversible compression
• Claude Code + Vertex
• Benchmarks