mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
Deployed fb953eb with MkDocs version: 1.6.1
This commit is contained in:
parent
11e44e1143
commit
c2f1fe400b
8 changed files with 285 additions and 225 deletions
|
|
@ -2068,8 +2068,11 @@
|
|||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-3-1"><a id="__codelineno-3-1" name="__codelineno-3-1" href="#__codelineno-3-1"></a><span class="c1"># Claude Code</span>
|
||||
</span><span id="__span-3-2"><a id="__codelineno-3-2" name="__codelineno-3-2" href="#__codelineno-3-2"></a><span class="nv">ANTHROPIC_BASE_URL</span><span class="o">=</span>http://localhost:8787<span class="w"> </span>claude
|
||||
</span><span id="__span-3-3"><a id="__codelineno-3-3" name="__codelineno-3-3" href="#__codelineno-3-3"></a>
|
||||
</span><span id="__span-3-4"><a id="__codelineno-3-4" name="__codelineno-3-4" href="#__codelineno-3-4"></a><span class="c1"># OpenAI-compatible clients</span>
|
||||
</span><span id="__span-3-5"><a id="__codelineno-3-5" name="__codelineno-3-5" href="#__codelineno-3-5"></a><span class="nv">OPENAI_BASE_URL</span><span class="o">=</span>http://localhost:8787/v1<span class="w"> </span>your-app
|
||||
</span><span id="__span-3-4"><a id="__codelineno-3-4" name="__codelineno-3-4" href="#__codelineno-3-4"></a><span class="c1"># GitHub Copilot CLI (default Anthropic-style proxy route)</span>
|
||||
</span><span id="__span-3-5"><a id="__codelineno-3-5" name="__codelineno-3-5" href="#__codelineno-3-5"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>copilot<span class="w"> </span>--<span class="w"> </span>--model<span class="w"> </span>claude-sonnet-4-20250514
|
||||
</span><span id="__span-3-6"><a id="__codelineno-3-6" name="__codelineno-3-6" href="#__codelineno-3-6"></a>
|
||||
</span><span id="__span-3-7"><a id="__codelineno-3-7" name="__codelineno-3-7" href="#__codelineno-3-7"></a><span class="c1"># OpenAI-compatible clients</span>
|
||||
</span><span id="__span-3-8"><a id="__codelineno-3-8" name="__codelineno-3-8" href="#__codelineno-3-8"></a><span class="nv">OPENAI_BASE_URL</span><span class="o">=</span>http://localhost:8787/v1<span class="w"> </span>your-app
|
||||
</span></code></pre></div>
|
||||
<p>That's it! All your requests now go through Headroom and get optimized automatically.</p>
|
||||
<h2 id="quick-start-python-sdk">Quick Start: Python SDK<a class="headerlink" href="#quick-start-python-sdk" title="Permanent link">¶</a></h2>
|
||||
|
|
|
|||
|
|
@ -2503,10 +2503,11 @@
|
|||
</div>
|
||||
<div class="tabbed-block">
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-4-1"><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>claude<span class="w"> </span><span class="c1"># Claude Code</span>
|
||||
</span><span id="__span-4-2"><a id="__codelineno-4-2" name="__codelineno-4-2" href="#__codelineno-4-2"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>codex<span class="w"> </span><span class="c1"># OpenAI Codex CLI</span>
|
||||
</span><span id="__span-4-3"><a id="__codelineno-4-3" name="__codelineno-4-3" href="#__codelineno-4-3"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>aider<span class="w"> </span><span class="c1"># Aider</span>
|
||||
</span><span id="__span-4-4"><a id="__codelineno-4-4" name="__codelineno-4-4" href="#__codelineno-4-4"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>cursor<span class="w"> </span><span class="c1"># Cursor</span>
|
||||
</span><span id="__span-4-5"><a id="__codelineno-4-5" name="__codelineno-4-5" href="#__codelineno-4-5"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>openclaw<span class="w"> </span><span class="c1"># OpenClaw plugin bootstrap</span>
|
||||
</span><span id="__span-4-2"><a id="__codelineno-4-2" name="__codelineno-4-2" href="#__codelineno-4-2"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>copilot<span class="w"> </span>--<span class="w"> </span>--model<span class="w"> </span>claude-sonnet-4-20250514
|
||||
</span><span id="__span-4-3"><a id="__codelineno-4-3" name="__codelineno-4-3" href="#__codelineno-4-3"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>codex<span class="w"> </span><span class="c1"># OpenAI Codex CLI</span>
|
||||
</span><span id="__span-4-4"><a id="__codelineno-4-4" name="__codelineno-4-4" href="#__codelineno-4-4"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>aider<span class="w"> </span><span class="c1"># Aider</span>
|
||||
</span><span id="__span-4-5"><a id="__codelineno-4-5" name="__codelineno-4-5" href="#__codelineno-4-5"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>cursor<span class="w"> </span><span class="c1"># Cursor</span>
|
||||
</span><span id="__span-4-6"><a id="__codelineno-4-6" name="__codelineno-4-6" href="#__codelineno-4-6"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>openclaw<span class="w"> </span><span class="c1"># OpenClaw plugin bootstrap</span>
|
||||
</span></code></pre></div>
|
||||
<p>Starts the proxy, points your tool at it, compresses everything automatically.</p>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -2312,9 +2312,9 @@
|
|||
<td>1 line</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Claude Code / Cursor</td>
|
||||
<td>Claude Code / Cursor / Copilot CLI</td>
|
||||
<td><a href="#proxy">Headroom proxy</a></td>
|
||||
<td>1 env var</td>
|
||||
<td>1 command or env var</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Agno agents</td>
|
||||
|
|
@ -2475,91 +2475,98 @@
|
|||
- <code>x-headroom-tokens-saved: 1234</code> — tokens removed</p>
|
||||
<hr />
|
||||
<h2 id="proxy">Proxy<a class="headerlink" href="#proxy" title="Permanent link">¶</a></h2>
|
||||
<p>The Headroom proxy is a standalone HTTP server. Best for non-Python apps or tools that only support base URL configuration (Claude Code, Cursor).</p>
|
||||
<p>The Headroom proxy is a standalone HTTP server. Best for non-Python apps or tools that only support base URL configuration (Claude Code, Cursor, GitHub Copilot CLI).</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-10-1"><a id="__codelineno-10-1" name="__codelineno-10-1" href="#__codelineno-10-1"></a>pip<span class="w"> </span>install<span class="w"> </span><span class="s2">"headroom-ai[all]"</span>
|
||||
</span><span id="__span-10-2"><a id="__codelineno-10-2" name="__codelineno-10-2" href="#__codelineno-10-2"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--port<span class="w"> </span><span class="m">8787</span>
|
||||
</span></code></pre></div>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-11-1"><a id="__codelineno-11-1" name="__codelineno-11-1" href="#__codelineno-11-1"></a><span class="c1"># Claude Code</span>
|
||||
</span><span id="__span-11-2"><a id="__codelineno-11-2" name="__codelineno-11-2" href="#__codelineno-11-2"></a><span class="nv">ANTHROPIC_BASE_URL</span><span class="o">=</span>http://localhost:8787<span class="w"> </span>claude
|
||||
</span><span id="__span-11-3"><a id="__codelineno-11-3" name="__codelineno-11-3" href="#__codelineno-11-3"></a>
|
||||
</span><span id="__span-11-4"><a id="__codelineno-11-4" name="__codelineno-11-4" href="#__codelineno-11-4"></a><span class="c1"># Cursor / Any OpenAI client</span>
|
||||
</span><span id="__span-11-5"><a id="__codelineno-11-5" name="__codelineno-11-5" href="#__codelineno-11-5"></a><span class="nv">OPENAI_BASE_URL</span><span class="o">=</span>http://localhost:8787/v1<span class="w"> </span>cursor
|
||||
</span><span id="__span-11-4"><a id="__codelineno-11-4" name="__codelineno-11-4" href="#__codelineno-11-4"></a><span class="c1"># GitHub Copilot CLI</span>
|
||||
</span><span id="__span-11-5"><a id="__codelineno-11-5" name="__codelineno-11-5" href="#__codelineno-11-5"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>copilot<span class="w"> </span>--<span class="w"> </span>--model<span class="w"> </span>claude-sonnet-4-20250514
|
||||
</span><span id="__span-11-6"><a id="__codelineno-11-6" name="__codelineno-11-6" href="#__codelineno-11-6"></a>
|
||||
</span><span id="__span-11-7"><a id="__codelineno-11-7" name="__codelineno-11-7" href="#__codelineno-11-7"></a><span class="c1"># Cursor / Any OpenAI client</span>
|
||||
</span><span id="__span-11-8"><a id="__codelineno-11-8" name="__codelineno-11-8" href="#__codelineno-11-8"></a><span class="nv">OPENAI_BASE_URL</span><span class="o">=</span>http://localhost:8787/v1<span class="w"> </span>cursor
|
||||
</span></code></pre></div>
|
||||
<p>For translated backends, the Copilot wrapper can switch to Headroom's OpenAI-compatible route:</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-12-1"><a id="__codelineno-12-1" name="__codelineno-12-1" href="#__codelineno-12-1"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>copilot<span class="w"> </span>--backend<span class="w"> </span>anyllm<span class="w"> </span>--anyllm-provider<span class="w"> </span>groq<span class="w"> </span>--<span class="w"> </span>--model<span class="w"> </span>gpt-4o
|
||||
</span></code></pre></div>
|
||||
<p>By default, <code>headroom wrap copilot</code> installs <code>rtk</code> and appends token-optimized shell guidance to <code>.github/copilot-instructions.md</code> so Copilot sessions reuse the same command-saving conventions as other wrapped agent CLIs. Use <code>--no-rtk</code> to skip that step.</p>
|
||||
<h3 id="with-cloud-providers">With Cloud Providers<a class="headerlink" href="#with-cloud-providers" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-12-1"><a id="__codelineno-12-1" name="__codelineno-12-1" href="#__codelineno-12-1"></a><span class="c1"># AWS Bedrock</span>
|
||||
</span><span id="__span-12-2"><a id="__codelineno-12-2" name="__codelineno-12-2" href="#__codelineno-12-2"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--backend<span class="w"> </span>bedrock<span class="w"> </span>--region<span class="w"> </span>us-east-1
|
||||
</span><span id="__span-12-3"><a id="__codelineno-12-3" name="__codelineno-12-3" href="#__codelineno-12-3"></a>
|
||||
</span><span id="__span-12-4"><a id="__codelineno-12-4" name="__codelineno-12-4" href="#__codelineno-12-4"></a><span class="c1"># Google Vertex AI</span>
|
||||
</span><span id="__span-12-5"><a id="__codelineno-12-5" name="__codelineno-12-5" href="#__codelineno-12-5"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--backend<span class="w"> </span>vertex_ai<span class="w"> </span>--region<span class="w"> </span>us-central1
|
||||
</span><span id="__span-12-6"><a id="__codelineno-12-6" name="__codelineno-12-6" href="#__codelineno-12-6"></a>
|
||||
</span><span id="__span-12-7"><a id="__codelineno-12-7" name="__codelineno-12-7" href="#__codelineno-12-7"></a><span class="c1"># Azure OpenAI</span>
|
||||
</span><span id="__span-12-8"><a id="__codelineno-12-8" name="__codelineno-12-8" href="#__codelineno-12-8"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--backend<span class="w"> </span>azure
|
||||
</span><span id="__span-12-9"><a id="__codelineno-12-9" name="__codelineno-12-9" href="#__codelineno-12-9"></a>
|
||||
</span><span id="__span-12-10"><a id="__codelineno-12-10" name="__codelineno-12-10" href="#__codelineno-12-10"></a><span class="c1"># OpenRouter (400+ models)</span>
|
||||
</span><span id="__span-12-11"><a id="__codelineno-12-11" name="__codelineno-12-11" href="#__codelineno-12-11"></a><span class="nv">OPENROUTER_API_KEY</span><span class="o">=</span>sk-or-...<span class="w"> </span>headroom<span class="w"> </span>proxy<span class="w"> </span>--backend<span class="w"> </span>openrouter
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-13-1"><a id="__codelineno-13-1" name="__codelineno-13-1" href="#__codelineno-13-1"></a><span class="c1"># AWS Bedrock</span>
|
||||
</span><span id="__span-13-2"><a id="__codelineno-13-2" name="__codelineno-13-2" href="#__codelineno-13-2"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--backend<span class="w"> </span>bedrock<span class="w"> </span>--region<span class="w"> </span>us-east-1
|
||||
</span><span id="__span-13-3"><a id="__codelineno-13-3" name="__codelineno-13-3" href="#__codelineno-13-3"></a>
|
||||
</span><span id="__span-13-4"><a id="__codelineno-13-4" name="__codelineno-13-4" href="#__codelineno-13-4"></a><span class="c1"># Google Vertex AI</span>
|
||||
</span><span id="__span-13-5"><a id="__codelineno-13-5" name="__codelineno-13-5" href="#__codelineno-13-5"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--backend<span class="w"> </span>vertex_ai<span class="w"> </span>--region<span class="w"> </span>us-central1
|
||||
</span><span id="__span-13-6"><a id="__codelineno-13-6" name="__codelineno-13-6" href="#__codelineno-13-6"></a>
|
||||
</span><span id="__span-13-7"><a id="__codelineno-13-7" name="__codelineno-13-7" href="#__codelineno-13-7"></a><span class="c1"># Azure OpenAI</span>
|
||||
</span><span id="__span-13-8"><a id="__codelineno-13-8" name="__codelineno-13-8" href="#__codelineno-13-8"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--backend<span class="w"> </span>azure
|
||||
</span><span id="__span-13-9"><a id="__codelineno-13-9" name="__codelineno-13-9" href="#__codelineno-13-9"></a>
|
||||
</span><span id="__span-13-10"><a id="__codelineno-13-10" name="__codelineno-13-10" href="#__codelineno-13-10"></a><span class="c1"># OpenRouter (400+ models)</span>
|
||||
</span><span id="__span-13-11"><a id="__codelineno-13-11" name="__codelineno-13-11" href="#__codelineno-13-11"></a><span class="nv">OPENROUTER_API_KEY</span><span class="o">=</span>sk-or-...<span class="w"> </span>headroom<span class="w"> </span>proxy<span class="w"> </span>--backend<span class="w"> </span>openrouter
|
||||
</span></code></pre></div>
|
||||
<p>See <a href="../proxy/">Proxy Documentation</a> for all options.</p>
|
||||
<hr />
|
||||
<h2 id="agno">Agno<a class="headerlink" href="#agno" title="Permanent link">¶</a></h2>
|
||||
<p>Full integration with the Agno agent framework.</p>
|
||||
<div class="language-python highlight"><pre><span></span><code><span id="__span-13-1"><a id="__codelineno-13-1" name="__codelineno-13-1" href="#__codelineno-13-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">agno.agent</span><span class="w"> </span><span class="kn">import</span> <span class="n">Agent</span>
|
||||
</span><span id="__span-13-2"><a id="__codelineno-13-2" name="__codelineno-13-2" href="#__codelineno-13-2"></a><span class="kn">from</span><span class="w"> </span><span class="nn">agno.models.anthropic</span><span class="w"> </span><span class="kn">import</span> <span class="n">Claude</span>
|
||||
</span><span id="__span-13-3"><a id="__codelineno-13-3" name="__codelineno-13-3" href="#__codelineno-13-3"></a><span class="kn">from</span><span class="w"> </span><span class="nn">headroom.integrations.agno</span><span class="w"> </span><span class="kn">import</span> <span class="n">HeadroomAgnoModel</span>
|
||||
</span><span id="__span-13-4"><a id="__codelineno-13-4" name="__codelineno-13-4" href="#__codelineno-13-4"></a>
|
||||
</span><span id="__span-13-5"><a id="__codelineno-13-5" name="__codelineno-13-5" href="#__codelineno-13-5"></a><span class="n">model</span> <span class="o">=</span> <span class="n">HeadroomAgnoModel</span><span class="p">(</span><span class="n">Claude</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="s2">"claude-sonnet-4-20250514"</span><span class="p">))</span>
|
||||
</span><span id="__span-13-6"><a id="__codelineno-13-6" name="__codelineno-13-6" href="#__codelineno-13-6"></a><span class="n">agent</span> <span class="o">=</span> <span class="n">Agent</span><span class="p">(</span><span class="n">model</span><span class="o">=</span><span class="n">model</span><span class="p">,</span> <span class="n">tools</span><span class="o">=</span><span class="p">[</span><span class="n">your_tools</span><span class="p">])</span>
|
||||
</span><span id="__span-13-7"><a id="__codelineno-13-7" name="__codelineno-13-7" href="#__codelineno-13-7"></a><span class="n">response</span> <span class="o">=</span> <span class="n">agent</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="s2">"Investigate the issue"</span><span class="p">)</span>
|
||||
</span><span id="__span-13-8"><a id="__codelineno-13-8" name="__codelineno-13-8" href="#__codelineno-13-8"></a>
|
||||
</span><span id="__span-13-9"><a id="__codelineno-13-9" name="__codelineno-13-9" href="#__codelineno-13-9"></a><span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Tokens saved: </span><span class="si">{</span><span class="n">model</span><span class="o">.</span><span class="n">total_tokens_saved</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<div class="language-python highlight"><pre><span></span><code><span id="__span-14-1"><a id="__codelineno-14-1" name="__codelineno-14-1" href="#__codelineno-14-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">agno.agent</span><span class="w"> </span><span class="kn">import</span> <span class="n">Agent</span>
|
||||
</span><span id="__span-14-2"><a id="__codelineno-14-2" name="__codelineno-14-2" href="#__codelineno-14-2"></a><span class="kn">from</span><span class="w"> </span><span class="nn">agno.models.anthropic</span><span class="w"> </span><span class="kn">import</span> <span class="n">Claude</span>
|
||||
</span><span id="__span-14-3"><a id="__codelineno-14-3" name="__codelineno-14-3" href="#__codelineno-14-3"></a><span class="kn">from</span><span class="w"> </span><span class="nn">headroom.integrations.agno</span><span class="w"> </span><span class="kn">import</span> <span class="n">HeadroomAgnoModel</span>
|
||||
</span><span id="__span-14-4"><a id="__codelineno-14-4" name="__codelineno-14-4" href="#__codelineno-14-4"></a>
|
||||
</span><span id="__span-14-5"><a id="__codelineno-14-5" name="__codelineno-14-5" href="#__codelineno-14-5"></a><span class="n">model</span> <span class="o">=</span> <span class="n">HeadroomAgnoModel</span><span class="p">(</span><span class="n">Claude</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="s2">"claude-sonnet-4-20250514"</span><span class="p">))</span>
|
||||
</span><span id="__span-14-6"><a id="__codelineno-14-6" name="__codelineno-14-6" href="#__codelineno-14-6"></a><span class="n">agent</span> <span class="o">=</span> <span class="n">Agent</span><span class="p">(</span><span class="n">model</span><span class="o">=</span><span class="n">model</span><span class="p">,</span> <span class="n">tools</span><span class="o">=</span><span class="p">[</span><span class="n">your_tools</span><span class="p">])</span>
|
||||
</span><span id="__span-14-7"><a id="__codelineno-14-7" name="__codelineno-14-7" href="#__codelineno-14-7"></a><span class="n">response</span> <span class="o">=</span> <span class="n">agent</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="s2">"Investigate the issue"</span><span class="p">)</span>
|
||||
</span><span id="__span-14-8"><a id="__codelineno-14-8" name="__codelineno-14-8" href="#__codelineno-14-8"></a>
|
||||
</span><span id="__span-14-9"><a id="__codelineno-14-9" name="__codelineno-14-9" href="#__codelineno-14-9"></a><span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Tokens saved: </span><span class="si">{</span><span class="n">model</span><span class="o">.</span><span class="n">total_tokens_saved</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span></code></pre></div>
|
||||
<p>See <a href="../agno/">Agno Guide</a> for hooks, multi-provider, and streaming.</p>
|
||||
<hr />
|
||||
<h2 id="langchain">LangChain<a class="headerlink" href="#langchain" title="Permanent link">¶</a></h2>
|
||||
<p>Full integration with LangChain — chat models, memory, retrievers, tool wrappers, and streaming.</p>
|
||||
<div class="language-python highlight"><pre><span></span><code><span id="__span-14-1"><a id="__codelineno-14-1" name="__codelineno-14-1" href="#__codelineno-14-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">langchain_openai</span><span class="w"> </span><span class="kn">import</span> <span class="n">ChatOpenAI</span>
|
||||
</span><span id="__span-14-2"><a id="__codelineno-14-2" name="__codelineno-14-2" href="#__codelineno-14-2"></a><span class="kn">from</span><span class="w"> </span><span class="nn">headroom.integrations</span><span class="w"> </span><span class="kn">import</span> <span class="n">HeadroomChatModel</span>
|
||||
</span><span id="__span-14-3"><a id="__codelineno-14-3" name="__codelineno-14-3" href="#__codelineno-14-3"></a>
|
||||
</span><span id="__span-14-4"><a id="__codelineno-14-4" name="__codelineno-14-4" href="#__codelineno-14-4"></a><span class="n">llm</span> <span class="o">=</span> <span class="n">HeadroomChatModel</span><span class="p">(</span><span class="n">ChatOpenAI</span><span class="p">(</span><span class="n">model</span><span class="o">=</span><span class="s2">"gpt-4o"</span><span class="p">))</span>
|
||||
</span><span id="__span-14-5"><a id="__codelineno-14-5" name="__codelineno-14-5" href="#__codelineno-14-5"></a><span class="n">response</span> <span class="o">=</span> <span class="n">llm</span><span class="o">.</span><span class="n">invoke</span><span class="p">(</span><span class="s2">"Hello!"</span><span class="p">)</span>
|
||||
<div class="language-python highlight"><pre><span></span><code><span id="__span-15-1"><a id="__codelineno-15-1" name="__codelineno-15-1" href="#__codelineno-15-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">langchain_openai</span><span class="w"> </span><span class="kn">import</span> <span class="n">ChatOpenAI</span>
|
||||
</span><span id="__span-15-2"><a id="__codelineno-15-2" name="__codelineno-15-2" href="#__codelineno-15-2"></a><span class="kn">from</span><span class="w"> </span><span class="nn">headroom.integrations</span><span class="w"> </span><span class="kn">import</span> <span class="n">HeadroomChatModel</span>
|
||||
</span><span id="__span-15-3"><a id="__codelineno-15-3" name="__codelineno-15-3" href="#__codelineno-15-3"></a>
|
||||
</span><span id="__span-15-4"><a id="__codelineno-15-4" name="__codelineno-15-4" href="#__codelineno-15-4"></a><span class="n">llm</span> <span class="o">=</span> <span class="n">HeadroomChatModel</span><span class="p">(</span><span class="n">ChatOpenAI</span><span class="p">(</span><span class="n">model</span><span class="o">=</span><span class="s2">"gpt-4o"</span><span class="p">))</span>
|
||||
</span><span id="__span-15-5"><a id="__codelineno-15-5" name="__codelineno-15-5" href="#__codelineno-15-5"></a><span class="n">response</span> <span class="o">=</span> <span class="n">llm</span><span class="o">.</span><span class="n">invoke</span><span class="p">(</span><span class="s2">"Hello!"</span><span class="p">)</span>
|
||||
</span></code></pre></div>
|
||||
<p>See <a href="../langchain/">LangChain Guide</a> for details and known limitations.</p>
|
||||
<hr />
|
||||
<h2 id="typescript-sdk">TypeScript SDK<a class="headerlink" href="#typescript-sdk" title="Permanent link">¶</a></h2>
|
||||
<p>For Node.js, Next.js, and any TypeScript/JavaScript application.</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-15-1"><a id="__codelineno-15-1" name="__codelineno-15-1" href="#__codelineno-15-1"></a>npm<span class="w"> </span>install<span class="w"> </span>headroom-ai
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-16-1"><a id="__codelineno-16-1" name="__codelineno-16-1" href="#__codelineno-16-1"></a>npm<span class="w"> </span>install<span class="w"> </span>headroom-ai
|
||||
</span></code></pre></div>
|
||||
<p>See the <a href="../typescript-sdk/">TypeScript SDK Guide</a> for full documentation including Vercel AI SDK middleware, OpenAI SDK wrapper, and Anthropic SDK wrapper.</p>
|
||||
<hr />
|
||||
<h2 id="openclaw">OpenClaw<a class="headerlink" href="#openclaw" title="Permanent link">¶</a></h2>
|
||||
<p>Context compression plugin for <a href="https://github.com/openclaw/openclaw">OpenClaw</a> agents.</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-16-1"><a id="__codelineno-16-1" name="__codelineno-16-1" href="#__codelineno-16-1"></a>pip<span class="w"> </span>install<span class="w"> </span><span class="s2">"headroom-ai[proxy]"</span>
|
||||
</span><span id="__span-16-2"><a id="__codelineno-16-2" name="__codelineno-16-2" href="#__codelineno-16-2"></a>openclaw<span class="w"> </span>plugins<span class="w"> </span>install<span class="w"> </span>headroom-openclaw
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-17-1"><a id="__codelineno-17-1" name="__codelineno-17-1" href="#__codelineno-17-1"></a>pip<span class="w"> </span>install<span class="w"> </span><span class="s2">"headroom-ai[proxy]"</span>
|
||||
</span><span id="__span-17-2"><a id="__codelineno-17-2" name="__codelineno-17-2" href="#__codelineno-17-2"></a>openclaw<span class="w"> </span>plugins<span class="w"> </span>install<span class="w"> </span>headroom-openclaw
|
||||
</span></code></pre></div>
|
||||
<p>Configure as context engine:
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-17-1"><a id="__codelineno-17-1" name="__codelineno-17-1" href="#__codelineno-17-1"></a><span class="p">{</span><span class="w"> </span><span class="nt">"plugins"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nt">"slots"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nt">"contextEngine"</span><span class="p">:</span><span class="w"> </span><span class="s2">"headroom"</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="p">}</span>
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-18-1"><a id="__codelineno-18-1" name="__codelineno-18-1" href="#__codelineno-18-1"></a><span class="p">{</span><span class="w"> </span><span class="nt">"plugins"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nt">"slots"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nt">"contextEngine"</span><span class="p">:</span><span class="w"> </span><span class="s2">"headroom"</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="p">}</span>
|
||||
</span></code></pre></div></p>
|
||||
<p>The plugin auto-detects a running Headroom proxy or starts one. Compression happens in <code>assemble()</code> — zero changes to the agent's behavior.</p>
|
||||
<p>See the <a href="https://github.com/chopratejas/headroom/tree/main/plugins/openclaw">OpenClaw plugin documentation</a> for full setup.</p>
|
||||
<hr />
|
||||
<h2 id="compression-hooks-advanced">Compression Hooks (Advanced)<a class="headerlink" href="#compression-hooks-advanced" title="Permanent link">¶</a></h2>
|
||||
<p>Customize compression behavior without modifying Headroom's code:</p>
|
||||
<div class="language-python highlight"><pre><span></span><code><span id="__span-18-1"><a id="__codelineno-18-1" name="__codelineno-18-1" href="#__codelineno-18-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">headroom</span><span class="w"> </span><span class="kn">import</span> <span class="n">compress</span><span class="p">,</span> <span class="n">CompressionHooks</span><span class="p">,</span> <span class="n">CompressContext</span>
|
||||
</span><span id="__span-18-2"><a id="__codelineno-18-2" name="__codelineno-18-2" href="#__codelineno-18-2"></a>
|
||||
</span><span id="__span-18-3"><a id="__codelineno-18-3" name="__codelineno-18-3" href="#__codelineno-18-3"></a><span class="k">class</span><span class="w"> </span><span class="nc">MyHooks</span><span class="p">(</span><span class="n">CompressionHooks</span><span class="p">):</span>
|
||||
</span><span id="__span-18-4"><a id="__codelineno-18-4" name="__codelineno-18-4" href="#__codelineno-18-4"></a> <span class="k">def</span><span class="w"> </span><span class="nf">pre_compress</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">messages</span><span class="p">,</span> <span class="n">ctx</span><span class="p">):</span>
|
||||
</span><span id="__span-18-5"><a id="__codelineno-18-5" name="__codelineno-18-5" href="#__codelineno-18-5"></a> <span class="c1"># Modify messages before compression (dedup, filter, inject)</span>
|
||||
</span><span id="__span-18-6"><a id="__codelineno-18-6" name="__codelineno-18-6" href="#__codelineno-18-6"></a> <span class="k">return</span> <span class="n">messages</span>
|
||||
</span><span id="__span-18-7"><a id="__codelineno-18-7" name="__codelineno-18-7" href="#__codelineno-18-7"></a>
|
||||
</span><span id="__span-18-8"><a id="__codelineno-18-8" name="__codelineno-18-8" href="#__codelineno-18-8"></a> <span class="k">def</span><span class="w"> </span><span class="nf">compute_biases</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">messages</span><span class="p">,</span> <span class="n">ctx</span><span class="p">):</span>
|
||||
</span><span id="__span-18-9"><a id="__codelineno-18-9" name="__codelineno-18-9" href="#__codelineno-18-9"></a> <span class="c1"># Per-message compression aggressiveness</span>
|
||||
</span><span id="__span-18-10"><a id="__codelineno-18-10" name="__codelineno-18-10" href="#__codelineno-18-10"></a> <span class="c1"># >1.0 = keep more, <1.0 = compress more</span>
|
||||
</span><span id="__span-18-11"><a id="__codelineno-18-11" name="__codelineno-18-11" href="#__codelineno-18-11"></a> <span class="k">return</span> <span class="p">{</span><span class="mi">5</span><span class="p">:</span> <span class="mf">1.5</span><span class="p">,</span> <span class="mi">6</span><span class="p">:</span> <span class="mf">0.5</span><span class="p">}</span> <span class="c1"># Keep message 5, compress message 6</span>
|
||||
</span><span id="__span-18-12"><a id="__codelineno-18-12" name="__codelineno-18-12" href="#__codelineno-18-12"></a>
|
||||
</span><span id="__span-18-13"><a id="__codelineno-18-13" name="__codelineno-18-13" href="#__codelineno-18-13"></a> <span class="k">def</span><span class="w"> </span><span class="nf">post_compress</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">event</span><span class="p">):</span>
|
||||
</span><span id="__span-18-14"><a id="__codelineno-18-14" name="__codelineno-18-14" href="#__codelineno-18-14"></a> <span class="c1"># Observe results (logging, analytics, learning)</span>
|
||||
</span><span id="__span-18-15"><a id="__codelineno-18-15" name="__codelineno-18-15" href="#__codelineno-18-15"></a> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Saved </span><span class="si">{</span><span class="n">event</span><span class="o">.</span><span class="n">tokens_saved</span><span class="si">}</span><span class="s2"> tokens"</span><span class="p">)</span>
|
||||
</span><span id="__span-18-16"><a id="__codelineno-18-16" name="__codelineno-18-16" href="#__codelineno-18-16"></a>
|
||||
</span><span id="__span-18-17"><a id="__codelineno-18-17" name="__codelineno-18-17" href="#__codelineno-18-17"></a><span class="n">result</span> <span class="o">=</span> <span class="n">compress</span><span class="p">(</span><span class="n">messages</span><span class="p">,</span> <span class="n">model</span><span class="o">=</span><span class="s2">"gpt-4o"</span><span class="p">,</span> <span class="n">hooks</span><span class="o">=</span><span class="n">MyHooks</span><span class="p">())</span>
|
||||
<div class="language-python highlight"><pre><span></span><code><span id="__span-19-1"><a id="__codelineno-19-1" name="__codelineno-19-1" href="#__codelineno-19-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">headroom</span><span class="w"> </span><span class="kn">import</span> <span class="n">compress</span><span class="p">,</span> <span class="n">CompressionHooks</span><span class="p">,</span> <span class="n">CompressContext</span>
|
||||
</span><span id="__span-19-2"><a id="__codelineno-19-2" name="__codelineno-19-2" href="#__codelineno-19-2"></a>
|
||||
</span><span id="__span-19-3"><a id="__codelineno-19-3" name="__codelineno-19-3" href="#__codelineno-19-3"></a><span class="k">class</span><span class="w"> </span><span class="nc">MyHooks</span><span class="p">(</span><span class="n">CompressionHooks</span><span class="p">):</span>
|
||||
</span><span id="__span-19-4"><a id="__codelineno-19-4" name="__codelineno-19-4" href="#__codelineno-19-4"></a> <span class="k">def</span><span class="w"> </span><span class="nf">pre_compress</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">messages</span><span class="p">,</span> <span class="n">ctx</span><span class="p">):</span>
|
||||
</span><span id="__span-19-5"><a id="__codelineno-19-5" name="__codelineno-19-5" href="#__codelineno-19-5"></a> <span class="c1"># Modify messages before compression (dedup, filter, inject)</span>
|
||||
</span><span id="__span-19-6"><a id="__codelineno-19-6" name="__codelineno-19-6" href="#__codelineno-19-6"></a> <span class="k">return</span> <span class="n">messages</span>
|
||||
</span><span id="__span-19-7"><a id="__codelineno-19-7" name="__codelineno-19-7" href="#__codelineno-19-7"></a>
|
||||
</span><span id="__span-19-8"><a id="__codelineno-19-8" name="__codelineno-19-8" href="#__codelineno-19-8"></a> <span class="k">def</span><span class="w"> </span><span class="nf">compute_biases</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">messages</span><span class="p">,</span> <span class="n">ctx</span><span class="p">):</span>
|
||||
</span><span id="__span-19-9"><a id="__codelineno-19-9" name="__codelineno-19-9" href="#__codelineno-19-9"></a> <span class="c1"># Per-message compression aggressiveness</span>
|
||||
</span><span id="__span-19-10"><a id="__codelineno-19-10" name="__codelineno-19-10" href="#__codelineno-19-10"></a> <span class="c1"># >1.0 = keep more, <1.0 = compress more</span>
|
||||
</span><span id="__span-19-11"><a id="__codelineno-19-11" name="__codelineno-19-11" href="#__codelineno-19-11"></a> <span class="k">return</span> <span class="p">{</span><span class="mi">5</span><span class="p">:</span> <span class="mf">1.5</span><span class="p">,</span> <span class="mi">6</span><span class="p">:</span> <span class="mf">0.5</span><span class="p">}</span> <span class="c1"># Keep message 5, compress message 6</span>
|
||||
</span><span id="__span-19-12"><a id="__codelineno-19-12" name="__codelineno-19-12" href="#__codelineno-19-12"></a>
|
||||
</span><span id="__span-19-13"><a id="__codelineno-19-13" name="__codelineno-19-13" href="#__codelineno-19-13"></a> <span class="k">def</span><span class="w"> </span><span class="nf">post_compress</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">event</span><span class="p">):</span>
|
||||
</span><span id="__span-19-14"><a id="__codelineno-19-14" name="__codelineno-19-14" href="#__codelineno-19-14"></a> <span class="c1"># Observe results (logging, analytics, learning)</span>
|
||||
</span><span id="__span-19-15"><a id="__codelineno-19-15" name="__codelineno-19-15" href="#__codelineno-19-15"></a> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Saved </span><span class="si">{</span><span class="n">event</span><span class="o">.</span><span class="n">tokens_saved</span><span class="si">}</span><span class="s2"> tokens"</span><span class="p">)</span>
|
||||
</span><span id="__span-19-16"><a id="__codelineno-19-16" name="__codelineno-19-16" href="#__codelineno-19-16"></a>
|
||||
</span><span id="__span-19-17"><a id="__codelineno-19-17" name="__codelineno-19-17" href="#__codelineno-19-17"></a><span class="n">result</span> <span class="o">=</span> <span class="n">compress</span><span class="p">(</span><span class="n">messages</span><span class="p">,</span> <span class="n">model</span><span class="o">=</span><span class="s2">"gpt-4o"</span><span class="p">,</span> <span class="n">hooks</span><span class="o">=</span><span class="n">MyHooks</span><span class="p">())</span>
|
||||
</span></code></pre></div>
|
||||
<p>See <a href="../ARCHITECTURE/">Architecture</a> for how hooks integrate with the pipeline.</p>
|
||||
<hr />
|
||||
|
|
|
|||
306
proxy/index.html
306
proxy/index.html
|
|
@ -776,6 +776,23 @@
|
|||
</span>
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="Starting the Proxy">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#common-agent-cli-entrypoints" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
Common agent CLI entrypoints
|
||||
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
|
|
@ -2145,6 +2162,23 @@
|
|||
</span>
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="Starting the Proxy">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#common-agent-cli-entrypoints" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
Common agent CLI entrypoints
|
||||
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
|
|
@ -2479,19 +2513,30 @@
|
|||
</span><span id="__span-0-11"><a id="__codelineno-0-11" name="__codelineno-0-11" href="#__codelineno-0-11"></a><span class="w"> </span>--log-file<span class="w"> </span>/var/log/headroom.jsonl<span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-0-12"><a id="__codelineno-0-12" name="__codelineno-0-12" href="#__codelineno-0-12"></a><span class="w"> </span>--budget<span class="w"> </span><span class="m">100</span>.0
|
||||
</span></code></pre></div>
|
||||
<h3 id="common-agent-cli-entrypoints">Common agent CLI entrypoints<a class="headerlink" href="#common-agent-cli-entrypoints" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-1-1"><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a><span class="c1"># Claude Code</span>
|
||||
</span><span id="__span-1-2"><a id="__codelineno-1-2" name="__codelineno-1-2" href="#__codelineno-1-2"></a><span class="nv">ANTHROPIC_BASE_URL</span><span class="o">=</span>http://localhost:8787<span class="w"> </span>claude
|
||||
</span><span id="__span-1-3"><a id="__codelineno-1-3" name="__codelineno-1-3" href="#__codelineno-1-3"></a>
|
||||
</span><span id="__span-1-4"><a id="__codelineno-1-4" name="__codelineno-1-4" href="#__codelineno-1-4"></a><span class="c1"># GitHub Copilot CLI</span>
|
||||
</span><span id="__span-1-5"><a id="__codelineno-1-5" name="__codelineno-1-5" href="#__codelineno-1-5"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>copilot<span class="w"> </span>--<span class="w"> </span>--model<span class="w"> </span>claude-sonnet-4-20250514
|
||||
</span><span id="__span-1-6"><a id="__codelineno-1-6" name="__codelineno-1-6" href="#__codelineno-1-6"></a>
|
||||
</span><span id="__span-1-7"><a id="__codelineno-1-7" name="__codelineno-1-7" href="#__codelineno-1-7"></a><span class="c1"># OpenAI-compatible clients</span>
|
||||
</span><span id="__span-1-8"><a id="__codelineno-1-8" name="__codelineno-1-8" href="#__codelineno-1-8"></a><span class="nv">OPENAI_BASE_URL</span><span class="o">=</span>http://localhost:8787/v1<span class="w"> </span>your-app
|
||||
</span></code></pre></div>
|
||||
<p><code>headroom wrap copilot</code> uses Copilot CLI's BYOK provider settings under the hood. In <code>provider-type=auto</code>, it chooses Headroom's Anthropic route for the default proxy backend and the OpenAI-compatible <code>/v1</code> route for translated backends such as <code>anyllm</code> and LiteLLM.</p>
|
||||
<p>Anonymous aggregate telemetry is enabled by default. Opt out with <code>HEADROOM_TELEMETRY=off</code> or <code>headroom proxy --no-telemetry</code>. Downstream apps can set <code>HEADROOM_SDK=headroom-app</code> to override the anonymous telemetry <code>sdk</code> label; the default remains <code>proxy</code>.</p>
|
||||
<p>Operational OTEL metrics are configured separately and are <strong>off by default</strong>. Install <code>headroom-ai[proxy,otel]</code> and set:</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-1-1"><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a><span class="nv">HEADROOM_OTEL_METRICS_ENABLED</span><span class="o">=</span><span class="m">1</span>
|
||||
</span><span id="__span-1-2"><a id="__codelineno-1-2" name="__codelineno-1-2" href="#__codelineno-1-2"></a><span class="nv">HEADROOM_OTEL_METRICS_EXPORTER</span><span class="o">=</span>otlp_http
|
||||
</span><span id="__span-1-3"><a id="__codelineno-1-3" name="__codelineno-1-3" href="#__codelineno-1-3"></a><span class="nv">HEADROOM_OTEL_METRICS_ENDPOINT</span><span class="o">=</span>http://127.0.0.1:4318/v1/metrics
|
||||
</span><span id="__span-1-4"><a id="__codelineno-1-4" name="__codelineno-1-4" href="#__codelineno-1-4"></a><span class="nv">HEADROOM_OTEL_SERVICE_NAME</span><span class="o">=</span>headroom-proxy
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-2-1"><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a><span class="nv">HEADROOM_OTEL_METRICS_ENABLED</span><span class="o">=</span><span class="m">1</span>
|
||||
</span><span id="__span-2-2"><a id="__codelineno-2-2" name="__codelineno-2-2" href="#__codelineno-2-2"></a><span class="nv">HEADROOM_OTEL_METRICS_EXPORTER</span><span class="o">=</span>otlp_http
|
||||
</span><span id="__span-2-3"><a id="__codelineno-2-3" name="__codelineno-2-3" href="#__codelineno-2-3"></a><span class="nv">HEADROOM_OTEL_METRICS_ENDPOINT</span><span class="o">=</span>http://127.0.0.1:4318/v1/metrics
|
||||
</span><span id="__span-2-4"><a id="__codelineno-2-4" name="__codelineno-2-4" href="#__codelineno-2-4"></a><span class="nv">HEADROOM_OTEL_SERVICE_NAME</span><span class="o">=</span>headroom-proxy
|
||||
</span></code></pre></div>
|
||||
<p>Use <code>HEADROOM_OTEL_METRICS_EXPORTER=console</code> for local smoke testing. <code>HEADROOM_TELEMETRY</code> controls the anonymous data-flywheel beacon only; it does not disable or enable OTEL export.</p>
|
||||
<p>Langfuse can be enabled alongside this OTEL path for <strong>trace ingestion</strong>. Langfuse does <strong>not</strong> ingest OTEL metrics, so Headroom keeps metrics and Langfuse traces as complementary signals:</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-2-1"><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a><span class="nv">HEADROOM_LANGFUSE_ENABLED</span><span class="o">=</span><span class="m">1</span>
|
||||
</span><span id="__span-2-2"><a id="__codelineno-2-2" name="__codelineno-2-2" href="#__codelineno-2-2"></a><span class="nv">LANGFUSE_PUBLIC_KEY</span><span class="o">=</span>pk-lf-...
|
||||
</span><span id="__span-2-3"><a id="__codelineno-2-3" name="__codelineno-2-3" href="#__codelineno-2-3"></a><span class="nv">LANGFUSE_SECRET_KEY</span><span class="o">=</span>sk-lf-...
|
||||
</span><span id="__span-2-4"><a id="__codelineno-2-4" name="__codelineno-2-4" href="#__codelineno-2-4"></a><span class="nv">LANGFUSE_BASE_URL</span><span class="o">=</span>https://cloud.langfuse.com
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-3-1"><a id="__codelineno-3-1" name="__codelineno-3-1" href="#__codelineno-3-1"></a><span class="nv">HEADROOM_LANGFUSE_ENABLED</span><span class="o">=</span><span class="m">1</span>
|
||||
</span><span id="__span-3-2"><a id="__codelineno-3-2" name="__codelineno-3-2" href="#__codelineno-3-2"></a><span class="nv">LANGFUSE_PUBLIC_KEY</span><span class="o">=</span>pk-lf-...
|
||||
</span><span id="__span-3-3"><a id="__codelineno-3-3" name="__codelineno-3-3" href="#__codelineno-3-3"></a><span class="nv">LANGFUSE_SECRET_KEY</span><span class="o">=</span>sk-lf-...
|
||||
</span><span id="__span-3-4"><a id="__codelineno-3-4" name="__codelineno-3-4" href="#__codelineno-3-4"></a><span class="nv">LANGFUSE_BASE_URL</span><span class="o">=</span>https://cloud.langfuse.com
|
||||
</span></code></pre></div>
|
||||
<p>When configured, Headroom emits OTLP traces for the shared compression pipeline to Langfuse while continuing to expose metrics through <code>/metrics</code> and OTEL metric exporters.</p>
|
||||
<h2 id="command-line-options">Command Line Options<a class="headerlink" href="#command-line-options" title="Permanent link">¶</a></h2>
|
||||
|
|
@ -2559,8 +2604,8 @@
|
|||
<li><code>cache</code> mode: prioritize provider prefix cache stability. Prior turns are frozen; only the newest turn is mutable.</li>
|
||||
</ul>
|
||||
<p>Set via CLI or env:</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-3-1"><a id="__codelineno-3-1" name="__codelineno-3-1" href="#__codelineno-3-1"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--mode<span class="w"> </span>token
|
||||
</span><span id="__span-3-2"><a id="__codelineno-3-2" name="__codelineno-3-2" href="#__codelineno-3-2"></a><span class="nv">HEADROOM_MODE</span><span class="o">=</span>cache<span class="w"> </span>headroom<span class="w"> </span>proxy
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-4-1"><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--mode<span class="w"> </span>token
|
||||
</span><span id="__span-4-2"><a id="__codelineno-4-2" name="__codelineno-4-2" href="#__codelineno-4-2"></a><span class="nv">HEADROOM_MODE</span><span class="o">=</span>cache<span class="w"> </span>headroom<span class="w"> </span>proxy
|
||||
</span></code></pre></div>
|
||||
<p>When to pick each:</p>
|
||||
<ul>
|
||||
|
|
@ -2597,11 +2642,11 @@
|
|||
</table>
|
||||
<p>By default, the proxy uses <strong>IntelligentContextManager</strong> which scores messages by multiple factors (recency, semantic similarity, TOIN-learned patterns, error indicators, forward references) and drops lowest-scored messages first. This is smarter than simple age-based truncation.</p>
|
||||
<p><strong>CCR Integration:</strong> When messages are dropped, they're stored in CCR so the LLM can retrieve them if needed. The inserted marker includes the CCR reference. Drops are also recorded to TOIN, so the system learns which message patterns are important across all users.</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-4-1"><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="c1"># Use legacy RollingWindow (drops oldest first)</span>
|
||||
</span><span id="__span-4-2"><a id="__codelineno-4-2" name="__codelineno-4-2" href="#__codelineno-4-2"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--no-intelligent-context
|
||||
</span><span id="__span-4-3"><a id="__codelineno-4-3" name="__codelineno-4-3" href="#__codelineno-4-3"></a>
|
||||
</span><span id="__span-4-4"><a id="__codelineno-4-4" name="__codelineno-4-4" href="#__codelineno-4-4"></a><span class="c1"># Disable semantic scoring (faster, but less intelligent)</span>
|
||||
</span><span id="__span-4-5"><a id="__codelineno-4-5" name="__codelineno-4-5" href="#__codelineno-4-5"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--no-intelligent-scoring
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-5-1"><a id="__codelineno-5-1" name="__codelineno-5-1" href="#__codelineno-5-1"></a><span class="c1"># Use legacy RollingWindow (drops oldest first)</span>
|
||||
</span><span id="__span-5-2"><a id="__codelineno-5-2" name="__codelineno-5-2" href="#__codelineno-5-2"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--no-intelligent-context
|
||||
</span><span id="__span-5-3"><a id="__codelineno-5-3" name="__codelineno-5-3" href="#__codelineno-5-3"></a>
|
||||
</span><span id="__span-5-4"><a id="__codelineno-5-4" name="__codelineno-5-4" href="#__codelineno-5-4"></a><span class="c1"># Disable semantic scoring (faster, but less intelligent)</span>
|
||||
</span><span id="__span-5-5"><a id="__codelineno-5-5" name="__codelineno-5-5" href="#__codelineno-5-5"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--no-intelligent-scoring
|
||||
</span></code></pre></div>
|
||||
<h3 id="llmlingua-options-ml-compression">LLMLingua Options (ML Compression)<a class="headerlink" href="#llmlingua-options-ml-compression" title="Permanent link">¶</a></h3>
|
||||
<table>
|
||||
|
|
@ -2631,71 +2676,72 @@
|
|||
</tbody>
|
||||
</table>
|
||||
<p><strong>Note:</strong> LLMLingua requires additional dependencies: <code>pip install headroom-ai[llmlingua]</code></p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-5-1"><a id="__codelineno-5-1" name="__codelineno-5-1" href="#__codelineno-5-1"></a><span class="c1"># Enable LLMLingua with GPU acceleration</span>
|
||||
</span><span id="__span-5-2"><a id="__codelineno-5-2" name="__codelineno-5-2" href="#__codelineno-5-2"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--llmlingua<span class="w"> </span>--llmlingua-device<span class="w"> </span>cuda
|
||||
</span><span id="__span-5-3"><a id="__codelineno-5-3" name="__codelineno-5-3" href="#__codelineno-5-3"></a>
|
||||
</span><span id="__span-5-4"><a id="__codelineno-5-4" name="__codelineno-5-4" href="#__codelineno-5-4"></a><span class="c1"># More aggressive compression (keep only 20%)</span>
|
||||
</span><span id="__span-5-5"><a id="__codelineno-5-5" name="__codelineno-5-5" href="#__codelineno-5-5"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--llmlingua<span class="w"> </span>--llmlingua-rate<span class="w"> </span><span class="m">0</span>.2
|
||||
</span><span id="__span-5-6"><a id="__codelineno-5-6" name="__codelineno-5-6" href="#__codelineno-5-6"></a>
|
||||
</span><span id="__span-5-7"><a id="__codelineno-5-7" name="__codelineno-5-7" href="#__codelineno-5-7"></a><span class="c1"># Conservative compression for code (keep 50%)</span>
|
||||
</span><span id="__span-5-8"><a id="__codelineno-5-8" name="__codelineno-5-8" href="#__codelineno-5-8"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--llmlingua<span class="w"> </span>--llmlingua-rate<span class="w"> </span><span class="m">0</span>.5
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-6-1"><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a><span class="c1"># Enable LLMLingua with GPU acceleration</span>
|
||||
</span><span id="__span-6-2"><a id="__codelineno-6-2" name="__codelineno-6-2" href="#__codelineno-6-2"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--llmlingua<span class="w"> </span>--llmlingua-device<span class="w"> </span>cuda
|
||||
</span><span id="__span-6-3"><a id="__codelineno-6-3" name="__codelineno-6-3" href="#__codelineno-6-3"></a>
|
||||
</span><span id="__span-6-4"><a id="__codelineno-6-4" name="__codelineno-6-4" href="#__codelineno-6-4"></a><span class="c1"># More aggressive compression (keep only 20%)</span>
|
||||
</span><span id="__span-6-5"><a id="__codelineno-6-5" name="__codelineno-6-5" href="#__codelineno-6-5"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--llmlingua<span class="w"> </span>--llmlingua-rate<span class="w"> </span><span class="m">0</span>.2
|
||||
</span><span id="__span-6-6"><a id="__codelineno-6-6" name="__codelineno-6-6" href="#__codelineno-6-6"></a>
|
||||
</span><span id="__span-6-7"><a id="__codelineno-6-7" name="__codelineno-6-7" href="#__codelineno-6-7"></a><span class="c1"># Conservative compression for code (keep 50%)</span>
|
||||
</span><span id="__span-6-8"><a id="__codelineno-6-8" name="__codelineno-6-8" href="#__codelineno-6-8"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--llmlingua<span class="w"> </span>--llmlingua-rate<span class="w"> </span><span class="m">0</span>.5
|
||||
</span></code></pre></div>
|
||||
<h2 id="api-endpoints">API Endpoints<a class="headerlink" href="#api-endpoints" title="Permanent link">¶</a></h2>
|
||||
<h3 id="liveness">Liveness<a class="headerlink" href="#liveness" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-6-1"><a id="__codelineno-6-1" name="__codelineno-6-1" href="#__codelineno-6-1"></a>curl<span class="w"> </span>http://localhost:8787/livez
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-7-1"><a id="__codelineno-7-1" name="__codelineno-7-1" href="#__codelineno-7-1"></a>curl<span class="w"> </span>http://localhost:8787/livez
|
||||
</span></code></pre></div>
|
||||
<p>Response:
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-7-1"><a id="__codelineno-7-1" name="__codelineno-7-1" href="#__codelineno-7-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-7-2"><a id="__codelineno-7-2" name="__codelineno-7-2" href="#__codelineno-7-2"></a><span class="w"> </span><span class="nt">"service"</span><span class="p">:</span><span class="w"> </span><span class="s2">"headroom-proxy"</span><span class="p">,</span>
|
||||
</span><span id="__span-7-3"><a id="__codelineno-7-3" name="__codelineno-7-3" href="#__codelineno-7-3"></a><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">,</span>
|
||||
</span><span id="__span-7-4"><a id="__codelineno-7-4" name="__codelineno-7-4" href="#__codelineno-7-4"></a><span class="w"> </span><span class="nt">"alive"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-7-5"><a id="__codelineno-7-5" name="__codelineno-7-5" href="#__codelineno-7-5"></a><span class="w"> </span><span class="nt">"version"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0.5.21"</span><span class="p">,</span>
|
||||
</span><span id="__span-7-6"><a id="__codelineno-7-6" name="__codelineno-7-6" href="#__codelineno-7-6"></a><span class="w"> </span><span class="nt">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2026-04-10T16:36:25Z"</span><span class="p">,</span>
|
||||
</span><span id="__span-7-7"><a id="__codelineno-7-7" name="__codelineno-7-7" href="#__codelineno-7-7"></a><span class="w"> </span><span class="nt">"uptime_seconds"</span><span class="p">:</span><span class="w"> </span><span class="mf">12.483</span>
|
||||
</span><span id="__span-7-8"><a id="__codelineno-7-8" name="__codelineno-7-8" href="#__codelineno-7-8"></a><span class="p">}</span>
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-8-1"><a id="__codelineno-8-1" name="__codelineno-8-1" href="#__codelineno-8-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-8-2"><a id="__codelineno-8-2" name="__codelineno-8-2" href="#__codelineno-8-2"></a><span class="w"> </span><span class="nt">"service"</span><span class="p">:</span><span class="w"> </span><span class="s2">"headroom-proxy"</span><span class="p">,</span>
|
||||
</span><span id="__span-8-3"><a id="__codelineno-8-3" name="__codelineno-8-3" href="#__codelineno-8-3"></a><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">,</span>
|
||||
</span><span id="__span-8-4"><a id="__codelineno-8-4" name="__codelineno-8-4" href="#__codelineno-8-4"></a><span class="w"> </span><span class="nt">"alive"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-8-5"><a id="__codelineno-8-5" name="__codelineno-8-5" href="#__codelineno-8-5"></a><span class="w"> </span><span class="nt">"version"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0.5.21"</span><span class="p">,</span>
|
||||
</span><span id="__span-8-6"><a id="__codelineno-8-6" name="__codelineno-8-6" href="#__codelineno-8-6"></a><span class="w"> </span><span class="nt">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2026-04-10T16:36:25Z"</span><span class="p">,</span>
|
||||
</span><span id="__span-8-7"><a id="__codelineno-8-7" name="__codelineno-8-7" href="#__codelineno-8-7"></a><span class="w"> </span><span class="nt">"uptime_seconds"</span><span class="p">:</span><span class="w"> </span><span class="mf">12.483</span>
|
||||
</span><span id="__span-8-8"><a id="__codelineno-8-8" name="__codelineno-8-8" href="#__codelineno-8-8"></a><span class="p">}</span>
|
||||
</span></code></pre></div></p>
|
||||
<h3 id="readiness">Readiness<a class="headerlink" href="#readiness" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-8-1"><a id="__codelineno-8-1" name="__codelineno-8-1" href="#__codelineno-8-1"></a>curl<span class="w"> </span>http://localhost:8787/readyz
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-9-1"><a id="__codelineno-9-1" name="__codelineno-9-1" href="#__codelineno-9-1"></a>curl<span class="w"> </span>http://localhost:8787/readyz
|
||||
</span></code></pre></div>
|
||||
<p>Response:
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-9-1"><a id="__codelineno-9-1" name="__codelineno-9-1" href="#__codelineno-9-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-9-2"><a id="__codelineno-9-2" name="__codelineno-9-2" href="#__codelineno-9-2"></a><span class="w"> </span><span class="nt">"service"</span><span class="p">:</span><span class="w"> </span><span class="s2">"headroom-proxy"</span><span class="p">,</span>
|
||||
</span><span id="__span-9-3"><a id="__codelineno-9-3" name="__codelineno-9-3" href="#__codelineno-9-3"></a><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">,</span>
|
||||
</span><span id="__span-9-4"><a id="__codelineno-9-4" name="__codelineno-9-4" href="#__codelineno-9-4"></a><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-9-5"><a id="__codelineno-9-5" name="__codelineno-9-5" href="#__codelineno-9-5"></a><span class="w"> </span><span class="nt">"version"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0.5.21"</span><span class="p">,</span>
|
||||
</span><span id="__span-9-6"><a id="__codelineno-9-6" name="__codelineno-9-6" href="#__codelineno-9-6"></a><span class="w"> </span><span class="nt">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2026-04-10T16:36:25Z"</span><span class="p">,</span>
|
||||
</span><span id="__span-9-7"><a id="__codelineno-9-7" name="__codelineno-9-7" href="#__codelineno-9-7"></a><span class="w"> </span><span class="nt">"uptime_seconds"</span><span class="p">:</span><span class="w"> </span><span class="mf">12.483</span><span class="p">,</span>
|
||||
</span><span id="__span-9-8"><a id="__codelineno-9-8" name="__codelineno-9-8" href="#__codelineno-9-8"></a><span class="w"> </span><span class="nt">"checks"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
|
||||
</span><span id="__span-9-9"><a id="__codelineno-9-9" name="__codelineno-9-9" href="#__codelineno-9-9"></a><span class="w"> </span><span class="nt">"startup"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-9-10"><a id="__codelineno-9-10" name="__codelineno-9-10" href="#__codelineno-9-10"></a><span class="w"> </span><span class="nt">"http_client"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-9-11"><a id="__codelineno-9-11" name="__codelineno-9-11" href="#__codelineno-9-11"></a><span class="w"> </span><span class="nt">"cache"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-9-12"><a id="__codelineno-9-12" name="__codelineno-9-12" href="#__codelineno-9-12"></a><span class="w"> </span><span class="nt">"rate_limiter"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-9-13"><a id="__codelineno-9-13" name="__codelineno-9-13" href="#__codelineno-9-13"></a><span class="w"> </span><span class="nt">"memory"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"disabled"</span><span class="p">}</span>
|
||||
</span><span id="__span-9-14"><a id="__codelineno-9-14" name="__codelineno-9-14" href="#__codelineno-9-14"></a><span class="w"> </span><span class="p">}</span>
|
||||
</span><span id="__span-9-15"><a id="__codelineno-9-15" name="__codelineno-9-15" href="#__codelineno-9-15"></a><span class="p">}</span>
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-10-1"><a id="__codelineno-10-1" name="__codelineno-10-1" href="#__codelineno-10-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-10-2"><a id="__codelineno-10-2" name="__codelineno-10-2" href="#__codelineno-10-2"></a><span class="w"> </span><span class="nt">"service"</span><span class="p">:</span><span class="w"> </span><span class="s2">"headroom-proxy"</span><span class="p">,</span>
|
||||
</span><span id="__span-10-3"><a id="__codelineno-10-3" name="__codelineno-10-3" href="#__codelineno-10-3"></a><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">,</span>
|
||||
</span><span id="__span-10-4"><a id="__codelineno-10-4" name="__codelineno-10-4" href="#__codelineno-10-4"></a><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-10-5"><a id="__codelineno-10-5" name="__codelineno-10-5" href="#__codelineno-10-5"></a><span class="w"> </span><span class="nt">"version"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0.5.21"</span><span class="p">,</span>
|
||||
</span><span id="__span-10-6"><a id="__codelineno-10-6" name="__codelineno-10-6" href="#__codelineno-10-6"></a><span class="w"> </span><span class="nt">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2026-04-10T16:36:25Z"</span><span class="p">,</span>
|
||||
</span><span id="__span-10-7"><a id="__codelineno-10-7" name="__codelineno-10-7" href="#__codelineno-10-7"></a><span class="w"> </span><span class="nt">"uptime_seconds"</span><span class="p">:</span><span class="w"> </span><span class="mf">12.483</span><span class="p">,</span>
|
||||
</span><span id="__span-10-8"><a id="__codelineno-10-8" name="__codelineno-10-8" href="#__codelineno-10-8"></a><span class="w"> </span><span class="nt">"checks"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
|
||||
</span><span id="__span-10-9"><a id="__codelineno-10-9" name="__codelineno-10-9" href="#__codelineno-10-9"></a><span class="w"> </span><span class="nt">"startup"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-10-10"><a id="__codelineno-10-10" name="__codelineno-10-10" href="#__codelineno-10-10"></a><span class="w"> </span><span class="nt">"http_client"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-10-11"><a id="__codelineno-10-11" name="__codelineno-10-11" href="#__codelineno-10-11"></a><span class="w"> </span><span class="nt">"cache"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-10-12"><a id="__codelineno-10-12" name="__codelineno-10-12" href="#__codelineno-10-12"></a><span class="w"> </span><span class="nt">"rate_limiter"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-10-13"><a id="__codelineno-10-13" name="__codelineno-10-13" href="#__codelineno-10-13"></a><span class="w"> </span><span class="nt">"memory"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"disabled"</span><span class="p">}</span>
|
||||
</span><span id="__span-10-14"><a id="__codelineno-10-14" name="__codelineno-10-14" href="#__codelineno-10-14"></a><span class="w"> </span><span class="p">}</span>
|
||||
</span><span id="__span-10-15"><a id="__codelineno-10-15" name="__codelineno-10-15" href="#__codelineno-10-15"></a><span class="p">}</span>
|
||||
</span></code></pre></div></p>
|
||||
<p><code>/readyz</code> returns HTTP 503 when Headroom has not completed startup or a required enabled subsystem is unavailable. This is the endpoint used by the container health checks.</p>
|
||||
<h3 id="aggregate-health">Aggregate Health<a class="headerlink" href="#aggregate-health" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-10-1"><a id="__codelineno-10-1" name="__codelineno-10-1" href="#__codelineno-10-1"></a>curl<span class="w"> </span>http://localhost:8787/health
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-11-1"><a id="__codelineno-11-1" name="__codelineno-11-1" href="#__codelineno-11-1"></a>curl<span class="w"> </span>http://localhost:8787/health
|
||||
</span></code></pre></div>
|
||||
<p>Response:
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-11-1"><a id="__codelineno-11-1" name="__codelineno-11-1" href="#__codelineno-11-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-11-2"><a id="__codelineno-11-2" name="__codelineno-11-2" href="#__codelineno-11-2"></a><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">,</span>
|
||||
</span><span id="__span-11-3"><a id="__codelineno-11-3" name="__codelineno-11-3" href="#__codelineno-11-3"></a><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-11-4"><a id="__codelineno-11-4" name="__codelineno-11-4" href="#__codelineno-11-4"></a><span class="w"> </span><span class="nt">"version"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0.5.21"</span><span class="p">,</span>
|
||||
</span><span id="__span-11-5"><a id="__codelineno-11-5" name="__codelineno-11-5" href="#__codelineno-11-5"></a><span class="w"> </span><span class="nt">"config"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
|
||||
</span><span id="__span-11-6"><a id="__codelineno-11-6" name="__codelineno-11-6" href="#__codelineno-11-6"></a><span class="w"> </span><span class="nt">"optimize"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-11-7"><a id="__codelineno-11-7" name="__codelineno-11-7" href="#__codelineno-11-7"></a><span class="w"> </span><span class="nt">"cache"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-11-8"><a id="__codelineno-11-8" name="__codelineno-11-8" href="#__codelineno-11-8"></a><span class="w"> </span><span class="nt">"rate_limit"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span>
|
||||
</span><span id="__span-11-9"><a id="__codelineno-11-9" name="__codelineno-11-9" href="#__codelineno-11-9"></a><span class="w"> </span><span class="p">},</span>
|
||||
</span><span id="__span-11-10"><a id="__codelineno-11-10" name="__codelineno-11-10" href="#__codelineno-11-10"></a><span class="w"> </span><span class="nt">"checks"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
|
||||
</span><span id="__span-11-11"><a id="__codelineno-11-11" name="__codelineno-11-11" href="#__codelineno-11-11"></a><span class="w"> </span><span class="nt">"startup"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-11-12"><a id="__codelineno-11-12" name="__codelineno-11-12" href="#__codelineno-11-12"></a><span class="w"> </span><span class="nt">"http_client"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">}</span>
|
||||
</span><span id="__span-11-13"><a id="__codelineno-11-13" name="__codelineno-11-13" href="#__codelineno-11-13"></a><span class="w"> </span><span class="p">}</span>
|
||||
</span><span id="__span-11-14"><a id="__codelineno-11-14" name="__codelineno-11-14" href="#__codelineno-11-14"></a><span class="p">}</span>
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-12-1"><a id="__codelineno-12-1" name="__codelineno-12-1" href="#__codelineno-12-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-12-2"><a id="__codelineno-12-2" name="__codelineno-12-2" href="#__codelineno-12-2"></a><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">,</span>
|
||||
</span><span id="__span-12-3"><a id="__codelineno-12-3" name="__codelineno-12-3" href="#__codelineno-12-3"></a><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-12-4"><a id="__codelineno-12-4" name="__codelineno-12-4" href="#__codelineno-12-4"></a><span class="w"> </span><span class="nt">"version"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0.5.21"</span><span class="p">,</span>
|
||||
</span><span id="__span-12-5"><a id="__codelineno-12-5" name="__codelineno-12-5" href="#__codelineno-12-5"></a><span class="w"> </span><span class="nt">"config"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
|
||||
</span><span id="__span-12-6"><a id="__codelineno-12-6" name="__codelineno-12-6" href="#__codelineno-12-6"></a><span class="w"> </span><span class="nt">"backend"</span><span class="p">:</span><span class="w"> </span><span class="s2">"anthropic"</span><span class="p">,</span>
|
||||
</span><span id="__span-12-7"><a id="__codelineno-12-7" name="__codelineno-12-7" href="#__codelineno-12-7"></a><span class="w"> </span><span class="nt">"optimize"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-12-8"><a id="__codelineno-12-8" name="__codelineno-12-8" href="#__codelineno-12-8"></a><span class="w"> </span><span class="nt">"cache"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||||
</span><span id="__span-12-9"><a id="__codelineno-12-9" name="__codelineno-12-9" href="#__codelineno-12-9"></a><span class="w"> </span><span class="nt">"rate_limit"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span>
|
||||
</span><span id="__span-12-10"><a id="__codelineno-12-10" name="__codelineno-12-10" href="#__codelineno-12-10"></a><span class="w"> </span><span class="p">},</span>
|
||||
</span><span id="__span-12-11"><a id="__codelineno-12-11" name="__codelineno-12-11" href="#__codelineno-12-11"></a><span class="w"> </span><span class="nt">"checks"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
|
||||
</span><span id="__span-12-12"><a id="__codelineno-12-12" name="__codelineno-12-12" href="#__codelineno-12-12"></a><span class="w"> </span><span class="nt">"startup"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">},</span>
|
||||
</span><span id="__span-12-13"><a id="__codelineno-12-13" name="__codelineno-12-13" href="#__codelineno-12-13"></a><span class="w"> </span><span class="nt">"http_client"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nt">"enabled"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"ready"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w"> </span><span class="nt">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"healthy"</span><span class="p">}</span>
|
||||
</span><span id="__span-12-14"><a id="__codelineno-12-14" name="__codelineno-12-14" href="#__codelineno-12-14"></a><span class="w"> </span><span class="p">}</span>
|
||||
</span><span id="__span-12-15"><a id="__codelineno-12-15" name="__codelineno-12-15" href="#__codelineno-12-15"></a><span class="p">}</span>
|
||||
</span></code></pre></div></p>
|
||||
<h3 id="detailed-statistics">Detailed Statistics<a class="headerlink" href="#detailed-statistics" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-12-1"><a id="__codelineno-12-1" name="__codelineno-12-1" href="#__codelineno-12-1"></a>curl<span class="w"> </span>http://localhost:8787/stats
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-13-1"><a id="__codelineno-13-1" name="__codelineno-13-1" href="#__codelineno-13-1"></a>curl<span class="w"> </span>http://localhost:8787/stats
|
||||
</span></code></pre></div>
|
||||
<p><code>/stats</code> remains the live/session-oriented endpoint and now also includes a
|
||||
<code>persistent_savings</code> block with durable proxy compression lifetime totals plus a
|
||||
|
|
@ -2711,7 +2757,7 @@ observed TTL breakdowns under <code>prefix_cache</code>:</p>
|
|||
<p>These are provider-reported observations, not configured TTL and not remaining
|
||||
expiration time.</p>
|
||||
<h3 id="historical-savings">Historical Savings<a class="headerlink" href="#historical-savings" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-13-1"><a id="__codelineno-13-1" name="__codelineno-13-1" href="#__codelineno-13-1"></a>curl<span class="w"> </span>http://localhost:8787/stats-history
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-14-1"><a id="__codelineno-14-1" name="__codelineno-14-1" href="#__codelineno-14-1"></a>curl<span class="w"> </span>http://localhost:8787/stats-history
|
||||
</span></code></pre></div>
|
||||
<p><code>/stats-history</code> exposes durable proxy compression history for dashboards and
|
||||
other Headroom frontends. It returns:</p>
|
||||
|
|
@ -2725,49 +2771,49 @@ other Headroom frontends. It returns:</p>
|
|||
Set <code>HEADROOM_SAVINGS_PATH</code> to override the location.</p>
|
||||
<p><code>/dashboard</code> uses this endpoint directly for its historical view, including the
|
||||
daily/weekly/monthly rollups and built-in JSON / CSV export buttons.</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-14-1"><a id="__codelineno-14-1" name="__codelineno-14-1" href="#__codelineno-14-1"></a>curl<span class="w"> </span><span class="s2">"http://localhost:8787/stats-history?format=csv&series=weekly"</span>
|
||||
</span><span id="__span-14-2"><a id="__codelineno-14-2" name="__codelineno-14-2" href="#__codelineno-14-2"></a>curl<span class="w"> </span><span class="s2">"http://localhost:8787/stats-history?format=csv&series=monthly"</span>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-15-1"><a id="__codelineno-15-1" name="__codelineno-15-1" href="#__codelineno-15-1"></a>curl<span class="w"> </span><span class="s2">"http://localhost:8787/stats-history?format=csv&series=weekly"</span>
|
||||
</span><span id="__span-15-2"><a id="__codelineno-15-2" name="__codelineno-15-2" href="#__codelineno-15-2"></a>curl<span class="w"> </span><span class="s2">"http://localhost:8787/stats-history?format=csv&series=monthly"</span>
|
||||
</span></code></pre></div>
|
||||
<h3 id="prometheus-metrics">Prometheus Metrics<a class="headerlink" href="#prometheus-metrics" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-15-1"><a id="__codelineno-15-1" name="__codelineno-15-1" href="#__codelineno-15-1"></a>curl<span class="w"> </span>http://localhost:8787/metrics
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-16-1"><a id="__codelineno-16-1" name="__codelineno-16-1" href="#__codelineno-16-1"></a>curl<span class="w"> </span>http://localhost:8787/metrics
|
||||
</span></code></pre></div>
|
||||
<p><code>/metrics</code> remains the built-in Prometheus-formatted operational view. The proxy now also emits the same operational events through the OTEL facade when OTEL metrics are configured.</p>
|
||||
<h3 id="llm-apis">LLM APIs<a class="headerlink" href="#llm-apis" title="Permanent link">¶</a></h3>
|
||||
<p>The proxy supports both Anthropic and OpenAI API formats:</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-16-1"><a id="__codelineno-16-1" name="__codelineno-16-1" href="#__codelineno-16-1"></a><span class="c1"># Anthropic format</span>
|
||||
</span><span id="__span-16-2"><a id="__codelineno-16-2" name="__codelineno-16-2" href="#__codelineno-16-2"></a>POST<span class="w"> </span>/v1/messages
|
||||
</span><span id="__span-16-3"><a id="__codelineno-16-3" name="__codelineno-16-3" href="#__codelineno-16-3"></a>
|
||||
</span><span id="__span-16-4"><a id="__codelineno-16-4" name="__codelineno-16-4" href="#__codelineno-16-4"></a><span class="c1"># OpenAI format</span>
|
||||
</span><span id="__span-16-5"><a id="__codelineno-16-5" name="__codelineno-16-5" href="#__codelineno-16-5"></a>POST<span class="w"> </span>/v1/chat/completions
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-17-1"><a id="__codelineno-17-1" name="__codelineno-17-1" href="#__codelineno-17-1"></a><span class="c1"># Anthropic format</span>
|
||||
</span><span id="__span-17-2"><a id="__codelineno-17-2" name="__codelineno-17-2" href="#__codelineno-17-2"></a>POST<span class="w"> </span>/v1/messages
|
||||
</span><span id="__span-17-3"><a id="__codelineno-17-3" name="__codelineno-17-3" href="#__codelineno-17-3"></a>
|
||||
</span><span id="__span-17-4"><a id="__codelineno-17-4" name="__codelineno-17-4" href="#__codelineno-17-4"></a><span class="c1"># OpenAI format</span>
|
||||
</span><span id="__span-17-5"><a id="__codelineno-17-5" name="__codelineno-17-5" href="#__codelineno-17-5"></a>POST<span class="w"> </span>/v1/chat/completions
|
||||
</span></code></pre></div>
|
||||
<h3 id="post-v1compress"><code>POST /v1/compress</code><a class="headerlink" href="#post-v1compress" title="Permanent link">¶</a></h3>
|
||||
<p>Compression-only endpoint. Compresses messages without calling any LLM. Used by the <a href="../typescript-sdk/">TypeScript SDK</a> and any HTTP client that wants compression as a service.</p>
|
||||
<p><strong>Request:</strong>
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-17-1"><a id="__codelineno-17-1" name="__codelineno-17-1" href="#__codelineno-17-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-17-2"><a id="__codelineno-17-2" name="__codelineno-17-2" href="#__codelineno-17-2"></a><span class="w"> </span><span class="nt">"messages"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="err">...</span><span class="p">],</span><span class="w"> </span><span class="c1">// OpenAI chat format</span>
|
||||
</span><span id="__span-17-3"><a id="__codelineno-17-3" name="__codelineno-17-3" href="#__codelineno-17-3"></a><span class="w"> </span><span class="nt">"model"</span><span class="p">:</span><span class="w"> </span><span class="s2">"gpt-4o"</span><span class="w"> </span><span class="c1">// model name (for token counting)</span>
|
||||
</span><span id="__span-17-4"><a id="__codelineno-17-4" name="__codelineno-17-4" href="#__codelineno-17-4"></a><span class="p">}</span>
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-18-1"><a id="__codelineno-18-1" name="__codelineno-18-1" href="#__codelineno-18-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-18-2"><a id="__codelineno-18-2" name="__codelineno-18-2" href="#__codelineno-18-2"></a><span class="w"> </span><span class="nt">"messages"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="err">...</span><span class="p">],</span><span class="w"> </span><span class="c1">// OpenAI chat format</span>
|
||||
</span><span id="__span-18-3"><a id="__codelineno-18-3" name="__codelineno-18-3" href="#__codelineno-18-3"></a><span class="w"> </span><span class="nt">"model"</span><span class="p">:</span><span class="w"> </span><span class="s2">"gpt-4o"</span><span class="w"> </span><span class="c1">// model name (for token counting)</span>
|
||||
</span><span id="__span-18-4"><a id="__codelineno-18-4" name="__codelineno-18-4" href="#__codelineno-18-4"></a><span class="p">}</span>
|
||||
</span></code></pre></div></p>
|
||||
<p><strong>Response:</strong>
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-18-1"><a id="__codelineno-18-1" name="__codelineno-18-1" href="#__codelineno-18-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-18-2"><a id="__codelineno-18-2" name="__codelineno-18-2" href="#__codelineno-18-2"></a><span class="w"> </span><span class="nt">"messages"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="err">...</span><span class="p">],</span><span class="w"> </span><span class="c1">// compressed messages</span>
|
||||
</span><span id="__span-18-3"><a id="__codelineno-18-3" name="__codelineno-18-3" href="#__codelineno-18-3"></a><span class="w"> </span><span class="nt">"tokens_before"</span><span class="p">:</span><span class="w"> </span><span class="mi">15000</span><span class="p">,</span>
|
||||
</span><span id="__span-18-4"><a id="__codelineno-18-4" name="__codelineno-18-4" href="#__codelineno-18-4"></a><span class="w"> </span><span class="nt">"tokens_after"</span><span class="p">:</span><span class="w"> </span><span class="mi">3500</span><span class="p">,</span>
|
||||
</span><span id="__span-18-5"><a id="__codelineno-18-5" name="__codelineno-18-5" href="#__codelineno-18-5"></a><span class="w"> </span><span class="nt">"tokens_saved"</span><span class="p">:</span><span class="w"> </span><span class="mi">11500</span><span class="p">,</span>
|
||||
</span><span id="__span-18-6"><a id="__codelineno-18-6" name="__codelineno-18-6" href="#__codelineno-18-6"></a><span class="w"> </span><span class="nt">"compression_ratio"</span><span class="p">:</span><span class="w"> </span><span class="mf">0.23</span><span class="p">,</span>
|
||||
</span><span id="__span-18-7"><a id="__codelineno-18-7" name="__codelineno-18-7" href="#__codelineno-18-7"></a><span class="w"> </span><span class="nt">"transforms_applied"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"router:smart_crusher:0.35"</span><span class="p">],</span>
|
||||
</span><span id="__span-18-8"><a id="__codelineno-18-8" name="__codelineno-18-8" href="#__codelineno-18-8"></a><span class="w"> </span><span class="nt">"ccr_hashes"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"a1b2c3"</span><span class="p">]</span>
|
||||
</span><span id="__span-18-9"><a id="__codelineno-18-9" name="__codelineno-18-9" href="#__codelineno-18-9"></a><span class="p">}</span>
|
||||
<div class="language-json highlight"><pre><span></span><code><span id="__span-19-1"><a id="__codelineno-19-1" name="__codelineno-19-1" href="#__codelineno-19-1"></a><span class="p">{</span>
|
||||
</span><span id="__span-19-2"><a id="__codelineno-19-2" name="__codelineno-19-2" href="#__codelineno-19-2"></a><span class="w"> </span><span class="nt">"messages"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="err">...</span><span class="p">],</span><span class="w"> </span><span class="c1">// compressed messages</span>
|
||||
</span><span id="__span-19-3"><a id="__codelineno-19-3" name="__codelineno-19-3" href="#__codelineno-19-3"></a><span class="w"> </span><span class="nt">"tokens_before"</span><span class="p">:</span><span class="w"> </span><span class="mi">15000</span><span class="p">,</span>
|
||||
</span><span id="__span-19-4"><a id="__codelineno-19-4" name="__codelineno-19-4" href="#__codelineno-19-4"></a><span class="w"> </span><span class="nt">"tokens_after"</span><span class="p">:</span><span class="w"> </span><span class="mi">3500</span><span class="p">,</span>
|
||||
</span><span id="__span-19-5"><a id="__codelineno-19-5" name="__codelineno-19-5" href="#__codelineno-19-5"></a><span class="w"> </span><span class="nt">"tokens_saved"</span><span class="p">:</span><span class="w"> </span><span class="mi">11500</span><span class="p">,</span>
|
||||
</span><span id="__span-19-6"><a id="__codelineno-19-6" name="__codelineno-19-6" href="#__codelineno-19-6"></a><span class="w"> </span><span class="nt">"compression_ratio"</span><span class="p">:</span><span class="w"> </span><span class="mf">0.23</span><span class="p">,</span>
|
||||
</span><span id="__span-19-7"><a id="__codelineno-19-7" name="__codelineno-19-7" href="#__codelineno-19-7"></a><span class="w"> </span><span class="nt">"transforms_applied"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"router:smart_crusher:0.35"</span><span class="p">],</span>
|
||||
</span><span id="__span-19-8"><a id="__codelineno-19-8" name="__codelineno-19-8" href="#__codelineno-19-8"></a><span class="w"> </span><span class="nt">"ccr_hashes"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"a1b2c3"</span><span class="p">]</span>
|
||||
</span><span id="__span-19-9"><a id="__codelineno-19-9" name="__codelineno-19-9" href="#__codelineno-19-9"></a><span class="p">}</span>
|
||||
</span></code></pre></div></p>
|
||||
<p><strong>Headers:</strong>
|
||||
- <code>x-headroom-bypass: true</code> — skip compression, return messages as-is</p>
|
||||
<p><strong>Error responses:</strong> 400 (missing fields), 401 (bad API key), 503 (compression failed)</p>
|
||||
<h2 id="using-with-claude-code">Using with Claude Code<a class="headerlink" href="#using-with-claude-code" title="Permanent link">¶</a></h2>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-19-1"><a id="__codelineno-19-1" name="__codelineno-19-1" href="#__codelineno-19-1"></a><span class="c1"># Start proxy</span>
|
||||
</span><span id="__span-19-2"><a id="__codelineno-19-2" name="__codelineno-19-2" href="#__codelineno-19-2"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--port<span class="w"> </span><span class="m">8787</span>
|
||||
</span><span id="__span-19-3"><a id="__codelineno-19-3" name="__codelineno-19-3" href="#__codelineno-19-3"></a>
|
||||
</span><span id="__span-19-4"><a id="__codelineno-19-4" name="__codelineno-19-4" href="#__codelineno-19-4"></a><span class="c1"># In another terminal</span>
|
||||
</span><span id="__span-19-5"><a id="__codelineno-19-5" name="__codelineno-19-5" href="#__codelineno-19-5"></a><span class="nv">ANTHROPIC_BASE_URL</span><span class="o">=</span>http://localhost:8787<span class="w"> </span>claude
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-20-1"><a id="__codelineno-20-1" name="__codelineno-20-1" href="#__codelineno-20-1"></a><span class="c1"># Start proxy</span>
|
||||
</span><span id="__span-20-2"><a id="__codelineno-20-2" name="__codelineno-20-2" href="#__codelineno-20-2"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--port<span class="w"> </span><span class="m">8787</span>
|
||||
</span><span id="__span-20-3"><a id="__codelineno-20-3" name="__codelineno-20-3" href="#__codelineno-20-3"></a>
|
||||
</span><span id="__span-20-4"><a id="__codelineno-20-4" name="__codelineno-20-4" href="#__codelineno-20-4"></a><span class="c1"># In another terminal</span>
|
||||
</span><span id="__span-20-5"><a id="__codelineno-20-5" name="__codelineno-20-5" href="#__codelineno-20-5"></a><span class="nv">ANTHROPIC_BASE_URL</span><span class="o">=</span>http://localhost:8787<span class="w"> </span>claude
|
||||
</span></code></pre></div>
|
||||
<h2 id="using-with-cursor">Using with Cursor<a class="headerlink" href="#using-with-cursor" title="Permanent link">¶</a></h2>
|
||||
<ol>
|
||||
|
|
@ -2775,31 +2821,31 @@ daily/weekly/monthly rollups and built-in JSON / CSV export buttons.</p>
|
|||
<li>In Cursor settings, set the base URL to <code>http://localhost:8787</code></li>
|
||||
</ol>
|
||||
<h2 id="using-with-openai-sdk">Using with OpenAI SDK<a class="headerlink" href="#using-with-openai-sdk" title="Permanent link">¶</a></h2>
|
||||
<div class="language-python highlight"><pre><span></span><code><span id="__span-20-1"><a id="__codelineno-20-1" name="__codelineno-20-1" href="#__codelineno-20-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">openai</span><span class="w"> </span><span class="kn">import</span> <span class="n">OpenAI</span>
|
||||
</span><span id="__span-20-2"><a id="__codelineno-20-2" name="__codelineno-20-2" href="#__codelineno-20-2"></a>
|
||||
</span><span id="__span-20-3"><a id="__codelineno-20-3" name="__codelineno-20-3" href="#__codelineno-20-3"></a><span class="n">client</span> <span class="o">=</span> <span class="n">OpenAI</span><span class="p">(</span>
|
||||
</span><span id="__span-20-4"><a id="__codelineno-20-4" name="__codelineno-20-4" href="#__codelineno-20-4"></a> <span class="n">base_url</span><span class="o">=</span><span class="s2">"http://localhost:8787/v1"</span><span class="p">,</span>
|
||||
</span><span id="__span-20-5"><a id="__codelineno-20-5" name="__codelineno-20-5" href="#__codelineno-20-5"></a> <span class="n">api_key</span><span class="o">=</span><span class="s2">"your-api-key"</span><span class="p">,</span> <span class="c1"># Still needed for upstream</span>
|
||||
</span><span id="__span-20-6"><a id="__codelineno-20-6" name="__codelineno-20-6" href="#__codelineno-20-6"></a><span class="p">)</span>
|
||||
<div class="language-python highlight"><pre><span></span><code><span id="__span-21-1"><a id="__codelineno-21-1" name="__codelineno-21-1" href="#__codelineno-21-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">openai</span><span class="w"> </span><span class="kn">import</span> <span class="n">OpenAI</span>
|
||||
</span><span id="__span-21-2"><a id="__codelineno-21-2" name="__codelineno-21-2" href="#__codelineno-21-2"></a>
|
||||
</span><span id="__span-21-3"><a id="__codelineno-21-3" name="__codelineno-21-3" href="#__codelineno-21-3"></a><span class="n">client</span> <span class="o">=</span> <span class="n">OpenAI</span><span class="p">(</span>
|
||||
</span><span id="__span-21-4"><a id="__codelineno-21-4" name="__codelineno-21-4" href="#__codelineno-21-4"></a> <span class="n">base_url</span><span class="o">=</span><span class="s2">"http://localhost:8787/v1"</span><span class="p">,</span>
|
||||
</span><span id="__span-21-5"><a id="__codelineno-21-5" name="__codelineno-21-5" href="#__codelineno-21-5"></a> <span class="n">api_key</span><span class="o">=</span><span class="s2">"your-api-key"</span><span class="p">,</span> <span class="c1"># Still needed for upstream</span>
|
||||
</span><span id="__span-21-6"><a id="__codelineno-21-6" name="__codelineno-21-6" href="#__codelineno-21-6"></a><span class="p">)</span>
|
||||
</span></code></pre></div>
|
||||
<h2 id="features">Features<a class="headerlink" href="#features" title="Permanent link">¶</a></h2>
|
||||
<h3 id="llmlingua-ml-compression-opt-in">LLMLingua ML Compression (Opt-In)<a class="headerlink" href="#llmlingua-ml-compression-opt-in" title="Permanent link">¶</a></h3>
|
||||
<p>When enabled, the proxy uses Microsoft's LLMLingua-2 model for ML-based token compression:</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-21-1"><a id="__codelineno-21-1" name="__codelineno-21-1" href="#__codelineno-21-1"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--llmlingua
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-22-1"><a id="__codelineno-22-1" name="__codelineno-22-1" href="#__codelineno-22-1"></a>headroom<span class="w"> </span>proxy<span class="w"> </span>--llmlingua
|
||||
</span></code></pre></div>
|
||||
<p><strong>How it works:</strong>
|
||||
- LLMLinguaCompressor is added to the transform pipeline (before RollingWindow)
|
||||
- Automatically detects content type (JSON, code, text) and adjusts compression
|
||||
- Stores original content in CCR for retrieval if needed</p>
|
||||
<p><strong>Startup feedback:</strong></p>
|
||||
<div class="language-text highlight"><pre><span></span><code><span id="__span-22-1"><a id="__codelineno-22-1" name="__codelineno-22-1" href="#__codelineno-22-1"></a># When enabled and available:
|
||||
</span><span id="__span-22-2"><a id="__codelineno-22-2" name="__codelineno-22-2" href="#__codelineno-22-2"></a>LLMLingua: ENABLED (device=cuda, rate=0.3)
|
||||
</span><span id="__span-22-3"><a id="__codelineno-22-3" name="__codelineno-22-3" href="#__codelineno-22-3"></a>
|
||||
</span><span id="__span-22-4"><a id="__codelineno-22-4" name="__codelineno-22-4" href="#__codelineno-22-4"></a># When installed but not enabled (helpful hint):
|
||||
</span><span id="__span-22-5"><a id="__codelineno-22-5" name="__codelineno-22-5" href="#__codelineno-22-5"></a>LLMLingua: available (enable with --llmlingua for ML compression)
|
||||
</span><span id="__span-22-6"><a id="__codelineno-22-6" name="__codelineno-22-6" href="#__codelineno-22-6"></a>
|
||||
</span><span id="__span-22-7"><a id="__codelineno-22-7" name="__codelineno-22-7" href="#__codelineno-22-7"></a># When enabled but not installed:
|
||||
</span><span id="__span-22-8"><a id="__codelineno-22-8" name="__codelineno-22-8" href="#__codelineno-22-8"></a>WARNING: LLMLingua requested but not installed. Install with: pip install headroom-ai[llmlingua]
|
||||
<div class="language-text highlight"><pre><span></span><code><span id="__span-23-1"><a id="__codelineno-23-1" name="__codelineno-23-1" href="#__codelineno-23-1"></a># When enabled and available:
|
||||
</span><span id="__span-23-2"><a id="__codelineno-23-2" name="__codelineno-23-2" href="#__codelineno-23-2"></a>LLMLingua: ENABLED (device=cuda, rate=0.3)
|
||||
</span><span id="__span-23-3"><a id="__codelineno-23-3" name="__codelineno-23-3" href="#__codelineno-23-3"></a>
|
||||
</span><span id="__span-23-4"><a id="__codelineno-23-4" name="__codelineno-23-4" href="#__codelineno-23-4"></a># When installed but not enabled (helpful hint):
|
||||
</span><span id="__span-23-5"><a id="__codelineno-23-5" name="__codelineno-23-5" href="#__codelineno-23-5"></a>LLMLingua: available (enable with --llmlingua for ML compression)
|
||||
</span><span id="__span-23-6"><a id="__codelineno-23-6" name="__codelineno-23-6" href="#__codelineno-23-6"></a>
|
||||
</span><span id="__span-23-7"><a id="__codelineno-23-7" name="__codelineno-23-7" href="#__codelineno-23-7"></a># When enabled but not installed:
|
||||
</span><span id="__span-23-8"><a id="__codelineno-23-8" name="__codelineno-23-8" href="#__codelineno-23-8"></a>WARNING: LLMLingua requested but not installed. Install with: pip install headroom-ai[llmlingua]
|
||||
</span></code></pre></div>
|
||||
<p><strong>Why opt-in?</strong>
|
||||
| Concern | Default Proxy | With LLMLingua |
|
||||
|
|
@ -2832,37 +2878,37 @@ daily/weekly/monthly rollups and built-in JSON / CSV export buttons.</p>
|
|||
</ul>
|
||||
<h3 id="prometheus-metrics_1">Prometheus Metrics<a class="headerlink" href="#prometheus-metrics_1" title="Permanent link">¶</a></h3>
|
||||
<p>Export metrics for monitoring:</p>
|
||||
<div class="language-text highlight"><pre><span></span><code><span id="__span-23-1"><a id="__codelineno-23-1" name="__codelineno-23-1" href="#__codelineno-23-1"></a>headroom_requests_total
|
||||
</span><span id="__span-23-2"><a id="__codelineno-23-2" name="__codelineno-23-2" href="#__codelineno-23-2"></a>headroom_tokens_saved_total
|
||||
</span><span id="__span-23-3"><a id="__codelineno-23-3" name="__codelineno-23-3" href="#__codelineno-23-3"></a>headroom_cost_usd_total
|
||||
</span><span id="__span-23-4"><a id="__codelineno-23-4" name="__codelineno-23-4" href="#__codelineno-23-4"></a>headroom_latency_ms_sum
|
||||
<div class="language-text highlight"><pre><span></span><code><span id="__span-24-1"><a id="__codelineno-24-1" name="__codelineno-24-1" href="#__codelineno-24-1"></a>headroom_requests_total
|
||||
</span><span id="__span-24-2"><a id="__codelineno-24-2" name="__codelineno-24-2" href="#__codelineno-24-2"></a>headroom_tokens_saved_total
|
||||
</span><span id="__span-24-3"><a id="__codelineno-24-3" name="__codelineno-24-3" href="#__codelineno-24-3"></a>headroom_cost_usd_total
|
||||
</span><span id="__span-24-4"><a id="__codelineno-24-4" name="__codelineno-24-4" href="#__codelineno-24-4"></a>headroom_latency_ms_sum
|
||||
</span></code></pre></div>
|
||||
<h2 id="configuration-via-environment">Configuration via Environment<a class="headerlink" href="#configuration-via-environment" title="Permanent link">¶</a></h2>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-24-1"><a id="__codelineno-24-1" name="__codelineno-24-1" href="#__codelineno-24-1"></a><span class="nb">export</span><span class="w"> </span><span class="nv">HEADROOM_HOST</span><span class="o">=</span><span class="m">0</span>.0.0.0
|
||||
</span><span id="__span-24-2"><a id="__codelineno-24-2" name="__codelineno-24-2" href="#__codelineno-24-2"></a><span class="nb">export</span><span class="w"> </span><span class="nv">HEADROOM_PORT</span><span class="o">=</span><span class="m">8787</span>
|
||||
</span><span id="__span-24-3"><a id="__codelineno-24-3" name="__codelineno-24-3" href="#__codelineno-24-3"></a><span class="nb">export</span><span class="w"> </span><span class="nv">HEADROOM_BUDGET</span><span class="o">=</span><span class="m">100</span>.0
|
||||
</span><span id="__span-24-4"><a id="__codelineno-24-4" name="__codelineno-24-4" href="#__codelineno-24-4"></a><span class="nb">export</span><span class="w"> </span><span class="nv">OPENAI_TARGET_API_URL</span><span class="o">=</span>https://custom.openai.endpoint.com
|
||||
</span><span id="__span-24-5"><a id="__codelineno-24-5" name="__codelineno-24-5" href="#__codelineno-24-5"></a>headroom<span class="w"> </span>proxy
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-25-1"><a id="__codelineno-25-1" name="__codelineno-25-1" href="#__codelineno-25-1"></a><span class="nb">export</span><span class="w"> </span><span class="nv">HEADROOM_HOST</span><span class="o">=</span><span class="m">0</span>.0.0.0
|
||||
</span><span id="__span-25-2"><a id="__codelineno-25-2" name="__codelineno-25-2" href="#__codelineno-25-2"></a><span class="nb">export</span><span class="w"> </span><span class="nv">HEADROOM_PORT</span><span class="o">=</span><span class="m">8787</span>
|
||||
</span><span id="__span-25-3"><a id="__codelineno-25-3" name="__codelineno-25-3" href="#__codelineno-25-3"></a><span class="nb">export</span><span class="w"> </span><span class="nv">HEADROOM_BUDGET</span><span class="o">=</span><span class="m">100</span>.0
|
||||
</span><span id="__span-25-4"><a id="__codelineno-25-4" name="__codelineno-25-4" href="#__codelineno-25-4"></a><span class="nb">export</span><span class="w"> </span><span class="nv">OPENAI_TARGET_API_URL</span><span class="o">=</span>https://custom.openai.endpoint.com
|
||||
</span><span id="__span-25-5"><a id="__codelineno-25-5" name="__codelineno-25-5" href="#__codelineno-25-5"></a>headroom<span class="w"> </span>proxy
|
||||
</span></code></pre></div>
|
||||
<h2 id="running-in-production">Running in Production<a class="headerlink" href="#running-in-production" title="Permanent link">¶</a></h2>
|
||||
<p>For production deployments:</p>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-25-1"><a id="__codelineno-25-1" name="__codelineno-25-1" href="#__codelineno-25-1"></a><span class="c1"># Use a process manager</span>
|
||||
</span><span id="__span-25-2"><a id="__codelineno-25-2" name="__codelineno-25-2" href="#__codelineno-25-2"></a>pip<span class="w"> </span>install<span class="w"> </span>gunicorn
|
||||
</span><span id="__span-25-3"><a id="__codelineno-25-3" name="__codelineno-25-3" href="#__codelineno-25-3"></a>
|
||||
</span><span id="__span-25-4"><a id="__codelineno-25-4" name="__codelineno-25-4" href="#__codelineno-25-4"></a><span class="c1"># Run with gunicorn</span>
|
||||
</span><span id="__span-25-5"><a id="__codelineno-25-5" name="__codelineno-25-5" href="#__codelineno-25-5"></a>gunicorn<span class="w"> </span>headroom.proxy.server:app<span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-25-6"><a id="__codelineno-25-6" name="__codelineno-25-6" href="#__codelineno-25-6"></a><span class="w"> </span>--workers<span class="w"> </span><span class="m">4</span><span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-25-7"><a id="__codelineno-25-7" name="__codelineno-25-7" href="#__codelineno-25-7"></a><span class="w"> </span>--bind<span class="w"> </span><span class="m">0</span>.0.0.0:8787<span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-25-8"><a id="__codelineno-25-8" name="__codelineno-25-8" href="#__codelineno-25-8"></a><span class="w"> </span>--worker-class<span class="w"> </span>uvicorn.workers.UvicornWorker
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-26-1"><a id="__codelineno-26-1" name="__codelineno-26-1" href="#__codelineno-26-1"></a><span class="c1"># Use a process manager</span>
|
||||
</span><span id="__span-26-2"><a id="__codelineno-26-2" name="__codelineno-26-2" href="#__codelineno-26-2"></a>pip<span class="w"> </span>install<span class="w"> </span>gunicorn
|
||||
</span><span id="__span-26-3"><a id="__codelineno-26-3" name="__codelineno-26-3" href="#__codelineno-26-3"></a>
|
||||
</span><span id="__span-26-4"><a id="__codelineno-26-4" name="__codelineno-26-4" href="#__codelineno-26-4"></a><span class="c1"># Run with gunicorn</span>
|
||||
</span><span id="__span-26-5"><a id="__codelineno-26-5" name="__codelineno-26-5" href="#__codelineno-26-5"></a>gunicorn<span class="w"> </span>headroom.proxy.server:app<span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-26-6"><a id="__codelineno-26-6" name="__codelineno-26-6" href="#__codelineno-26-6"></a><span class="w"> </span>--workers<span class="w"> </span><span class="m">4</span><span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-26-7"><a id="__codelineno-26-7" name="__codelineno-26-7" href="#__codelineno-26-7"></a><span class="w"> </span>--bind<span class="w"> </span><span class="m">0</span>.0.0.0:8787<span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-26-8"><a id="__codelineno-26-8" name="__codelineno-26-8" href="#__codelineno-26-8"></a><span class="w"> </span>--worker-class<span class="w"> </span>uvicorn.workers.UvicornWorker
|
||||
</span></code></pre></div>
|
||||
<p>Or with Docker:</p>
|
||||
<div class="language-dockerfile highlight"><pre><span></span><code><span id="__span-26-1"><a id="__codelineno-26-1" name="__codelineno-26-1" href="#__codelineno-26-1"></a><span class="k">FROM</span><span class="w"> </span><span class="s">python:3.11-slim</span>
|
||||
</span><span id="__span-26-2"><a id="__codelineno-26-2" name="__codelineno-26-2" href="#__codelineno-26-2"></a><span class="k">RUN</span><span class="w"> </span>apt-get<span class="w"> </span>update<span class="w"> </span><span class="o">&&</span><span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>-y<span class="w"> </span>--no-install-recommends<span class="w"> </span>build-essential<span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-26-3"><a id="__codelineno-26-3" name="__codelineno-26-3" href="#__codelineno-26-3"></a><span class="w"> </span><span class="o">&&</span><span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span><span class="s2">"headroom-ai[proxy]"</span><span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-26-4"><a id="__codelineno-26-4" name="__codelineno-26-4" href="#__codelineno-26-4"></a><span class="w"> </span><span class="o">&&</span><span class="w"> </span>apt-get<span class="w"> </span>purge<span class="w"> </span>-y<span class="w"> </span>build-essential<span class="w"> </span><span class="o">&&</span><span class="w"> </span>apt-get<span class="w"> </span>autoremove<span class="w"> </span>-y<span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-26-5"><a id="__codelineno-26-5" name="__codelineno-26-5" href="#__codelineno-26-5"></a><span class="w"> </span><span class="o">&&</span><span class="w"> </span>rm<span class="w"> </span>-rf<span class="w"> </span>/var/lib/apt/lists/*
|
||||
</span><span id="__span-26-6"><a id="__codelineno-26-6" name="__codelineno-26-6" href="#__codelineno-26-6"></a><span class="k">EXPOSE</span><span class="w"> </span><span class="s">8787</span>
|
||||
</span><span id="__span-26-7"><a id="__codelineno-26-7" name="__codelineno-26-7" href="#__codelineno-26-7"></a><span class="k">CMD</span><span class="w"> </span><span class="p">[</span><span class="s2">"headroom"</span><span class="p">,</span><span class="w"> </span><span class="s2">"proxy"</span><span class="p">,</span><span class="w"> </span><span class="s2">"--host"</span><span class="p">,</span><span class="w"> </span><span class="s2">"0.0.0.0"</span><span class="p">]</span>
|
||||
<div class="language-dockerfile highlight"><pre><span></span><code><span id="__span-27-1"><a id="__codelineno-27-1" name="__codelineno-27-1" href="#__codelineno-27-1"></a><span class="k">FROM</span><span class="w"> </span><span class="s">python:3.11-slim</span>
|
||||
</span><span id="__span-27-2"><a id="__codelineno-27-2" name="__codelineno-27-2" href="#__codelineno-27-2"></a><span class="k">RUN</span><span class="w"> </span>apt-get<span class="w"> </span>update<span class="w"> </span><span class="o">&&</span><span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>-y<span class="w"> </span>--no-install-recommends<span class="w"> </span>build-essential<span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-27-3"><a id="__codelineno-27-3" name="__codelineno-27-3" href="#__codelineno-27-3"></a><span class="w"> </span><span class="o">&&</span><span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span><span class="s2">"headroom-ai[proxy]"</span><span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-27-4"><a id="__codelineno-27-4" name="__codelineno-27-4" href="#__codelineno-27-4"></a><span class="w"> </span><span class="o">&&</span><span class="w"> </span>apt-get<span class="w"> </span>purge<span class="w"> </span>-y<span class="w"> </span>build-essential<span class="w"> </span><span class="o">&&</span><span class="w"> </span>apt-get<span class="w"> </span>autoremove<span class="w"> </span>-y<span class="w"> </span><span class="se">\</span>
|
||||
</span><span id="__span-27-5"><a id="__codelineno-27-5" name="__codelineno-27-5" href="#__codelineno-27-5"></a><span class="w"> </span><span class="o">&&</span><span class="w"> </span>rm<span class="w"> </span>-rf<span class="w"> </span>/var/lib/apt/lists/*
|
||||
</span><span id="__span-27-6"><a id="__codelineno-27-6" name="__codelineno-27-6" href="#__codelineno-27-6"></a><span class="k">EXPOSE</span><span class="w"> </span><span class="s">8787</span>
|
||||
</span><span id="__span-27-7"><a id="__codelineno-27-7" name="__codelineno-27-7" href="#__codelineno-27-7"></a><span class="k">CMD</span><span class="w"> </span><span class="p">[</span><span class="s2">"headroom"</span><span class="p">,</span><span class="w"> </span><span class="s2">"proxy"</span><span class="p">,</span><span class="w"> </span><span class="s2">"--host"</span><span class="p">,</span><span class="w"> </span><span class="s2">"0.0.0.0"</span><span class="p">]</span>
|
||||
</span></code></pre></div>
|
||||
<blockquote>
|
||||
<p><strong>Note:</strong> <code>build-essential</code> is required at install time because <code>headroom-ai</code> includes <code>hnswlib</code>, a C++ extension that must be compiled from source. It is removed after installation to keep the image slim.</p>
|
||||
|
|
|
|||
|
|
@ -2487,18 +2487,21 @@
|
|||
</span></code></pre></div>
|
||||
<h3 id="step-2-verify-its-running">Step 2: Verify It's Running<a class="headerlink" href="#step-2-verify-its-running" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-3-1"><a id="__codelineno-3-1" name="__codelineno-3-1" href="#__codelineno-3-1"></a>curl<span class="w"> </span>http://localhost:8787/health
|
||||
</span><span id="__span-3-2"><a id="__codelineno-3-2" name="__codelineno-3-2" href="#__codelineno-3-2"></a><span class="c1"># Expected: {"status": "healthy", "mode": "optimize", ...}</span>
|
||||
</span><span id="__span-3-2"><a id="__codelineno-3-2" name="__codelineno-3-2" href="#__codelineno-3-2"></a><span class="c1"># Expected: {"status":"healthy","ready":true,"config":{"backend":"anthropic",...},...}</span>
|
||||
</span></code></pre></div>
|
||||
<h3 id="step-3-point-your-client">Step 3: Point Your Client<a class="headerlink" href="#step-3-point-your-client" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-4-1"><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="c1"># Claude Code</span>
|
||||
</span><span id="__span-4-2"><a id="__codelineno-4-2" name="__codelineno-4-2" href="#__codelineno-4-2"></a><span class="nv">ANTHROPIC_BASE_URL</span><span class="o">=</span>http://localhost:8787<span class="w"> </span>claude
|
||||
</span><span id="__span-4-3"><a id="__codelineno-4-3" name="__codelineno-4-3" href="#__codelineno-4-3"></a>
|
||||
</span><span id="__span-4-4"><a id="__codelineno-4-4" name="__codelineno-4-4" href="#__codelineno-4-4"></a><span class="c1"># Cursor / Continue / any OpenAI client</span>
|
||||
</span><span id="__span-4-5"><a id="__codelineno-4-5" name="__codelineno-4-5" href="#__codelineno-4-5"></a><span class="nv">OPENAI_BASE_URL</span><span class="o">=</span>http://localhost:8787/v1<span class="w"> </span>your-app
|
||||
</span><span id="__span-4-4"><a id="__codelineno-4-4" name="__codelineno-4-4" href="#__codelineno-4-4"></a><span class="c1"># GitHub Copilot CLI (default Anthropic-style proxy route)</span>
|
||||
</span><span id="__span-4-5"><a id="__codelineno-4-5" name="__codelineno-4-5" href="#__codelineno-4-5"></a>headroom<span class="w"> </span>wrap<span class="w"> </span>copilot<span class="w"> </span>--<span class="w"> </span>--model<span class="w"> </span>claude-sonnet-4-20250514
|
||||
</span><span id="__span-4-6"><a id="__codelineno-4-6" name="__codelineno-4-6" href="#__codelineno-4-6"></a>
|
||||
</span><span id="__span-4-7"><a id="__codelineno-4-7" name="__codelineno-4-7" href="#__codelineno-4-7"></a><span class="c1"># Python</span>
|
||||
</span><span id="__span-4-8"><a id="__codelineno-4-8" name="__codelineno-4-8" href="#__codelineno-4-8"></a><span class="nb">export</span><span class="w"> </span><span class="nv">OPENAI_BASE_URL</span><span class="o">=</span>http://localhost:8787/v1
|
||||
</span><span id="__span-4-9"><a id="__codelineno-4-9" name="__codelineno-4-9" href="#__codelineno-4-9"></a>python<span class="w"> </span>your_script.py
|
||||
</span><span id="__span-4-7"><a id="__codelineno-4-7" name="__codelineno-4-7" href="#__codelineno-4-7"></a><span class="c1"># Cursor / Continue / any OpenAI client</span>
|
||||
</span><span id="__span-4-8"><a id="__codelineno-4-8" name="__codelineno-4-8" href="#__codelineno-4-8"></a><span class="nv">OPENAI_BASE_URL</span><span class="o">=</span>http://localhost:8787/v1<span class="w"> </span>your-app
|
||||
</span><span id="__span-4-9"><a id="__codelineno-4-9" name="__codelineno-4-9" href="#__codelineno-4-9"></a>
|
||||
</span><span id="__span-4-10"><a id="__codelineno-4-10" name="__codelineno-4-10" href="#__codelineno-4-10"></a><span class="c1"># Python</span>
|
||||
</span><span id="__span-4-11"><a id="__codelineno-4-11" name="__codelineno-4-11" href="#__codelineno-4-11"></a><span class="nb">export</span><span class="w"> </span><span class="nv">OPENAI_BASE_URL</span><span class="o">=</span>http://localhost:8787/v1
|
||||
</span><span id="__span-4-12"><a id="__codelineno-4-12" name="__codelineno-4-12" href="#__codelineno-4-12"></a>python<span class="w"> </span>your_script.py
|
||||
</span></code></pre></div>
|
||||
<h3 id="step-4-check-savings">Step 4: Check Savings<a class="headerlink" href="#step-4-check-savings" title="Permanent link">¶</a></h3>
|
||||
<div class="language-bash highlight"><pre><span></span><code><span id="__span-5-1"><a id="__codelineno-5-1" name="__codelineno-5-1" href="#__codelineno-5-1"></a>curl<span class="w"> </span>http://localhost:8787/stats
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
62
sitemap.xml
62
sitemap.xml
|
|
@ -2,126 +2,126 @@
|
|||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/ARCHITECTURE/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/LATENCY_BENCHMARKS/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/LIMITATIONS/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/agno/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/api/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/benchmarks/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/ccr/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/compression/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/configuration/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/errors/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/getting-started/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/image-compression/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/integration-guide/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/langchain/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/learn/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/llmlingua/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/macos-deployment/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/mcp/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/memory/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/metrics/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/proxy/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/quickstart/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/sdk/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/shared-context/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/strands/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/text-compression/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/transforms/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/troubleshooting/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/typescript-sdk/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://chopratejas.github.io/headroom/plans/dynamic-smart-crusher/</loc>
|
||||
<lastmod>2026-04-10</lastmod>
|
||||
<lastmod>2026-04-11</lastmod>
|
||||
</url>
|
||||
</urlset>
|
||||
BIN
sitemap.xml.gz
BIN
sitemap.xml.gz
Binary file not shown.
Loading…
Add table
Add a link
Reference in a new issue