fix(stats): report one "Tokens Saved" headline across every harness (#2737)

## Description

The "tokens saved" figure a user sees depended on which harness they
ran. Headroom saves tool-definition tokens in two accounting shapes,
both legitimate, but the rule was never written down — so two harnesses
silently dropped savings and three surfaces open-coded the sum
differently.

- **Compaction** rewrites the tool array, so both endpoints are
countable → handlers fold the delta into
`original_tokens`/`optimized_tokens`, keeping `tok_before - tok_after ==
tok_saved` coherent.
- **Deferral / hook shrink** removes schemas `count_messages` never sees
→ can only be recorded as a tag, additive to `tokens_saved`.

`tool_schema_savings_policy` now owns the sum via
`headline_tokens_saved()`, and every reporting surface routes through
it.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

Producer gaps (both Anthropic — i.e. Claude Code, the primary harness):

- `anthropic:tool_schema_compaction` / `anthropic:tool_desc_compaction`
computed their savings, debug-logged them, and **discarded them**. Now
folded at the final recount, mirroring the OpenAI chat handler. A
14-tool array drops 786 tokens that previously reported `tok_saved=0`.
- Anthropic never wrote `turn_hook_tools_saved_tokens` at all, so a
turn-hook extension that shrinks tools got zero credit there while
OpenAI credited it. Now tagged.

Reporting gaps:

- `headroom perf` printed `Total saved (messages)` and `Tool saved` as
rival lines — on a tool-heavy session the headline read `0` and the real
win looked like a footnote. Now one `Tokens saved:` headline with a
messages/tool-schemas breakdown.
- `active_savings_percent` divided a **compression-only numerator** by a
denominator that already included compacted tool schema, undercounting
every tool-heavy session. Numerator is now all-layers, with deferred
schemas added to both sides.
- The headline and its percent now share a numerator. Previously the
dashboard tile showed an all-layers total next to a compression-only
percent.
- Session summary and dashboard tile relabelled to `Tokens Saved`; the
tool-schema panel is labelled as a component (`Tokens Saved · Tool
Schemas`) rather than a rival metric.
- `outcome.py` had two drifted inline copies of the tag sum; both now
call the policy module that exists for it. `total_saved=` added to the
PERF line.
- JSON: added `total_tokens_saved` / `total_savings_pct`; existing
`tokens_saved` / `tool_saved` / `savings_pct` keys unchanged for
back-compat.

Not changed by design: the Codex per-component attribution sub-line
would need a 9th positional tuple element threaded through 4 unpack
sites, and its headline is already correct without it.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/ruff check headroom/ tests/test_tool_schema_savings_policy.py --exclude headroom/dashboard/templates
All checks passed!

$ .venv/bin/ruff format --check headroom/ tests/... --exclude headroom/dashboard/templates
510 files already formatted

$ .venv/bin/mypy headroom/
Success: no issues found in 508 source files

$ python -m pytest tests/test_tool_schema_savings_policy.py tests/test_cli_perf_format.py \
    tests/test_request_outcome.py tests/test_savings_tool_search_aggregation.py \
    tests/test_dashboard_token_savings.py tests/test_anthropic_compaction_transforms.py -q
70 passed, 1 warning in 6.15s

$ python -m pytest tests/test_handler_outcome_tag_invariant.py tests/test_cold_start_fast_pass.py \
    tests/test_anthropic_ccr_workspace_unbound.py tests/test_anthropic_pre_upstream_backpressure.py \
    tests/test_vertex_claude_compression.py tests/test_provider_route_specs.py -q
50 passed in 10.11s

$ python -m pytest tests/test_agent_savings.py tests/test_bundled_tools_savings.py \
    tests/test_codex_ws_savings_deferral.py tests/test_savings_ledger_before_forwarded.py \
    tests/test_savings_ledger_offload.py tests/test_proxy_savings_history.py \
    tests/test_proxy_dashboard_stats_cache.py tests/test_output_savings_cli.py -q
97 passed, 2 skipped in 18.60s

$ python -m pytest tests/test_tool_schema_compaction.py tests/test_openai_responses_context_compaction.py \
    tests/test_proxy_openai_cache_stability.py tests/test_codex_ws_compression_scheduler.py \
    tests/test_proxy_streaming_request_logger.py -q
66 passed, 1 skipped in 16.69s
```

## Real Behavior Proof

- **Environment:** macOS 26.4 arm64, Python 3.12.6, repo `.venv`.
Motivated by a real user proxy log (0.33.0, `client=opencode` →
nano-gpt, 722 requests) reporting 0.12% savings.

- **Exact command / steps (1) — the Anthropic fold, real compaction +
real provider tokenizer:**

```python
tok = AnthropicProvider().get_token_counter("claude-sonnet-4-6")
payload = {"tools": [ ...14 tools with $schema/title/examples... ]}
body, modified, bb, ba = compact_tools(payload)
```

**Observed:**

```text
modified=True  bytes 4503->2539  TOKENS 1650->864  delta=786
tok_before=6650 tok_after=5864 tok_saved=786  coherent=True
pre-fix: Claude Code reported tok_saved=0 and discarded 786 tokens
```

Pinned as `test_tool_schema_compaction_saves_real_tokens_not_just_bytes`
— it asserts a positive **token** delta (not just bytes), which is the
premise of folding at all.

- **Exact command / steps (2) — the report, on the reported session's
shape** (tool schemas carry the win, message compression is 0 because
everything routed to `excluded_tool`):

**Observed after:**

```text
Requests:     2
Tokens:       45,760 -> 45,760 (0.0% messages)
Tokens saved: 811 (1.7% reduction)
  · messages       0
  · tool schemas   811

JSON: {'total_tokens_saved': 811, 'total_savings_pct': 1.7, 'tokens_saved': 0,
       'tool_saved': 811, 'savings_pct': 0.0}
```

Before, the same input printed `Total saved: 0 tokens (messages)` as the
headline with `Tool saved: 811` beneath it.

- **Not tested:** no live proxy run against a real provider — the
Anthropic fold is proven at the accounting layer (real `compact_tools` +
real provider tokenizer) and via the existing handler suites, not by an
end-to-end Claude Code session. Dashboard changes are template-label
edits verified by reading `stats.tokens.saved` / `by_layer.tool_search`
shapes, not by a browser screenshot.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

headroom/dashboard/templates/dashboard.html

@@ -209,7 +209,7 @@
                 <div class="grid grid-cols-1 gap-4 mb-6 md:grid-cols-2 lg:grid-cols-3">
                     <!-- Token Savings (%) -->
                     <div class="bg-surface rounded-lg p-4 border border-border">
-                        <div class="text-xs text-gray-500 uppercase tracking-wide mb-1">Token Savings</div>
+                        <div class="text-xs text-gray-500 uppercase tracking-wide mb-1">Tokens Saved</div>
                         <div class="flex items-baseline gap-2">
                             <span class="text-3xl font-light tabular-nums text-accent" x-text="formatNumber(stats.tokens?.saved || 0)"></span>
                             <!-- Active compression ratio: savings as a fraction of tokens we
@@ -270,10 +270,13 @@
                         </template>
                     </div>
 
-                    <!-- Tool-Schema Deferral (tool search) — only rendered when there's a saving to show -->
+                    <!-- Tool-schema deferral: a COMPONENT of the Tokens Saved headline above
+                         (stats.tokens.saved is all-layers), not a rival metric. Labelled as
+                         such so a tool-heavy session doesn't read as "0 saved + some other
+                         number". Only rendered when there's a saving to show. -->
                     <template x-if="(stats.savings?.by_layer?.tool_search?.tokens || 0) > 0">
                         <div class="bg-surface rounded-lg p-4 border border-border">
-                            <div class="text-xs text-gray-500 uppercase tracking-wide mb-1">Tool-Schema Deferral</div>
+                            <div class="text-xs text-gray-500 uppercase tracking-wide mb-1">Tokens Saved · Tool Schemas</div>
                             <div class="flex items-baseline gap-2">
                                 <span class="text-3xl font-light tabular-nums text-emerald-400" x-text="formatNumber(stats.savings?.by_layer?.tool_search?.tokens || 0)"></span>
                                 <span class="text-sm text-gray-400">tokens</span>

headroom/perf/analyzer.py

@@ -483,16 +483,25 @@ def format_report(report: PerfReport) -> str:
         total_after = sum(r.tokens_after for r in records)
         total_saved = sum(r.tokens_saved for r in records)
         total_tool_saved = sum(r.tool_saved for r in records)
+        total_headline_saved = total_saved + total_tool_saved
         pct = (total_saved / total_before * 100) if total_before > 0 else 0
+        # All-layers denominator: deferred tool schemas were never in tok_before (they
+        # don't reach count_messages), so the pre-Headroom world is tok_before + them.
+        # Same construction the proxy's /api/stats uses for total_before_compression —
+        # the headline number and the headline percent must share a numerator, or the
+        # tile reads "60,920 saved (0.1%)" off two different definitions of saved.
+        headline_before = total_before + total_tool_saved
+        headline_pct = (total_headline_saved / headline_before * 100) if headline_before > 0 else 0
 
         lines.append(f"Requests:     {len(records)}")
-        lines.append(f"Tokens:       {total_before:,} -> {total_after:,} ({pct:.1f}% reduction)")
+        lines.append(f"Tokens:       {total_before:,} -> {total_after:,} ({pct:.1f}% messages)")
-        lines.append(f"Total saved:  {total_saved:,} tokens (messages)")
+        # ONE headline. Tool-schema deferral can't move tok_before/after (messages never
-        # Tool-schema savings (deferral + turn-hook tool shrink) are counted apart
+        # include tool bytes), so it used to render as a rival "Tool saved" line — which
-        # from message compression — messages never include tool bytes — so surface
+        # read as a side metric and hid the win on tool-heavy turns where tok_saved=0.
-        # them explicitly instead of hiding a tool-heavy turn's win behind tok_saved=0.
+        lines.append(f"Tokens saved: {total_headline_saved:,} ({headline_pct:.1f}% reduction)")
         if total_tool_saved > 0:
-            lines.append(f"Tool saved:   {total_tool_saved:,} tokens (tool schemas, deferral)")
+            lines.append(f"  · messages       {max(0, total_saved):,}")
+            lines.append(f"  · tool schemas   {total_tool_saved:,}")
         lines.append("")
 
         # Per-model breakdown with list prices
@@ -978,6 +987,7 @@ def build_perf_summary(report: PerfReport) -> dict:
     total_after = sum(r.tokens_after for r in records)
     total_saved = sum(r.tokens_saved for r in records)
     total_tool_saved = sum(r.tool_saved for r in records)
+    total_headline_saved = total_saved + total_tool_saved
 
     total_cr = sum(r.cache_read for r in records)
     total_cw = sum(r.cache_write for r in records)
@@ -1033,6 +1043,11 @@ def build_perf_summary(report: PerfReport) -> dict:
         "total_requests": len(records),
         "total_tokens_before": total_before,
         "total_tokens_after": total_after,
+        # total_tokens_saved is the headline (messages + tool-schema deferral); the two
+        # components stay for consumers that break the number down. See
+        # headroom.proxy.tool_schema_savings_policy.
+        "total_tokens_saved": total_headline_saved,
+        "total_savings_pct": _pct(total_headline_saved, total_before + total_tool_saved),
         "tokens_saved": total_saved,
         "tool_saved": total_tool_saved,
         "savings_pct": _pct(total_saved, total_before),

headroom/proxy/handlers/anthropic.py

@@ -2277,14 +2277,32 @@ class AnthropicHandlerMixin:
             # after tools are finalised (sorting, CCR injection) but before
             # the PRE_SEND pipeline event so extensions see the compacted
             # schema.  Mirrors the same pass that the OpenAI handler applies.
+            #
+            # Token accounting (see tool_schema_savings_policy): the compaction passes
+            # below rewrite the tool array, so both endpoints are countable and the
+            # delta is folded into original/optimized_tokens at the final recount.
+            # Without this the savings were computed, debug-logged, and discarded —
+            # Claude Code reported them nowhere.
+            _tool_tokens_before = 0
+            _tool_tokens_after = 0
+
+            def _count_tool_tokens(value: object) -> int:
+                try:
+                    return tokenizer.count_text(json.dumps(value, default=str))
+                except Exception:
+                    return 0
+
             _tools_compaction_started = time.time()
             try:
                 from headroom.proxy.tool_schema_compaction import compact_tools
 
+                _pre_compaction_tools = body.get("tools")
                 body, _tools_modified, _tools_before_bytes, _tools_after_bytes = compact_tools(body)
                 if _tools_modified:
                     tools = body["tools"]
                     transforms_applied.append("anthropic:tool_schema_compaction")
+                    _tool_tokens_before = _count_tool_tokens(_pre_compaction_tools)
+                    _tool_tokens_after = _count_tool_tokens(tools)
                     _tools_compaction_ms = (time.time() - _tools_compaction_started) * 1000
                     logger.debug(
                         "[%s] tool schema compaction: %d -> %d bytes (%.0f%% saved) in %.1fms",
@@ -2311,12 +2329,18 @@ class AnthropicHandlerMixin:
 
                 _desc_max = tool_desc_max_chars()
                 if _desc_max > 0:
+                    _pre_desc_tools = body.get("tools")
                     body, _desc_modified, _desc_before, _desc_after = compact_tool_descriptions(
                         body, _desc_max
                     )
                     if _desc_modified:
                         tools = body["tools"]
                         transforms_applied.append("anthropic:tool_desc_compaction")
+                        # Runs after schema compaction, so only seed "before" when that
+                        # pass didn't already; "after" always tracks the latest tools.
+                        if not _tool_tokens_before:
+                            _tool_tokens_before = _count_tool_tokens(_pre_desc_tools)
+                        _tool_tokens_after = _count_tool_tokens(tools)
                         logger.debug(
                             "[%s] tool description compaction: %d -> %d bytes (%.0f%% saved, max_chars=%d)",
                             request_id,
@@ -2466,6 +2490,8 @@ class AnthropicHandlerMixin:
                     _pre_hook_tokens = tokenizer.count_messages(optimized_messages)
                 except Exception:
                     _pre_hook_tokens = None
+                _th_tools_before = body.get("tools")
+                _th_tok_before = _count_tool_tokens(_th_tools_before) if _th_tools_before else 0
                 run_request_hooks(_req_ctx)
                 if _req_ctx.messages is not optimized_messages:
                     optimized_messages = _req_ctx.messages
@@ -2473,6 +2499,16 @@ class AnthropicHandlerMixin:
                 if _req_ctx.tools is not body.get("tools"):
                     tools = _req_ctx.tools
                     body["tools"] = tools
+                # A hook may shrink the tool array either by replacing it or in place,
+                # so measure the FINAL tools object. Deferral-shaped (removes schemas
+                # count_messages never saw), hence a tag rather than a fold — mirrors
+                # the OpenAI chat path so a turn-hook extension is credited on both.
+                _th_tok_after = _count_tool_tokens(_req_ctx.tools) if _req_ctx.tools else 0
+                _th_saved = max(0, _th_tok_before - _th_tok_after)
+                if _th_saved > 0:
+                    tags["turn_hook_tools_saved_tokens"] = (
+                        int(tags.get("turn_hook_tools_saved_tokens", 0) or 0) + _th_saved
+                    )
 
             # Consistency: report tok_before/tok_after with ONE tokenizer. The pipeline
             # and the handler use different token estimators, and cache-mode branches
@@ -2486,6 +2522,13 @@ class AnthropicHandlerMixin:
                 _orig_snapshot = original_client_messages  # noqa: F821 (bound at request start)
                 original_tokens = tokenizer.count_messages(_orig_snapshot)
                 optimized_tokens = tokenizer.count_messages(optimized_messages)
+                # Fold the tool-schema/desc compaction delta into BOTH endpoints so
+                # tok_before - tok_after == tok_saved stays coherent in the PERF line
+                # (count_messages never sees tool bytes). Same shape as the OpenAI chat
+                # handler; guarded so a no-op or inflating pass contributes nothing.
+                if 0 < _tool_tokens_after < _tool_tokens_before:
+                    original_tokens += _tool_tokens_before
+                    optimized_tokens += _tool_tokens_after
                 tokens_saved = max(0, original_tokens - optimized_tokens)
                 # Attribute the fold to the hook ONLY when the hook itself reduced
                 # tokens (same-tokenizer pre vs post) — not when the recount above

headroom/proxy/outcome.py

@@ -30,6 +30,11 @@ from dataclasses import dataclass, field
 from datetime import datetime
 from typing import Any
 
+from headroom.proxy.tool_schema_savings_policy import (
+    headline_tokens_saved,
+    tool_schema_saved_from_tags,
+)
+
 logger = logging.getLogger("headroom.proxy")
 
 
@@ -410,10 +415,7 @@ async def emit_request_outcome(handler: Any, outcome: RequestOutcome) -> None:
     # Tool-schema savings (deferral + turn-hook tool shrink) live in per-request
     # tags and never move tok_before/after; aggregate them into Metrics so the
     # session summary / cost summary / all-layers total can surface the layer.
-    _otags = outcome.tags or {}
+    tool_search_saved = tool_schema_saved_from_tags(outcome.tags or {})
-    tool_search_saved = int(_otags.get("tool_search_deferred_tokens", 0) or 0) + int(
-        _otags.get("turn_hook_tools_saved_tokens", 0) or 0
-    )
 
     # 1. Prometheus / SavingsTracker.
     await handler.metrics.record_request(
@@ -498,21 +500,20 @@ async def emit_request_outcome(handler: Any, outcome: RequestOutcome) -> None:
     #    line unchanged, and gives ``headroom perf --client X``
     #    parsers a clean key to filter on.
     client_part = f" client={outcome.client}" if outcome.client else ""
-    # Tool-schema savings are tracked separately from message compression: tool
+    # Tool-schema DEFERRAL savings can't move tok_before/after (those count messages
-    # deferral (defer_loading) and turn-hook tool shrink don't move tok_before/after
+    # only), so a tool-heavy turn shows tok_saved=0 while genuinely saving thousands of
-    # (those count messages only), so a tool-heavy turn shows tok_saved=0 while
+    # tool-definition tokens. `tool_saved` carries that component and `total_saved` is
-    # genuinely saving thousands of tool-schema tokens. Surface it as its own field
+    # the sum every user-facing surface reports — see tool_schema_savings_policy for why
-    # so `headroom perf` / log readers see the whole picture.
+    # compaction is already inside tok_saved and must not be added twice.
-    _tags = outcome.tags or {}
+    tool_saved = tool_schema_saved_from_tags(outcome.tags or {})
-    tool_saved = int(_tags.get("tool_search_deferred_tokens", 0) or 0) + int(
+    total_saved = headline_tokens_saved(outcome.tokens_saved, outcome.tags or {})
-        _tags.get("turn_hook_tools_saved_tokens", 0) or 0
-    )
     logger.info(
         f"[{outcome.request_id}] PERF "
         f"model={outcome.model} msgs={outcome.num_messages} "
         f"tok_before={outcome.original_tokens} tok_after={outcome.optimized_tokens} "
         f"tok_saved={outcome.tokens_saved} "
         f"tool_saved={tool_saved} "
+        f"total_saved={total_saved} "
         f"cache_read={outcome.cache_read_tokens} cache_write={outcome.cache_write_tokens} "
         f"cache_hit_pct={outcome.cache_hit_pct} "
         f"opt_ms={outcome.overhead_ms:.0f} "

headroom/proxy/server.py

@@ -1917,11 +1917,13 @@ class HeadroomProxy(
         logger.info(f"Failed:                {m.requests_failed}")
         logger.info(f"Input tokens:          {m.tokens_input_total:,}")
         logger.info(f"Output tokens:         {m.tokens_output_total:,}")
-        logger.info(f"Tokens saved:          {m.tokens_saved_total:,}")
+        # ONE headline: message compression + tool-schema deferral. Deferral can't move
+        # tok_before/after (tool bytes never reach count_messages), so it used to print
+        # as a separate line that read like a side metric rather than savings.
+        logger.info(f"Tokens saved:          {m.tokens_saved_total + m.tool_search_saved_total:,}")
         if m.tool_search_saved_total > 0:
-            # Tool-schema deferral / turn-hook tool shrink — counted apart from
+            logger.info(f"  messages:            {m.tokens_saved_total:,}")
-            # message compression (tool bytes never move tok_before/after).
+            logger.info(f"  tool schemas:        {m.tool_search_saved_total:,}")
-            logger.info(f"Tool schemas deferred: {m.tool_search_saved_total:,}")
         # Active-compression ratio: savings as a fraction of what we
         # *attempted* to compress (extracted units + tool schema),
         # NOT the whole request. The full-request denominator is
@@ -3887,9 +3889,18 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
                 # should show — it answers "are we doing well *when we
                 # have something to compress?*" rather than diluting the
                 # win by frozen-prefix bytes we never touched.
+                # All-layers numerator, matching denominator: `attempted_input_tokens`
+                # already counts tool schemas we COMPACTED, so pairing it with a
+                # compression-only numerator undercounted every tool-heavy session.
+                # Deferred schemas are added to both sides — they were attempted work
+                # that succeeded completely.
                 "active_savings_percent": round(
-                    (proxy_compression_tokens / attempted_input_tokens * 100)
+                    (
-                    if attempted_input_tokens > 0
+                        all_layers_tokens_saved
+                        / (attempted_input_tokens + m.tool_search_saved_total)
+                        * 100
+                    )
+                    if (attempted_input_tokens + m.tool_search_saved_total) > 0
                     else 0,
                     2,
                 ),

headroom/proxy/tool_schema_savings_policy.py

@@ -1,4 +1,27 @@
-"""Tool-schema savings attribution policy for proxy stats."""
+"""Tool-schema savings attribution policy for proxy stats.
+
+Headroom saves input tokens in two accounting shapes, and the split is not a
+style choice — it follows from what each transform can observe:
+
+* **Compaction** (``*:tool_schema_compaction``, ``*:tool_desc_compaction``)
+  rewrites the tool array in place, so both endpoints are countable. Handlers
+  fold the delta into ``original_tokens``/``optimized_tokens``, which keeps
+  ``tok_before - tok_after == tok_saved`` coherent in the PERF line. It is
+  therefore ALREADY inside ``tokens_saved`` and must never be added again.
+* **Deferral / hook shrink** (tool search, turn hooks) removes schemas that
+  ``count_messages`` never saw, so it cannot move ``original_tokens``. It is
+  recorded in per-request tags and is ADDITIVE to ``tokens_saved``.
+
+The one rule a caller needs: the headline is
+``tokens_saved + tool_schema_saved_from_tags(tags)``. Use
+:func:`headline_tokens_saved` rather than open-coding it — three surfaces had
+drifted inline copies of that sum, and two harnesses were silently dropping
+their compaction savings entirely because the convention was never written down.
+
+Adding a new tool-schema-shrinking feature? If it moves the tool array, fold it
+in the handler like the compaction sites do. If it defers schemas, add its tag
+name to :data:`TOOL_SCHEMA_SAVINGS_TAGS` and every surface picks it up.
+"""
 
 from __future__ import annotations
 
@@ -13,6 +36,8 @@ def tool_schema_saved_from_tags(tags: object) -> int:
 
     The summed tags are set only on paths where Headroom performed the deferral,
     so clients that already had tool search enabled contribute zero here.
+
+    These tags are additive to ``tokens_saved`` — see the module docstring.
     """
     if not isinstance(tags, dict):
         return 0
@@ -24,3 +49,23 @@ def tool_schema_saved_from_tags(tags: object) -> int:
         except (TypeError, ValueError):
             continue
     return total
+
+
+def headline_tokens_saved(tokens_saved: object, tags: object) -> int:
+    """Return the single "Tokens saved" figure for one request.
+
+    This is the only correct total to show a user: message compression plus the
+    tool-definition tokens that never entered the context. Every reporting
+    surface (PERF line, ``headroom perf``, ``/api/stats``, dashboard, session
+    summary) must route through here so they cannot disagree.
+
+    Clamped at zero: handlers already revert any inflation before forwarding, so
+    a negative is a token-count artifact that never reached the model.
+    """
+    base = 0
+    if isinstance(tokens_saved, (int, float, str)) and not isinstance(tokens_saved, bool):
+        try:
+            base = int(tokens_saved)
+        except (TypeError, ValueError):
+            base = 0
+    return max(0, base + tool_schema_saved_from_tags(tags))

tests/test_tool_schema_savings_policy.py

@@ -2,6 +2,7 @@ from __future__ import annotations
 
 from headroom.proxy.tool_schema_savings_policy import (
     TOOL_SCHEMA_SAVINGS_TAGS,
+    headline_tokens_saved,
     tool_schema_saved_from_tags,
 )
 
@@ -41,3 +42,101 @@ def test_tool_schema_savings_tags_are_stable() -> None:
         "tool_search_deferred_tokens",
         "turn_hook_tools_saved_tokens",
     )
+
+
+# ── headline_tokens_saved: the one figure every surface reports ────────────────
+# Headroom saves tool-definition tokens in two accounting shapes — compaction
+# folds into tokens_saved, deferral is tagged and additive. Both existed before
+# but the rule was never written down, so two harnesses dropped their compaction
+# savings and three surfaces open-coded the sum. These cases pin the contract.
+
+
+def test_folded_compaction_is_not_counted_twice() -> None:
+    """Compaction is ALREADY inside tokens_saved (handlers fold both endpoints).
+
+    Adding an attribution amount back on top would inflate every tool-heavy turn.
+    """
+    assert headline_tokens_saved(420, {}) == 420
+
+
+def test_deferral_tags_are_additive_to_tokens_saved() -> None:
+    """Deferral removes schemas count_messages never saw, so it can't be folded."""
+    tags = {"tool_search_deferred_tokens": 9639}
+    assert headline_tokens_saved(0, tags) == 9639
+    assert headline_tokens_saved(1_000, tags) == 10_639
+
+
+def test_headline_identical_across_harnesses_for_equivalent_work() -> None:
+    """A 500-token saving reports as 500 whichever accounting shape produced it.
+
+    Anthropic/Claude Code folds its compaction; a Codex deferral is tagged. Same
+    real saving, same headline — that equivalence is the point of the helper.
+    """
+    anthropic_folded = headline_tokens_saved(500, {})
+    codex_tagged = headline_tokens_saved(0, {"tool_search_deferred_tokens": 500})
+    assert anthropic_folded == codex_tagged == 500
+
+
+def test_headline_survives_malformed_tags() -> None:
+    for tags in (None, {}, "not-a-dict", {"tool_search_deferred_tokens": None}):
+        assert headline_tokens_saved(10, tags) == 10
+    assert headline_tokens_saved(10, {"tool_search_deferred_tokens": "abc"}) == 10
+    assert headline_tokens_saved(None, None) == 0
+
+
+def test_headline_clamps_negative_message_savings() -> None:
+    """Handlers revert inflation before forwarding, so a negative is a count artifact."""
+    assert headline_tokens_saved(-5, {}) == 0
+    assert headline_tokens_saved(-5, {"tool_search_deferred_tokens": 100}) == 95
+
+
+def test_tool_schema_compaction_saves_real_tokens_not_just_bytes() -> None:
+    """The premise of folding compaction into tokens_saved on every handler.
+
+    Compaction strips annotation keys ($schema/title/examples). If that only moved
+    bytes that tokenize to nothing, the fold would be worthless — so pin a positive
+    TOKEN delta on a realistically-shaped tool array, and pin that folding it into
+    both endpoints keeps ``tok_before - tok_after == tok_saved`` coherent.
+    """
+    import json
+
+    from headroom.providers.anthropic import AnthropicProvider
+    from headroom.proxy.tool_schema_compaction import compact_tools
+
+    tok = AnthropicProvider().get_token_counter("claude-sonnet-4-6")
+    payload = {
+        "tools": [
+            {
+                "name": f"tool_{i}",
+                "description": "Does   a    thing.\n\n  Returns text.",
+                "input_schema": {
+                    "$schema": "https://json-schema.org/draft/2020-12/schema",
+                    "title": f"tool_{i}_schema",
+                    "examples": [{"path": "/tmp/x"}, {"path": "/tmp/y"}],
+                    "type": "object",
+                    "properties": {"path": {"type": "string", "description": "File path"}},
+                    "required": ["path"],
+                },
+            }
+            for i in range(14)
+        ]
+    }
+    before_tools = payload["tools"]
+    body, modified, _bytes_before, _bytes_after = compact_tools(payload)
+    assert modified is True
+
+    tool_before = tok.count_text(json.dumps(before_tools, default=str))
+    tool_after = tok.count_text(json.dumps(body["tools"], default=str))
+    assert tool_after < tool_before, "compaction must shrink tool TOKENS, not only bytes"
+
+    # Mirrors the fold each handler applies at its final recount, with zero message
+    # compression — the shape that used to report tok_saved=0 on Claude Code.
+    original_tokens = optimized_tokens = 5_000
+    if 0 < tool_after < tool_before:
+        original_tokens += tool_before
+        optimized_tokens += tool_after
+    tokens_saved = max(0, original_tokens - optimized_tokens)
+
+    assert tokens_saved == tool_before - tool_after
+    assert original_tokens - optimized_tokens == tokens_saved
+    assert headline_tokens_saved(tokens_saved, {}) == tokens_saved