mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
6bdc8c44a3
## Description
<!-- Briefly explain the change and why it is needed. -->
The metrics docs describe the `headroom_*` Prometheus metric family and
suggest example Grafana panels, but ship no importable dashboard — users
have to build one by hand. This adds a ready-to-import Grafana dashboard
built **only** on documented metric names (`headroom_requests_total`,
`headroom_tokens_saved_total`, `headroom_tokens_input_total`, and the
`headroom_overhead_ms_*` millisecond summary), and links it from the
**Grafana Dashboard** section of `docs/content/docs/metrics.mdx`.
This is a docs/examples-only addition — no source code changes.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Added `examples/grafana/headroom-dashboard.json` — a ready-to-import
Grafana dashboard (7 panels, uid `headroom-compression`) built entirely
on Headroom's documented `/metrics` names. Panels cover tokens saved,
input tokens, request rate, average processing overhead
(`headroom_overhead_ms_sum` / `headroom_overhead_ms_count` with
min/max), tokens-saved/sec, and request rate by pool. It uses **no
histograms** (the proxy emits none). The `pool`/`source` template
variables use regex matchers (`=~`) so they are optional and match
series without those labels.
- Updated `docs/content/docs/metrics.mdx` — linked the new dashboard
from the **Grafana Dashboard** section with import instructions, keeping
the existing ad-hoc PromQL query table alongside it.
## Testing
<!-- Check what you actually ran, then paste the real command output
below. -->
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
Docs/examples-only change, manually verified: the dashboard JSON is
well-formed and every PromQL query references only the documented
`headroom_*` metric names from `docs/content/docs/metrics.mdx`.
### Test Output
```text
$ python3 -c "import json; d=json.load(open('examples/grafana/headroom-dashboard.json')); print('valid JSON,', len(d['panels']), 'panels, uid', d['uid'])"
valid JSON, 7 panels, uid headroom-compression
```
PromQL queries used by the panels (all against documented `headroom_*`
metrics):
```text
sum(headroom_tokens_saved_total{pool=~"$pool", hook=~"$hook"})
sum(headroom_tokens_input_total{pool=~"$pool", hook=~"$hook"})
sum(rate(headroom_requests_total{pool=~"$pool", hook=~"$hook"}[$__rate_interval]))
sum(rate(headroom_overhead_ms_sum{pool=~"$pool", hook=~"$hook"}[$__rate_interval])) / clamp_min(sum(rate(headroom_overhead_ms_count{pool=~"$pool", hook=~"$hook"}[$__rate_interval])), 1)
sum(rate(headroom_tokens_saved_total{pool=~"$pool", hook=~"$hook"}[$__rate_interval])) by (pool)
max(headroom_overhead_ms_max{pool=~"$pool", hook=~"$hook"})
min(headroom_overhead_ms_min{pool=~"$pool", hook=~"$hook"})
sum(rate(headroom_requests_total{pool=~"$pool", hook=~"$hook"}[$__rate_interval])) by (pool)
```
## Real Behavior Proof
- Environment: local checkout of the PR branch; Python 3 for JSON
validation.
- Exact command / steps: ran the JSON-validation command above (see Test
Output) — parses cleanly, reports 7 panels and uid
`headroom-compression`; then read every panel target and confirmed each
PromQL query references only metric names documented in
`docs/content/docs/metrics.mdx` (`headroom_requests_total`,
`headroom_tokens_saved_total`, `headroom_tokens_input_total`,
`headroom_overhead_ms_{sum,count,min,max}`). No histogram metrics are
referenced.
- Observed result: JSON is valid and importable via Grafana's
**Dashboards → New → Import → Upload**; no datasource UID is hard-coded,
so the importer prompts for a Prometheus datasource. Queries match the
documented metric family.
- Not tested: a full live Grafana import against a running proxy
scraping real `/metrics` was not performed in CI. Verification was
limited to JSON validity and query/metric-name correctness against the
documented metrics.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
Additive docs/examples only — no source code, tests, or runtime behavior
changed.
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A — dashboard is imported from JSON; see the PromQL and panel list
above.
## Additional Notes
<!-- Mention any N/A checklist items, tradeoffs, follow-ups, or
maintainer context. -->
Test-related checklist items are N/A: this is an additive docs/examples
change with no application code, so `pytest`/`mypy`/`ruff` and new unit
tests do not apply. The dashboard JSON was validated and its queries
checked against the documented metric names instead.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|---|---|---|
| .. | ||
| agent-orchestration.mdx | ||
| agno.mdx | ||
| anthropic-sdk.mdx | ||
| api-reference.mdx | ||
| architecture.mdx | ||
| benchmarks.mdx | ||
| cache-optimization.mdx | ||
| ccr.mdx | ||
| ci-cd-flows.mdx | ||
| claude-code-azure-foundry.mdx | ||
| claude-code-vertex.mdx | ||
| code-compression.mdx | ||
| community-savings.mdx | ||
| configuration.mdx | ||
| context-management.mdx | ||
| docker-install.mdx | ||
| errors.mdx | ||
| failure-learning.mdx | ||
| filesystem-contract.mdx | ||
| how-compression-works.mdx | ||
| image-compression.mdx | ||
| index.mdx | ||
| installation.mdx | ||
| langchain.mdx | ||
| limitations.mdx | ||
| litellm.mdx | ||
| local-llm-prefill.mdx | ||
| mcp.mdx | ||
| memory.mdx | ||
| meta.json | ||
| metrics.mdx | ||
| openai-sdk.mdx | ||
| opencode.mdx | ||
| persistent-installs.mdx | ||
| pipeline-extensions.mdx | ||
| proxy.mdx | ||
| quickstart.mdx | ||
| releases.mdx | ||
| savings.mdx | ||
| shared-context.mdx | ||
| simulation.mdx | ||
| smart-crusher.mdx | ||
| strands.mdx | ||
| text-and-logs.mdx | ||
| troubleshooting.mdx | ||
| vercel-ai-sdk.mdx | ||