mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
perf(subscription): skip transcripts older than the window in compute_window_tokens (#2861)
## Problem
`compute_window_tokens()` walks **every** `.jsonl` under
`~/.claude/projects` and runs
`json.loads()` on **every line**, only to discard the entries that fall
outside
`[start_ts, end_ts)`. `subscription/tracker._poll_loop` calls it every
**300 s**, so the
cost is paid continuously and grows with the user's history.
On one long-running install this meant **1,973 files / 1.1 GB / 261,003
lines re-parsed
every 5 minutes** — about 316 GB of JSON parsing per day.
The user-visible symptom is worse than the CPU bill: the poll pins **100
% CPU with zero
open connections** for ~12 s. That is exactly the signature external
watchdogs use to
detect a runaway loop, so the proxy kept being **restarted while it was
doing scheduled
work** (13 restarts / 13.5 CPU-hours on that host before we traced it
with `py-spy`).
Stack captured during one of those episodes:
```
raw_decode (json/decoder.py:356)
decode (json/decoder.py:337)
loads (json/__init__.py:346)
compute_window_tokens (headroom/subscription/session_tracking.py:127)
_compute_window_tokens_for_snapshot (headroom/subscription/tracker.py:872)
_maybe_poll (headroom/subscription/tracker.py:731)
_poll_loop (headroom/subscription/tracker.py:693)
```
## Fix
Transcripts are append-only and chronological, so a file whose `mtime`
predates the window
start cannot contain an entry inside the window. One guard before
opening the file:
```python
try:
if path.stat().st_mtime < start_ts:
continue
except OSError:
continue
```
## Measurement
Same install, same 5 h window, before vs after:
| | files read | lines parsed | time | result |
|---|---|---|---|---|
| before | 1,973 | 261,003 | **12.1 s** | `weighted_token_equivalent =
741388.0` |
| after | 14 (1,959 skipped) | 4,246 | **0.39 s** |
`weighted_token_equivalent = 741388.0` |
**Identical result, 31× faster.** In production the process CPU peak
over a full poll cycle
dropped from 100 % to 10 %.
## Notes
- Behaviour is unchanged: the guard only skips files that provably
cannot contribute.
- A further optimisation (not included here, to keep the change minimal)
is to read active
transcripts backwards and stop at the first entry older than `start_ts`.
The `mtime`
guard already removes ~99 % of the cost.
- Reproduced on 0.25.0, 0.27.0 and confirmed present in current `main`.
Co-authored-by: romulomorgan <oi@ialucas.com>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
parent
7f6950be34
commit
91d6bf33cd
1 changed files with 10 additions and 0 deletions
|
|
@ -136,6 +136,16 @@ def compute_window_tokens(start_ts: float, end_ts: float) -> WindowTokens:
|
|||
unattributed = WindowTokens()
|
||||
|
||||
for path in find_transcript_files():
|
||||
# Skip transcripts that cannot contain entries inside the window.
|
||||
# Transcripts are append-only and chronological, so a file whose mtime is
|
||||
# older than the window start has no entry within [start_ts, end_ts).
|
||||
# Without this guard every poll json.loads()es every line of every
|
||||
# transcript under ~/.claude/projects.
|
||||
try:
|
||||
if path.stat().st_mtime < start_ts:
|
||||
continue
|
||||
except OSError:
|
||||
continue
|
||||
for line in _read_transcript_lines(path):
|
||||
try:
|
||||
entry: dict[str, Any] = json.loads(line)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue