1 commit

Author	SHA1	Message	Date
Tejas Chopra	57bf720d5c	feat(router): route embedded & nested JSON through the compressor dispatch (#2623) ... ## Description `ContentRouter` only compressed JSON when the **whole** `tool_result` block was a single JSON value. JSON embedded inside larger output (`gh api` dumps, MCP tool results, `curl | jq` tails, log lines ending in a JSON blob) was invisible to the JSON compressors — and in practice that embedded shape is the large majority of JSON an agent actually sees. This adds a structural routing step: find balanced JSON spans at **any offset** in a block and route each one through the router's **existing, unchanged** `_apply_strategy_to_content`, splicing the result back with the surrounding bytes kept exact. Because each span takes the same dispatch path a whole-block JSON already takes, SmartCrusher/CodeCompressor register their `<<ccr:…>>` retrieval markers exactly as before — CCR is hash-keyed, so it is location-agnostic and unaffected by nesting. Closes # ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - New `headroom/transforms/recursive_json.py` — `route_embedded_json()`: deterministic balanced-span scan + splice; skips spans already carrying a `<<ccr:` marker (never re-compresses); token-gated. - One guarded call at the top of `_apply_strategy_to_content` plus an `_allow_embedded` **one-shot re-entrancy guard** (not a depth cap). - **No size/min or depth thresholds** — the only gates are correctness (round-trip) and benefit (token reduction). Strict no-op when a block has no embedded JSON, so the 97%+ of non-JSON blocks are byte-identical to today. - Deterministic + per-block → prefix-cache- and CCR-store-stable. - `tests/test_recursive_json.py`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_recursive_json.py tests/test_content_router_compact_json.py tests/test_content_router_tool_role_reversibility.py -q 19 passed in 5.90s $ ruff check headroom/transforms/recursive_json.py headroom/transforms/content_router.py tests/test_recursive_json.py All checks passed! $ mypy headroom/transforms/recursive_json.py headroom/transforms/content_router.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - **Environment:** local, `ContentRouter(ContentRouterConfig(lossless=False))`, pure-Python content detector. - **Exact steps:** `router.compress(block)` where `block` = prose with a 120-row JSON array embedded mid-text (`"I queried the ECS API ...\n[{...}]\nAll services healthy."`). - **Observed result:** `strategy_used=MIXED`; block **11,986 → 3,798 chars**; leading/trailing prose preserved byte-exact; the embedded JSON folded to a columnar table. Previously this block's embedded array was not routed to the JSON compressor at all. - **CCR:** a span already containing `<<ccr:` is passed through untouched (unit-tested); folded spans go through the unchanged dispatch, so markers register and resolve identically to whole-block JSON. - **Not tested:** live proxy end-to-end with markers force-enabled (covered by the unchanged dispatch path + unit tests); non-CC transcript shapes beyond the local corpus. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Scoped to the OSS structural-routing step only. The lossless *fold kinds* it routes into (JSON columnar / log template) are maintained in the `headroom-lossless-guard` extension. N/A: no docs/screenshots; `Closes #` left blank (no tracking issue). 🤖 Generated with [Claude Code](https://claude.com/claude-code)	2026-07-27 20:52:18 -07:00