## Description
`JSONStructureHandler._should_preserve_token` compared `len(token.text)`
— which includes both quote characters — against
`short_value_threshold`. A value of exactly threshold length was
rejected: the documented "20-char threshold" was effectively 18 chars of
payload. Stacked on #887.
Closes # <!-- compression-handler review -->
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/compression/handlers/json_handler.py`: strip quotes once at
the top of the string-value branch and use the payload length for both
the short-value and entropy checks.
- `tests/test_compression/test_json_handler.py`: regression test for a
value of exactly `short_value_threshold` length.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_compression/test_json_handler.py -q
33 passed
```
## Real Behavior Proof
- Environment: local macOS, Python 3.11, branch
`fix/json-quote-threshold` (stacked on #887).
- Exact command / steps: `pytest
tests/test_compression/test_json_handler.py -q`.
- Observed result: A 20-char value is preserved at a 20-char threshold;
previously it was dropped due to the +2 quote miscount.
- Not tested: End-to-end through the live proxy pipeline.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A — library change. See Test Output.
## Additional Notes
Stacked on #887 — review the top commit until that merges. PR 2 of 7.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
## Description
Two bugs in `JSONStructureHandler` that jointly defeated the "keep first
N array items fully" design. (1) Every comma under `array_depth > 0` was
counted as an array item separator — including commas *between keys
inside objects* — so for arrays of objects the first record's own keys
exhausted `max_array_items_full` and dropped values belonging to item 0.
(2) Fixing that unmasked a second bug: self-normalized Shannon entropy
scores English prose at 0.90+, above the 0.85 "identifier" threshold, so
every long description was preserved as a fake high-entropy identifier.
Closes # <!-- found during a compression-handler review -->
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/compression/handlers/json_handler.py`: replace depth-keyed
comma counting with a container stack so only commas whose immediate
enclosing container is an array advance that array's item index.
- `headroom/compression/handlers/json_handler.py`: gate the entropy
preservation check on a no-spaces identifier signal, so UUIDs/hashes
still pass but prose compresses.
- `tests/test_compression/test_json_handler.py`: regression tests for
object-comma counting, items past the threshold, and prose-vs-identifier
entropy.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_compression/test_json_handler.py -q
32 passed
```
## Real Behavior Proof
- Environment: local macOS, Python 3.11, branch
`fix/json-array-item-count`.
- Exact command / steps: `pytest
tests/test_compression/test_json_handler.py -q` plus an empirical mask
dump on `[{"a":1,"b":2,...}]`.
- Observed result: Values inside array item 0 are now preserved; long
prose values compress while UUIDs are retained (prose scored
0.906-0.929, UUID 0.956 — the threshold alone could not separate them).
- Not tested: End-to-end through the live proxy pipeline (the handler is
not yet wired into the proxy hot path).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A — library/compression change with no UI. See Test Output.
## Additional Notes
First of a 7-PR compression-handler review series.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
