mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-10 14:27:00 -04:00
1de35e775f
## Description `_check_tree_sitter()` in `headroom/compression/handlers/code_handler.py` only verified that `tree_sitter_language_pack` could be imported. When tree-sitter core and the language pack are built against different ABIs, the import succeeds but `parser.language = get_language(...)` raises at request time, silently falling back to the generic text compressor — with no warning, while the banner still reports code-aware as enabled. #1299 fixed the same class of bug in `transforms/code_compressor.py`. This PR is the defensive follow-up tracked by #1231: it applies the same parse probe to the compression **structure handler** so both code-aware paths are consistent. Closes #1231 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Replace the import-only probe in `code_handler._check_tree_sitter()` with a real parse probe: construct a `Parser`, assign a Python language, and parse `b"x = 1\n"` — if any step fails, mark unavailable - Log a WARNING when import succeeds but parsing fails, so the downgrade is visible - Add `TestAvailabilityProbe` covering the simulated ABI mismatch (-> False) and healthy install (-> True) cases > Note: an earlier revision of this PR also touched `transforms/code_compressor.py`, but that fix landed independently via #1299. After rebasing onto current `main`, this PR is scoped to the remaining `code_handler.py` gap only. ## Testing - [x] Unit tests pass (`pytest tests/test_compression/test_code_handler.py` -> 22 passed, 8 skipped) - [x] Linting / formatting pass (`ruff check .`, `ruff format --check .`) - [x] New tests added for new functionality ### Real Behavior Proof - Setup: Windows 11 (GBK locale), Python 3.10, tree-sitter NOT installed - Before fix: `_check_tree_sitter()` returns `True` on a partial/ABI-mismatched install (import succeeds), then silently degrades to the text compressor at request time with no warning - After fix: the probe parses a trivial snippet; an ABI mismatch is caught at probe time, `_check_tree_sitter()` returns `False`, and a WARNING is logged. `TestAvailabilityProbe::test_abi_mismatch_returns_false` reproduces this with a fake Parser whose `language` setter raises. Signed-off-by: RTCartist <wangshengb@buaa.edu.cn> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_code_handler.py | ||
| test_evals.py | ||
| test_json_handler.py | ||
| test_llm_eval.py | ||
| test_masks.py | ||
| test_universal.py | ||