mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
6d5516dcb8
## Description Adds PHP to `CodeAwareCompressor`, fixing #201. PHP was already *detected* as code (Magika labels in `headroom/compression/detector.py` include `php`, and the Rust `magika_detector.rs` lists it too) but there was no PHP `LangConfig`, so PHP content silently passed through uncompressed. This wires PHP through the tree-sitter compression path following the C# pattern (the most recently added, fully functional language — deliberately not the quarantined Perl path). A secondary detection bug is fixed along the way: PHP's `$variables` match Perl's prefilter regex, and the existing Perl-dominance guard in `detect_language` returned `UNKNOWN` for PHP files. An explicit `<?php` open tag — which no Perl source contains — now drops Perl from the candidate set before that guard runs. ## Type of Change - [ ] Bug fix - [x] New feature - [ ] Documentation update - [ ] Refactor - [ ] Other ## Changes Made - `headroom/transforms/code_compressor.py`: `CodeLanguage.PHP` + `phtml`/`php5`/`php7`/`php8` aliases; PHP `LangConfig` built from the actual tree-sitter-php grammar (node names verified by parsing samples): `namespace_use_declaration` imports, `function_definition`/`method_declaration` functions, `class_declaration`/`interface_declaration`/`trait_declaration` classes, `enum_declaration` types, `declaration_list` class bodies, `compound_statement` function bodies. `namespace_definition` maps to `package_node` so statement-scoped `namespace App;` hoists ahead of the `use` imports (required PHP ordering); the rare block-scoped `namespace A { }` form takes the same path and is preserved verbatim — valid output, just no compression inside the block. PHP prefilter regexes added; supported-languages error message updated; `<?php`-tag Perl disambiguation in `detect_language`. - `headroom/transforms/content_detector.py`: `php` entry in `_CODE_PATTERNS` so raw PHP classifies as `SOURCE_CODE` and reaches the code-aware route. - `tests/test_transforms/test_code_compressor.py`: new `TestPhpSupport` mirroring `TestCSharpSupport` — signatures preserved / bodies elided, `<?php` → `namespace` → `use` → declarations ordering, auto-detection despite the Perl sigil overlap, alias coercion, malformed passthrough. - `tests/test_code_compressor_language_alias.py`: `php` in the canonical list, `phtml` in the alias table. - `docs/content/docs/code-compression.mdx`: PHP added to the Tier 2 supported-languages row. No new dependency: `tree-sitter-language-pack` (the existing `[code]` extra) already ships the PHP grammar. No Rust changes needed. ## Testing - [x] New unit tests added and passing - [x] Full affected test suites pass locally **Test Output** ``` $ python -m pytest tests/test_transforms/test_code_compressor.py tests/test_code_compressor_language_alias.py -q ============================= 120 passed in 7.81s ============================= $ python -m pytest tests/test_transforms/ -q 3 failed, 443 passed # the 3 failures (kompress ONNX thread caps, kompress size gate, # text_crusher unicode parity) reproduce identically on a clean # upstream/main checkout in this environment — pre-existing local # ONNX runtime quirks, unrelated to this change $ ruff check . (0.15.17, CI-pinned) → All checks passed! | ruff format --check → clean $ mypy headroom/transforms/code_compressor.py headroom/transforms/content_detector.py --ignore-missing-imports Success: no issues found in 2 source files ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, tree-sitter + tree-sitter-language-pack (<1.0) installed, branch `feat/201-php-code-compression` off `upstream/main`. - Exact command / steps: parsed PHP samples (namespaced class w/ methods, block-scoped namespace, mixed HTML+PHP) with `tree_sitter_language_pack.get_parser('php')` to verify every node name used in the config; then ran `CodeAwareCompressor().compress(php_code, language="php")` and `compress(php_code)` (auto-detection) on a 48-line realistic service class. - Observed result: explicit and auto-detected paths both return `language=CodeLanguage.PHP`, `compression_ratio=0.64`, `syntax_valid=True`; method bodies elided to `// [N lines omitted]` while `<?php`, `namespace`, `use` lines, class header, and all signatures are preserved verbatim in the original order. Before the detection fix, auto-detection returned `UNKNOWN` (Perl prefilter dominance) — reproduced and then verified fixed. - Not tested: exotic PHP shapes (heredoc-heavy code, attributes `#[...]` on methods, interleaved multi-`<?php ?>` HTML templates beyond the basic mixed case); these fall back to verbatim preservation via the uncaptured-node pass or malformed-passthrough, both of which are covered by tests for the simple cases. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
71 lines
2.7 KiB
Python
71 lines
2.7 KiB
Python
"""Regression tests for language-hint / fence-tag coercion in code_compressor.
|
|
|
|
`CodeAwareCompressor.compress(code, language=...)` used to build the language
|
|
with `CodeLanguage(language.lower())`, which raises `ValueError` for anything
|
|
that is not an exact enum value. Common markdown fence tags and hints — `js`,
|
|
`ts`, `py` — are not enum values, so:
|
|
|
|
* direct callers (`compress(code, language="js")`) crashed, and
|
|
* inside the router the ValueError was swallowed, so ` ```js ` / ` ```ts ` /
|
|
` ```py ` fenced blocks silently skipped code-aware compression.
|
|
|
|
`coerce_language` maps aliases to the canonical language and returns UNKNOWN
|
|
(never raises) for unrecognized tags, letting the caller fall back to
|
|
content-based detection.
|
|
"""
|
|
|
|
import pytest
|
|
|
|
from headroom.transforms.code_compressor import CodeLanguage, coerce_language
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"alias,expected",
|
|
[
|
|
("js", CodeLanguage.JAVASCRIPT),
|
|
("jsx", CodeLanguage.JAVASCRIPT),
|
|
("node", CodeLanguage.JAVASCRIPT),
|
|
("ts", CodeLanguage.TYPESCRIPT),
|
|
("tsx", CodeLanguage.TYPESCRIPT),
|
|
("py", CodeLanguage.PYTHON),
|
|
("python3", CodeLanguage.PYTHON),
|
|
("golang", CodeLanguage.GO),
|
|
("rs", CodeLanguage.RUST),
|
|
("c++", CodeLanguage.CPP),
|
|
("phtml", CodeLanguage.PHP),
|
|
],
|
|
)
|
|
def test_coerce_language_maps_common_aliases(alias, expected):
|
|
assert coerce_language(alias) == expected
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"canonical",
|
|
["python", "javascript", "typescript", "go", "rust", "java", "c", "cpp", "perl", "php"],
|
|
)
|
|
def test_coerce_language_accepts_canonical_values(canonical):
|
|
assert coerce_language(canonical) == CodeLanguage(canonical)
|
|
|
|
|
|
def test_coerce_language_is_case_insensitive_and_trims():
|
|
assert coerce_language(" JS ") == CodeLanguage.JAVASCRIPT
|
|
assert coerce_language("Python") == CodeLanguage.PYTHON
|
|
|
|
|
|
@pytest.mark.parametrize("value", ["", " ", "not-a-language", "brainfuck", "yaml"])
|
|
def test_coerce_language_unknown_returns_unknown_not_valueerror(value):
|
|
# The whole point: never raise, so an unrecognized fence tag can fall back
|
|
# to content detection instead of crashing / being swallowed.
|
|
assert coerce_language(value) == CodeLanguage.UNKNOWN
|
|
|
|
|
|
def test_compress_with_alias_language_does_not_raise():
|
|
"""The direct API path must not raise on a common alias."""
|
|
from headroom.transforms.code_compressor import CodeAwareCompressor
|
|
|
|
code = "function add(a, b) {\n return a + b;\n}\n"
|
|
compressor = CodeAwareCompressor()
|
|
# Before the fix this raised ValueError: 'js' is not a valid CodeLanguage.
|
|
result = compressor.compress(code, language="js")
|
|
assert result is not None
|
|
assert result.compressed is not None
|