mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
fix(tests): repair three main-branch test failures (#2306)
## Description `main` CI is red on three independent test failures. All three are **test-side** bugs (stale cache, semantic merge conflict, stale mock) — no product code regressed. Each test passed in isolation but failed on `main`, and each also blocks the `chore: release main` PR (#1923). Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - **`test_l2_appends_transform_label`** — `tool_desc_max_chars()` memoises into a module global. An earlier test in shard 1 reads it with the env unset, pinning the cache to `0`, so this test's `setenv("HEADROOM_TOOL_DESC_MAX_CHARS=20")` was swallowed (`assert 0 == 20`). Reset the cache before reading and after, mirroring the sibling `test_l2_skips_label_when_disabled`. - **`test_dashboard_uses_cached_stats_and_lazy_history_feed_polling`** — semantic merge conflict: #2198 (persist lifetime metrics) intentionally retired the session-card `Filtered (lifetime)` row and moved CLI-filtering lifetime into the history tab as `Lifetime Saved`, while the assertion from #1433 still checked the old string. Assert the current `Lifetime Saved` label. - **`test_smart_crusher_log_fallback_runs_for_valid_json`** — stale mock: #1857 made token counting whitespace-aware, so the router now rates the JSON above the naive `len(content.split())==8` the no-op kompress mock reported, making it look like a saving and short-circuiting before the Log fallback. Mock now reports `_estimate_tokens(content)` to match the router. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) ### Test Output ```text $ pytest tests/test_anthropic_compaction_transforms.py \ tests/test_proxy_dashboard_stats_cache.py \ tests/test_transforms_content_router.py -q 78 passed, 1 skipped in 12.14s $ ruff check <the three files> All checks passed! $ ruff format --check <the three files> 3 files already formatted ``` ## Real Behavior Proof - Environment: local `.venv`, Python 3.12.6, pytest 9.0.2 (same three tests that fail on the `main` CI shards 1/3/4). - Exact command / steps: ran the three previously-failing tests by node id — all pass. Reproduced the shard-isolation failure for #1 by calling `tool_desc_max_chars()` with the env unset (cache → 0) before the test, confirmed the reset makes it pass. - Observed result: 3/3 target tests pass; 78 passed / 1 skipped across the three full files. - Not tested: full suite (unchanged product code); CI shards will re-run on this PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes `mypy headroom` (the CI-enforced scope) is unaffected — these edits touch only `tests/`, which CI does not type-check. Once this lands on `main`, the `chore: release main` PR (#1923) drops to just the `test_root_server_json_matches_builder` failure, which is the release version-bump `server.json` regen (not a code bug).
This commit is contained in:
parent
718c8dc559
commit
1d79e70f95
3 changed files with 16 additions and 4 deletions
|
|
@ -114,13 +114,18 @@ class TestAnthropicToolDescCompactionTransforms:
|
|||
``anthropic:tool_desc_compaction`` must appear in ``transforms_applied``."""
|
||||
|
||||
def test_l2_appends_transform_label(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
import headroom.proxy.tool_schema_compaction as _mod
|
||||
from headroom.proxy.tool_schema_compaction import (
|
||||
compact_tool_descriptions,
|
||||
tool_desc_max_chars,
|
||||
)
|
||||
|
||||
# Opt-in with a very short max so truncation triggers.
|
||||
# Opt-in with a very short max so truncation triggers. Reset the
|
||||
# per-process cache first: an earlier test in the shard may have read
|
||||
# the (unset) env and pinned max_chars to 0, which would swallow our
|
||||
# setenv below.
|
||||
monkeypatch.setenv("HEADROOM_TOOL_DESC_MAX_CHARS", "20")
|
||||
_mod._TOOL_DESC_MAX_CHARS = None
|
||||
|
||||
payload = _make_anthropic_payload_with_tools()
|
||||
max_chars = tool_desc_max_chars()
|
||||
|
|
@ -129,6 +134,8 @@ class TestAnthropicToolDescCompactionTransforms:
|
|||
body, modified, before, after = compact_tool_descriptions(payload, max_chars)
|
||||
assert modified is True
|
||||
assert before > after
|
||||
# Don't leak the cached 20 into later tests in this shard.
|
||||
_mod._TOOL_DESC_MAX_CHARS = None
|
||||
|
||||
def test_l2_skips_label_when_disabled(self) -> None:
|
||||
import headroom.proxy.tool_schema_compaction as _mod
|
||||
|
|
|
|||
|
|
@ -740,7 +740,9 @@ def test_dashboard_uses_cached_stats_and_lazy_history_feed_polling() -> None:
|
|||
assert "Lean-ctx" in html
|
||||
assert "Context Tool" in html
|
||||
assert "cliFilteringLabel + ' Filtered (this session)'" in html
|
||||
assert "cliFilteringLabel + ' Filtered (lifetime)'" in html
|
||||
# Lifetime CLI-filtering savings moved from the session card to the
|
||||
# history tab as "Lifetime Saved" in #2198 (persist lifetime metrics).
|
||||
assert "cliFilteringLabel) + ' Lifetime Saved'" in html
|
||||
|
||||
|
||||
def test_dashboard_session_metrics_do_not_repeat_proxy_tokens_without_new_context() -> None:
|
||||
|
|
|
|||
|
|
@ -693,11 +693,14 @@ def test_smart_crusher_log_fallback_runs_for_valid_json(
|
|||
|
||||
monkeypatch.setattr(router, "_get_smart_crusher", lambda: NoopSmartCrusher())
|
||||
monkeypatch.setattr(router, "_get_log_compressor", lambda: ShrinkingLogCompressor())
|
||||
# Kompress no-op → Log fallback fires.
|
||||
# Kompress no-op → Log fallback fires. A faithful no-op reports the same
|
||||
# token count the router computed for the (unchanged) content — using
|
||||
# _estimate_tokens, not a naive word split, so it isn't mistaken for a
|
||||
# saving once #1857's whitespace-aware counting rates the JSON above 8.
|
||||
monkeypatch.setattr(
|
||||
router,
|
||||
"_try_ml_compressor",
|
||||
lambda content, context, question=None: (content, len(content.split())),
|
||||
lambda content, context, question=None: (content, _estimate_tokens(content)),
|
||||
)
|
||||
|
||||
compressed, _compressed_tokens, strategy_chain = router._apply_strategy_to_content(
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue