mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
b5aa8a358e
## Description Extracts local compression strategy accounting out of `CompressionFeedback` into a pure cache-domain object. This keeps strategy counters, retrieval-rate math, pruning, and best-strategy selection independently testable while preserving the existing `LocalToolPattern` public API. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Added `CompressionStrategyOutcomes` as the strategy-outcome domain for compression/retrieval counters, pruning, retrieval rates, and recommendation selection. - Updated `LocalToolPattern` and `CompressionFeedback` to delegate strategy accounting to that domain while keeping existing fields and methods intact. - Added direct unit coverage for strategy outcome math and bounded pruning behavior. - Updated the LiteLLM callback hook signature to remain compatible with current LiteLLM typing and the existing three-argument call shape. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text python -m ruff check . All checks passed! python -m ruff format --check . 1095 files already formatted python -m mypy headroom --ignore-missing-imports headroom\proxy\server.py:1457: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] headroom\proxy\server.py:1468: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] Success: no issues found in 409 source files python -m pytest tests/test_compression_strategy_outcomes.py tests/test_ccr_feedback.py tests/test_toin_fixes.py tests/test_litellm_callback.py tests/test_compress_api.py::TestLiteLLMCallback -q collected 54 items 46 passed, 8 skipped in 6.58s ``` ## Real Behavior Proof - Environment: Windows, Python 3.13.13, clean worktree `C:\git\headroom-pr-slice5` - Exact command / steps: ran the lint, format, type-check, and focused pytest commands listed above. - Observed result: strategy outcome tests and existing feedback/TOIN/LiteLLM compatibility tests pass; repo-wide lint/type validation passes. - Not tested: full pytest suite and Docker/native CI jobs are left to GitHub Actions. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes - Documentation and changelog are N/A for this internal refactor. - Manual UI testing is N/A; this is cache feedback and integration callback logic. - Comment checklist is unchecked because the extracted object is intentionally straightforward and covered by tests.
45 lines
1.5 KiB
Python
45 lines
1.5 KiB
Python
from headroom.cache.compression_strategy_outcomes import CompressionStrategyOutcomes
|
|
|
|
|
|
def test_retrieval_rate_is_zero_without_strategy_compressions():
|
|
outcomes = CompressionStrategyOutcomes(retrievals={"sample": 2})
|
|
|
|
assert outcomes.retrieval_rate("sample") == 0.0
|
|
|
|
|
|
def test_best_strategy_requires_minimum_samples():
|
|
outcomes = CompressionStrategyOutcomes(
|
|
compressions={"under_sampled": 2, "sampled": 3},
|
|
retrievals={"under_sampled": 0, "sampled": 1},
|
|
)
|
|
|
|
assert outcomes.best_strategy() == "sampled"
|
|
|
|
|
|
def test_best_strategy_uses_lowest_retrieval_rate():
|
|
outcomes = CompressionStrategyOutcomes(
|
|
compressions={"top_n": 10, "smart_sample": 10},
|
|
retrievals={"top_n": 7, "smart_sample": 2},
|
|
)
|
|
|
|
assert outcomes.retrieval_rate("smart_sample") == 0.2
|
|
assert outcomes.best_strategy() == "smart_sample"
|
|
|
|
|
|
def test_recording_prunes_strategy_counters_to_bounded_high_signal_set():
|
|
outcomes = CompressionStrategyOutcomes(max_strategies=10, top_strategies_per_counter=8)
|
|
|
|
for index in range(30):
|
|
strategy = f"strategy_{index:02d}"
|
|
for _ in range(index + 1):
|
|
outcomes.record_compression(strategy)
|
|
|
|
for index in range(30):
|
|
strategy = f"strategy_{index:02d}"
|
|
for _ in range(30 - index):
|
|
outcomes.record_retrieval(strategy)
|
|
|
|
assert len(outcomes.compressions) <= 10
|
|
assert len(outcomes.retrievals) <= 10
|
|
assert "strategy_29" in outcomes.compressions
|
|
assert "strategy_00" in outcomes.retrievals
|