From c2fbb4eed0c47973f8baaa88e45be0e51e38e2e2 Mon Sep 17 00:00:00 2001 From: gglucass Date: Wed, 26 Aug 2026 02:58:11 +0200 Subject: [PATCH] test(agno): follow the metrics dataclass move in agno 3.0.0 (#3260) ## Description agno 3.0.0 (released 2026-08-24) removed the `agno.models.metrics` module; the per-message usage dataclass now lives at `agno.metrics` under the name `MessageMetrics`. The mock fixtures in `tests/test_integrations/agno/test_model.py` import the old path inline, and the `test-agno` CI job installs `wheel[dev,agno]` with an unpinned `agno>=1.0.0`, so it now resolves agno 3.0.0 and fails on every branch - including `main` (see the CI run for #3239's merge commit) and currently-open PRs. This resolves the class once at module level: prefer the pre-3 location, fall back to `MessageMetrics` on agno >= 3. `MessageMetrics` exists under both names in 2.x and the constructor kwargs the fixtures use (`input_tokens`, `output_tokens`, `total_tokens`) are unchanged, so both major versions stay green. Tests-only change; the runtime integration (`headroom/integrations/agno/`) never imported the removed module - the other 76 agno tests already pass on 3.0.0. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Replace the two inline `from agno.models.metrics import Metrics` imports in the `mock_agno_model` fixture with one module-level compat resolution that tries `agno.models.metrics.Metrics` (agno < 3) and falls back to `agno.metrics.MessageMetrics as Metrics` (agno >= 3). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev --extra agno --with agno==3.0.0 pytest tests/test_integrations/agno/ -q ================== 79 passed, 5 skipped, 1 warning in 19.87s =================== $ uv run --frozen --extra dev --extra agno --with agno==2.9.0 pytest tests/test_integrations/agno/ -q ================== 79 passed, 5 skipped, 1 warning in 11.60s =================== $ ruff check tests/test_integrations/agno/test_model.py All checks passed! $ ruff format --check tests/test_integrations/agno/test_model.py 1 file already formatted ``` ## Real Behavior Proof - Environment: macOS 15 (arm64), CPython 3.12, uv-managed venv; branch = upstream/main `6262c28a` + this one test commit. - Exact command / steps: On pristine upstream/main, `uv run --frozen --extra dev --extra agno --with agno==3.0.0 pytest tests/test_integrations/agno/ -q` reproduces the CI failure: 3 failed (`test_response_applies_optimization`, `test_response_stream_applies_optimization`, `test_model_wrapper_real_optimization`), all `ModuleNotFoundError: No module named 'agno.models.metrics'` - the same three failures as the `test-agno` job on current PRs. Applied this commit and re-ran the same command under agno 3.0.0 and agno 2.9.0. - Observed result: 79 passed / 5 skipped under both agno versions; the three fail-before tests pass. - Not tested: agno 1.x (the extra's floor); `mypy headroom` not re-run - no runtime module is touched. ## Runtime Rollout Safety - Rollout-managed feature(s): None - tests only. - Minimum rollout channel: Stable. - Stable/default behavior changed: No. No shipped code changes. - Kill switch / disable path: Not applicable (test-only change). - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert the single commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` - it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - No new regression test: the three existing tests are the regression - they fail on agno 3.0.0 without this change and pass with it. Docs untouched (test-only fix). - An alternative was pinning `agno<3` in the extra; not taken, since the runtime integration works unmodified on 3.0.0 and a pin would block users already on agno 3. Co-authored-by: Claude Fable 5 --- tests/test_integrations/agno/test_model.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tests/test_integrations/agno/test_model.py b/tests/test_integrations/agno/test_model.py index 26545996c..5a0bf990e 100644 --- a/tests/test_integrations/agno/test_model.py +++ b/tests/test_integrations/agno/test_model.py @@ -19,6 +19,11 @@ try: AGNO_AVAILABLE = True except ImportError: AGNO_AVAILABLE = False +else: + try: # agno < 3: the per-message usage dataclass lived at agno.models.metrics + from agno.models.metrics import Metrics + except ImportError: # agno >= 3 moved it to agno.metrics, renamed MessageMetrics + from agno.metrics import MessageMetrics as Metrics from headroom import HeadroomConfig, HeadroomMode @@ -50,8 +55,6 @@ def mock_agno_model(): # Mock invoke method (returns ModelResponse for Agno's response() loop) def mock_invoke(messages, **kwargs): - from agno.models.metrics import Metrics - # Create a proper ModelResponse that Agno's response() can process return ModelResponse( role="assistant", @@ -73,8 +76,6 @@ def mock_agno_model(): # Mock invoke_stream for streaming def mock_invoke_stream(messages, **kwargs): - from agno.models.metrics import Metrics - yield ModelResponse( role="assistant", content="Streaming...",