Commit graph

1 commit

Author SHA1 Message Date
Focused Instability
9b0c840dd7
fix(gemini): surface functionResponse payloads to waste-signal detection (#897)
## Problem

Fixes #819.

Gemini `functionResponse` parts are preserved verbatim on the wire (by
design — they are never compressed), but their payloads never reached
`parse_messages`: `_gemini_contents_to_messages` only extracts `text`
parts. Tool output — where most waste lives — contributed nothing to
waste detection on either Gemini path, so `json_bloat`, `repetition`,
and the new `reread` signal (#853/#854) were all blind to it.

## Fix (telemetry-only)

1. **`_gemini_contents_to_messages(...,
include_function_responses=True)`** — new keyword-only flag. When set,
each `functionResponse` payload is additionally emitted as a
`role="tool"` message (dict payloads JSON-serialized, strings passed
through, missing/`None` responses skipped). `preserved_indices`
semantics are unchanged: the entries are still restored verbatim on the
wire.
2. **`TransformPipeline.apply(..., waste_messages=...)`** — new optional
kwarg (popped before transforms, like `record_metrics`). When provided,
the waste-signal parse runs over this richer list instead of the
transform input. Transforms, token accounting, and savings deltas are
untouched — this is why the richer list is not simply fed to the
pipeline: compressed copies of preserved entries are discarded on
rebuild, which would corrupt savings reporting.
3. Both Gemini `generateContent` paths (native + Cloud Code Assist)
build the enriched list and pass it through.

The existing `role="tool"` parsing from #815 handles the rest:
tool_result blocks, waste flags, and reread grouping all apply.

## Tests

`tests/test_gemini_function_response_waste.py` — 11 new tests:
- conversion: default unchanged (regression), dict/string payloads,
missing response skipped, text-before-tool ordering, preserved_indices
unchanged, circular-reference fallback
- parsing: functionResponse payload produces tool_result blocks +
`json_bloat`; identical payloads far apart count as `reread`
- pipeline: `waste_messages` overrides the waste source, does not affect
transform output/token counts, falls back to transform input when absent

Full local sweep of touched suites: gemini multimodal, parser, safety
rails, canonical pipeline — green. The 13 failures in
`test_proxy_gemini_*_integration.py` are credential-dependent and
identical on clean `main`.

## Live proof

Mock Gemini upstream on a real port, proxy with `optimize=True`;
conversation with a large functionResponse payload served twice (5
messages apart) plus compressible model text:

```
waste_signals: { "json_bloat": 35003, "reread": 11673, ... }
PROOF OK: waste visible, wire verbatim
```

Upstream received both `functionResponse` entries byte-identical to the
client request.

## Known limitations / follow-ups

- The Cloud Code Assist path passes `waste_messages` but does not yet
consume `result.waste_signals` into a recorded outcome (pre-existing
gap; the native path records it).
- Requests where **all** content entries are preserved (pure
functionResponse/media conversations) early-exit before the pipeline and
still produce no waste signals.
- Codex/Responses-API counterpart is #820 (separate PR).

Co-authored-by: integration-check <integration@local>
2026-06-12 17:09:20 -05:00