Commit graph

1 commit

Author SHA1 Message Date
Focused Instability
5939004185
feat(evals): adversarial-input robustness grid for compressors (#918)
## Description

Closes #916. CompressionAttack (arXiv:2510.22963) showed that prompt
compressors are an attack surface for LLM middleware: adversarial text
in compressible content can preferentially survive compression
(amplifying injection density) or abuse compressor control surfaces.
Headroom has a concrete instance of the latter — content carrying a CCR
retrieval marker is pinned as already-compressed, so a spoofed marker
string in tool output could make content compression-immune.

This adds an offline, deterministic eval grid measuring both, with no
LLM, no API key, and no model download (Kompress disabled by default).

Closes #916.

## Type of Change

- [x] New feature (non-breaking change which adds functionality)

## Changes Made

- `headroom/evals/adversarial_grid.py`: payload corpus (instruction
override, fake system tag, fake tool directive, CCR marker spoof in
block + inline forms, steering imperative, benign control), realistic +
synthetic carriers (60-record JSON array, 150-line worker log), and a
payload-class × carrier × splice-position grid.
- Per-cell metrics: payload survival (normalization-tolerant
containment), benign-line survival baseline, and compression suppression
(payload-ratio minus clean-ratio — the marker-spoof immunity signal),
plus per-class aggregates.
- `headroom/cli/evals.py`: wire the grid into the evals CLI command.
- Tests in `tests/test_adversarial_grid.py`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] New tests added for new functionality
- [x] New and existing unit tests pass locally with my changes

### Test Output

```text
$ pytest tests/test_adversarial_grid.py -q
12 passed in 1.10s
```

## Real Behavior Proof

- Environment: local macOS, repo .venv, Python 3.11.9; offline (no API
key, Kompress disabled)
- Exact command / steps: rebased onto current main (dropping the
now-superseded codecov-upload commit — main already uploads per-shard
coverage via codecov-action@v5), then `pytest
tests/test_adversarial_grid.py -q`
- Observed result: 12/12 pass; grid runs deterministically with no
network/model access and reports survival + suppression metrics per
cell.
- Not tested: LLM-in-the-loop attack realism — out of scope by design;
this grid is the offline deterministic layer.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Additional Notes

Force-pushed after a rebase onto current main to resolve a
`.github/workflows/ci.yml` conflict introduced by #921: the standalone
codecov-upload commit was dropped because main now performs per-shard
coverage upload globally. PR payload is unchanged (adversarial grid +
tests).

---------

Co-authored-by: integration-check <integration@local>
2026-06-13 10:47:54 -05:00