This PR migrates AIL classes (Block, Statement, Expression, etc.) to Rust. Key changes include:
- Block, Statement, and Expression are native Rust objects. Every access to their properties will create a new Python class object. Therefore, `expr.dst is expr.dst` will always evaluate to False. You'll want to do `expr.dst == expr.dst` instead.
- However, keep in mind that `expr_0 == expr_1` can be expensive because equality checks may potentially go over two expression trees and compare every single node. In this case, you may want to resort to `expr_0.idx == expr_1.idx`. There are still a few places where `idx` is reused; we plan to remove all such cases and guarantee the uniqueness of `idx` for all Statements and Expressions during a single decompilation run.
- `type(expr) is Const` no longer works. You must use `isinstance(expr, Const)` instead. `isinstance(..., ExpressionCls/StatementCls)` is also more expensive than before due to the use of custom meta classes.
- New changes to AIL requires rebuilding the Rust component to land. You can do `python setup.py build_rust --inplace --release` to rebuild the angr Rust component in-place in an editable install.
---------
Co-authored-by: Kevin Phoenix <kevin@kphoenix.us>
* Enable ruff isort rule
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Refactor AIL sybolic execution to store block indexes in state.scratch
* Verify states are ail-mode
* Add magic to addr.setter to handle ail address tuples
* Add docs
* Icicle: auto-invalidate code cache on memory mutations.
Before this change, calling mem_write/mem_unmap/mem_protect from Python
on an address whose page had been executed would raise SelfModifyingCode
(from the VM's SMC detection) or silently leave a stale lifted/JIT
block in place, so subsequent runs executed the old code.
mem_write/mem_unmap/mem_protect now call an internal invalidate_code_range
that drops any BlockGroup, JIT compilation, and cached disassembly
overlapping the written range. SMC detection is disabled at construction
since explicit invalidation replaces it for our sync writes. restore_snapshot
additionally calls Vm::reset before Vm::restore so lifted code, JIT cache,
and prev_isa_mode — none of which the snapshot covers — are discarded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* IcicleEngine: re-sync simproc breakpoints on every run.
SimProcedure.call() creates new extern hooks on demand (for instance, the
continuation targets of __libc_start_main), so the simproc set grows
during execution. IcicleEngine's full-init path seeded the emu's
breakpoint table once, but neither the continuation nor the snapshot-
restore paths re-synced new entries.
Before this, a main() return landing on a dynamically-created
after_main extern silently fell through to zero bytes in the extern
region (SIGSEGV in fauxware fully- or partly-snapshot-mode runs).
add_breakpoint is idempotent so iterating over project._sim_procedures
before each run is cheap.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* IcicleEngine: fix continuation sync for BV addresses and stale path tracer.
Two related fixes uncovered while chasing fauxware failures when snapshot
mode is forced on:
1. The SimInspect mem_write hook only recorded dirty pages when
`mem_write_address` and `mem_write_length` were Python ints. In
practice both are typically claripy BitVectors (concrete addresses
wrapped), and `state.memory.store(addr, bvv)` is commonly called
without an explicit size, so `length` arrives as None. Normalise BV
args to int via `.concrete_value`, and fall back to
`mem_write_expr.size() // 8` when length is absent. Without this,
data written by SimProcedures between icicle runs (e.g. `read`
writing into the stack password buffer) never made it into
dirty_pages, so the next continuation-sync run left the emu holding
pre-read zeros and comparisons downstream took the wrong branch.
2. The path tracer isn't cleared on continuation, so
`emu.recent_blocks` accumulates blocks across every reused run.
`__convert_icicle_state_to_angr` then re-appends the whole list each
time, duplicating prior blocks in `state.history.bbl_addrs`. Call
`emu.clear_path_tracer()` at the start of a continuation run so
recent_blocks reflects only this run's blocks.
With these two fixes plus the earlier simproc-breakpoint re-sync, all
61 icicle/emulator/fuzzer tests pass with `_snapshot_mode` forced True,
matching the snapshot-off baseline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* IcicleEngine: make snapshot mode the default; remove the opt-in.
The first call always saves a snapshot of the fresh VM; subsequent calls
either continue with the cached emu or restore the snapshot and
delta-sync. Dropped `enable_snapshot_mode()` and the `_snapshot_mode`
flag — with the earlier sync fixes on this branch, the restore-based
path passes the same tests as the prior full-init-each-call behaviour
and runs roughly 3x faster on the fauxware suite.
- Removes the `enable_snapshot_mode()` method and its callers in three
tests (now redundant) and in the Rust fuzzer executor.
- Updates the class docstring.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The ConcreteEngine abstract base added an unnecessary layer on top of
SuccessorsEngine. Fold its single implementation responsibility into
IcicleEngine's process_successors, drop the concrete.py module, and
retype Emulator against SimState / SuccessorsEngine.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* faster
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: rileyseefeldt <riley@evilcomputer2.localdomain>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* adds checkpoint for converting angr state to icicle
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix ruff errors
* sync snapshot mapping/perms and added test
* lint fix
* changed test to use angr engine
* ruff
---------
Co-authored-by: rileyseefeldt <riley@evilcomputer2.localdomain>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: rileyseefeldt <seatbeltman@evilcomputer.localdomain>
* Fix varargs ingestion via ail
* Add test cases to ensure that multi and unused varargs are fine
* remove unnecessary comments
* remove unnecessary comments
* Raise error if fewer args passed in
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* it was good the way it was
* ignore arguments passed in after the size of args_vvars
* linter
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Rectify non-boolean expressions in ail bool exprs
* Add test for 1 instead of true
* linting
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* AIL Call can be a statement even when the callee returns a value (e.g., `memset(...)` used for side effects).
In that case `ret_expr`/`fp_ret_expr` will be None and the return value is intentionally unused.
Also, some SimProcedures may provide extra return values. Only error when a required return is missing.
* Add testcase for unused return
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move import to top-level
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Vexcall requires the state as the first parameter to handlers
* Add testcase that exercises vexccall
* Remove comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add AIL symbolic execution
* Fixes
* minor callstack fixes
* tests and lint
* lint
* "typecheck"
* tinker with SimStatePlugin.copy until typechecking works
* augh remove logic for copy with extra args
* Add testcase, fix actual bug that testcase found
* typecheck
* lint
* Address review comments
* Make ail_call_state more flexible
Icicle does not set the thumb bit when considering to break for a
breakpoint. So for instance, if a breakpoint is set at 0x1001,
Icicle will never hit this, it would instead hit 0x1000. Since angr
prefers to pretend the thumb bit is set in the pc this makes sure it
is always unset before being communicated to Icicle.
* IcicleEngine: Default to thumb mode for cortex-m states
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add AFL-style edge hitmap support to Icicle engine
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add block tracing capability to Icicle engine
* Add a test case
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix lint
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add Emulator and ConcreteEngine; Adapt IcicleEngine to ConcreteEngine
* Remove redundant assertions
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move test_emulator.py to tests/sim/
* Fix missed rename
* Add HeavyConcreteState to __all__
* Fix lint
* Put it in the right place
* Move it to the top level
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add icicle engine
* Improve docs and lint
* Format
* More lint
* Improve permissions mapping
* Add function for more complex arch conversion
* Improve page handling logic
* Add support and tests for thumb mode and switching
* Use backers instead of object segments to find all pages
* Add special case for gs on x86
* Fix thumb mode issues
* Reimplement icicle engine to integrate python bindings
* cargo fmt
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix checking of invalid instructions against the dirty addresses set.
When an instruction cannot be disassembled, VEX can output an IMark with
an instruction length 0. Before this change, such instructions would
never trigger the dirty addresses set, so even after they have been
overwritten with a valid instruction, the engine would continue
emulating the stale, pre-overwrite IR. Fix this by conservatively
assuming that unknown-length instruction have the maximum instruction
length when checking whether they overlap a dirty addresses.
* fix typos
This mostly fixes comments, but in a handful of places fixes bugs due to typos.
One example:
```
- insn_op_idx=None if cmsg.operand_idx == -1 else cmsg.opearnd_idx,
+ insn_op_idx=None if cmsg.operand_idx == -1 else cmsg.operand_idx,
```
* more typo fixes
* address lint issues
* more import fixes?
* address feedback from PR
---------
Co-authored-by: Brian Caswell <bcaswell@microsoft.com>
* Use Base.identical instead of BackendVSA.convert()
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Another one
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
