Compare commits

..

291 commits

Author SHA1 Message Date
dependabot[bot]
a8a5cabdfa
rust: bump pyo3 from 0.29.0 to 0.29.2 (#6800)
Some checks failed
CI / Rust Check (push) Failing after 2s
CI / rust_test (ubuntu-latest) (push) Failing after 2s
Test with coverage / Test-6 (push) Has been skipped
Test with coverage / Test-7 (push) Has been skipped
Test with coverage / Build (push) Failing after 2s
Test with coverage / Test (push) Has been skipped
Test with coverage / Test-1 (push) Has been skipped
Test with coverage / Test-2 (push) Has been skipped
Test with coverage / Test-3 (push) Has been skipped
Test with coverage / Test-4 (push) Has been skipped
Test with coverage / Test-5 (push) Has been skipped
Test with coverage / Test-8 (push) Has been skipped
Test with coverage / Test-9 (push) Has been skipped
Test with coverage / Report (push) Has been skipped
Test with coverage / Test Rust packages (push) Failing after 2s
CI / ci (push) Has been cancelled
CI / Test installation (macos-26, py3.12) (push) Has been cancelled
CI / Test installation (ubuntu-24.04, py3.14) (push) Has been cancelled
CI / Test installation (windows-2025, py3.12) (push) Has been cancelled
CI / rust_test (macos-latest) (push) Has been cancelled
CI / rust_test (windows-latest) (push) Has been cancelled
Bumps [pyo3](https://github.com/pyo3/pyo3) from 0.29.0 to 0.29.2.
- [Release notes](https://github.com/pyo3/pyo3/releases)
- [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pyo3/pyo3/compare/v0.29.0...v0.29.2)

---
updated-dependencies:
- dependency-name: pyo3
  dependency-version: 0.29.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-17 06:22:01 -07:00
dependabot[bot]
2165b8e2a5
ci: bump Swatinem/rust-cache from 2.9.1 to 2.9.2 (#6801)
Bumps [Swatinem/rust-cache](https://github.com/swatinem/rust-cache) from 2.9.1 to 2.9.2.
- [Release notes](https://github.com/swatinem/rust-cache/releases)
- [Changelog](https://github.com/Swatinem/rust-cache/blob/master/CHANGELOG.md)
- [Commits](c19371144d...6323deb102)

---
updated-dependencies:
- dependency-name: Swatinem/rust-cache
  dependency-version: 2.9.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-17 05:56:49 -07:00
dependabot[bot]
09e59e4204
ci: bump taiki-e/install-action from 2.85.5 to 2.85.10 (#6799)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.85.5 to 2.85.10.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](6a1bd70eaa...6c6fd71fe4)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.85.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-17 05:56:43 -07:00
Yan Shoshitaishvili
55530509ce
SimLinux: Stop pre-growing the stack past address zero. (#6806)
SimLinux.state_blank pre-grows the stack by a fixed 0x20 pages without
checking that 0x20 pages exist beneath the stack pointer. When they do
not, the allocation loop wraps past address 0 and hands out the remainder
at the top of the address space.

On x86-64 that is silent: blank_state(stack_end=0x10000) maps sixteen
stack pages from 0x0 up and sixteen more from 0xfffffffffffff000 down.
Where the wrap reaches a page the same call already handed out, the state
fails with SimSegfaultException("stack collided with heap") instead, and
where it reaches the loaded image it replaces it with blank pages.

Skip the pre-grow when that much space does not exist. Clamping it to the
space that does exist is not an option: the pre-allocated pages are not
backed by the loader, so a stack that reaches down to an image beneath it
hides that image. The pages that are skipped are still faulted in on
demand.

Also reject an allocation that does not fit beneath the top of the stack
in allocate_stack_pages() itself, so a caller that asks for one gets an
error rather than pages at the top of the address space.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 05:55:58 -07:00
Fish
503b1be066
SLiveness: Drop the redundant successors/predecessors caches. (#6843) 2026-08-14 03:13:58 -04:00
Yan Shoshitaishvili
2c95eb92e6
CFG: Warn when the regions to analyze cover no bytes. (#6825) 2026-08-13 17:26:22 -05:00
Fish
b80f4cf2af
Outliner: Rebuild phi statements; update output phis; return only region-defined vvars. (#6831)
* Rebuild phi statements instead of mutating them in place

* Update the phis of every dispatcher target

* Return only the variables the outlined region defines
2026-08-13 13:51:21 -07:00
Fish
41a7db2f17
SLiveness: propagate to predecessors instead of re-walking the graph (#6830) 2026-08-13 12:38:23 -07:00
Yan Shoshitaishvili
2bfaa7e8f7
CFGModel: Stop aborting when tidying a data reference at an unmapped address (#6811)
Fix #6770.
2026-08-11 13:27:06 -07:00
Fish
0c293dc0de
Phoenix: Fix incorrect virtualization of orphaned edges. (#6790) 2026-08-09 02:00:48 -07:00
Fish
ede5faf162
Decompiler: Add edits layer; add more tools to MCP. (#6789) 2026-08-09 01:48:07 -07:00
Fish
bdc1321845
RustCodeGen: Add more handlers. (#6788) 2026-08-08 09:28:22 -07:00
Fish
666fadabc5
Dephication: Keep the remapped assignment destination with source is not a vvar. (#6787) 2026-08-07 23:50:03 -07:00
Fish
fdd66945ea
CFGTransformationMixin: Rebuild conditional jumps when replacing branch targets. (#6786) 2026-08-07 23:46:34 -07:00
Fish
b0e3541dfd
SwitchClusterSimplifier: Do not merge different default nodes. (#6782) 2026-08-07 19:13:10 -07:00
Fish
780a80de79
Phoenix: Correct Loop.continue_addr. (#6785) 2026-08-07 18:41:56 -07:00
Fish
c751678d55
Decompiler: Handle compare-and-swap statements that CASIntrinsics leaves behind. (#6783) 2026-08-07 18:13:32 -07:00
Fish
08c7e48d5b
CFGFast: Mark bad blocks found by complete scanning as nodecode. (#6781) 2026-08-07 14:16:21 -07:00
Fish
c844a13eac
Clinic: Skip unnecessary simplification runs and recomputations. (#6780)
* Clinic: Skip the second pre-SSA-level1 _simplify_function when it cannot help.

* Clinic: Merge the two identical post-SSA-level1 _simplify_function calls.

* AILSimplifier, SLiveness: Skip recomputation that cannot change anything.
2026-08-07 12:06:49 -07:00
Fish
795eadfe84
Decompiler: Deterministically pick variables from sets. (#6778)
* Decompiler: Deterministically pick variables from sets.

* Add missing file.
2026-08-06 20:12:43 -05:00
Fish
0f4be85db4
Make CallSiteMaker a normal class instead of an Analysis. (#6776) 2026-08-06 08:44:29 -04:00
Fish
04b40a1eec
InlinedStringTransformationSimplifier: Pre-filter loops before symbolic execution. (#6775) 2026-08-05 20:00:09 -07:00
Fish
4018e26076
ExpressionNarrower: Compare rebuilt operands by value instead of identity. (#6774) 2026-08-05 22:34:33 -04:00
Fish
e76c4118a7
AILSimplifier: Only store replaced blocks into self.blocks. (#6773)
_rebuild_func_graph marks all blocks in self.blocks dirty, so adding unchanged blocks to self.blocks led to redundant
simplification runs after.
2026-08-05 22:34:13 -04:00
Fish
7d3c7c82ce
AILSimplifier: De-quadratify _unify_local_variables. (#6772) 2026-08-05 22:33:55 -04:00
Fish
b948687876
AIL: Fix identity comparisons against re-read statement attributes. (#6771) 2026-08-05 19:42:30 -05:00
Md7
1c99579f69
Fix #6744: avoid empty Or expression in memory.find (#6750) 2026-08-05 04:01:10 -07:00
angr-bot
f62f45363c Update version to 9.3.3.dev0 [ci skip] 2026-08-05 09:02:54 +00:00
Fish
3333f39fff
CFGFast: Make the smart scan nodecode ratio O(log n) (#6767) 2026-08-05 01:42:45 -07:00
Fish
7c2e3501e6
SimStruct: Remove _arch_memo to fix the cache leak. (#6757)
* SimStruct: Remove _arch_memo to fix the cache leak.

* Fix the comment

* Better typing.

* Fix RustSimTypes.

* More fixes.

* Fix caching for anonymous structs.

* SimUnion: Cache alignment.

* Convert returnty to a arch-ed returnty.

* RustSimType: Do not shadow SimType.with_arch.

* Rust: Bind an arch to types before they reach the type solver.

* RustSimType: Make it a subclass of SimType.
2026-08-05 01:12:55 -07:00
Yan Shoshitaishvili
f160178837
reaching_definitions: reject mismatched conversion widths (#6749)
* reaching_definitions: reject mismatched conversion widths

* Tests: satisfy RDA lint and type checks
2026-08-05 00:26:01 -07:00
Fish
71bf42c9b8
Decompiler: Test common C conditions. (#6762)
* Decompiler: Test common C conditions.

* Improve c-style null compatison implementation

* Make order-sensitive

---------

Co-authored-by: Kevin Phoenix <kevin@kphoenix.us>
2026-08-04 18:27:09 -07:00
Kevin Phoenix
fd235fcb90
icicle: Disable inspect and actions during memory sync-back (#6761) 2026-08-04 10:51:07 -07:00
Kevin Phoenix
f4b23d4444
UltraPage: Write to concrete_data as a single block instead of loop (#6760) 2026-08-04 10:11:26 -07:00
Kevin Phoenix
4a9c1454dc
icicle: Remove double-underscore methods (#6759) 2026-08-04 09:58:57 -07:00
Kevin Phoenix
233724f38e
icicle: Avoid repeatedly calling memory.permissions (#6758) 2026-08-04 09:54:45 -07:00
Fish
551df09fc5
Decompiler: Fix duplicate struct defs for pre-defined structs. (#6756) 2026-08-03 22:12:47 -07:00
Kevin Phoenix
f71b07cebf
CodeGen: Skip parenthesis on binops when representing as an unop (#6755) 2026-08-03 21:11:36 -07:00
pre-commit-ci[bot]
f9c789ff6a
[pre-commit.ci] pre-commit autoupdate (#6754)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.16.0 → v0.16.1](https://github.com/astral-sh/ruff-pre-commit/compare/v0.16.0...v0.16.1)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-08-03 11:15:42 -07:00
dependabot[bot]
15757fb2da
ci: bump taiki-e/install-action from 2.85.2 to 2.85.5 (#6753)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.85.2 to 2.85.5.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](41049aa566...6a1bd70eaa)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.85.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-03 09:37:54 -07:00
Yibo Liu
0f8e082f9a
Clinic: recover variable-length arrays (VLAs) (#6634) 2026-08-01 13:41:12 -07:00
Kevin Phoenix
49434bc3fc
Render truncations to non-C widths as masks instead of casts (#6741) 2026-07-31 17:05:47 -07:00
Kevin Phoenix
2f891d1d69
Update capstone to 5.0.9 (#6740) 2026-07-31 13:47:32 -07:00
Quintin Kong
d46e56f891
Fix signed division and remainder in the pcode engine (#6739)
* Fix signed division and remainder in the pcode engine

OpBehaviorIntSdiv and OpBehaviorIntSrem used Claripy's `/` and `%`, which are
unsigned bit-vector operations. For negative operands they therefore produced
the same results as the unsigned INT_DIV and INT_REM behaviors.

INT_SDIV now uses claripy.SDiv (truncation toward zero). INT_SREM is defined as
in1 - SDiv(in1, in2) * in2, giving a remainder with the dividend's sign, which
matches the p-code semantics documented in the class comments.

For 64-bit -5 and 2, INT_SDIV now yields -2 (0xfffffffffffffffe) and INT_SREM
yields -1 (0xffffffffffffffff) instead of large unsigned values.

The arithmetic behavior test table enables both INT_SDIV and INT_SREM with the
matching signed reference expressions, and a new concrete test checks mixed-sign
combinations (-5/2, 5/-2, -5/-2, ...) that an unsigned implementation cannot
satisfy.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Use claripy.SMod for INT_SREM

Per review, INT_SREM uses claripy.SMod directly instead of the equivalent
in1 - claripy.SDiv(in1, in2) * in2. Verified identical to a truncated-toward-zero
reference over 100k random 64-bit pairs, including the INT_MIN / -1 corner.

* Address pcode signed arithmetic review comments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-31 12:23:05 -07:00
Fish
61bac8ffd0
SimConstantVariable: Fix overflows and value out of range errors. (#6738)
* SimConstantVariable: Fix overflows and value out of range errors.

* Fix usages of SimConstantVariable.

* Mask the value.

* Fix negative values.
2026-07-29 13:58:20 -07:00
pre-commit-ci[bot]
d38cc5a019
[pre-commit.ci] pre-commit autoupdate (#6721)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.22 → v0.16.0](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.22...v0.16.0)

* Apply fixes

* Add values()

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kevin Phoenix <kevin@kphoenix.us>
2026-07-29 13:46:11 -07:00
angr-bot
a4b05a0ce9 Update version to 9.3.2.dev0 [ci skip] 2026-07-29 09:44:55 +00:00
Fish
7cedfbfa8b
Dephication: Correctly consider phi congruence classes. (#6735) 2026-07-29 01:53:02 -07:00
Fish
b9358da5da
AIL: Fix __eq__. (#6728)
* AIL: Merge likes/matches into a single mode-parameterized walk

* AIL: Make __eq__ idx-aware at every node, not just the root

* AIL: Stop hashing fields that equality does not compare

* AIL: Regression-test the hash/eq contract

* AIL: Compare bits in StringLiteral and Struct

* AIL: Replace the CMP_* constants with a CmpMode enum

* AIL: cargo fmt

* AIL: Compare and hash rounding_mode on Convert and BinaryOp

* Update comments.
2026-07-29 01:22:10 -07:00
Ati Priya
6933b065f0
Decompiler: rewrite CondBE and CondNB ccalls on amd64 (#6645)
* Decompiler: rewrite CondBE and CondNB ccalls on amd64

* Decompiler: fix inverted CondZ/CondNZ over G_CC_OP_COPY on amd64

* tests: add binary-driven regressions for CondBE/CondNB ccall recovery
2026-07-29 01:11:09 -07:00
Quintin Kong
508ac3a44c
aarch64: fix adcs/sbcs carry flag (select on cc_dep3, not cc_dep2) (#6702)
arm64g_calculate_flag_c selected the ADC*/SBC* carry-in with `cc_dep2 != 0`
(the second operand). The arm64 flag thunk layout puts the old carry in
cc_dep3 (angr's own comment: "DEP3 = oldC (in LSB)", matching VEX's
guest_arm64_helpers.c). So the C flag after adcs/sbcs was computed from an
operand value instead of the incoming carry.

Random operands usually mask it (cc_dep2 != 0 nearly always holds); the
equal-operand case exposes it, e.g. `sbcs x,y,y` must give C = oldC but
returned a value keyed on y. flag_n/z/v and the arm32 port (which correctly
uses cc_dep3) were unaffected.
2026-07-29 00:57:43 -07:00
Fish
6b2637c446
Dephication: Consider the vvar used in block-end jumps during intersection. (#6733)
* Dephication: Consider the vvar used in block-end jumps during intersection.

This is a subtlety in the Sreedhar et. al. paper.

* Fix test cases.
2026-07-29 00:42:21 -07:00
Fish
a7ae033c69
EagerEval: Fix broken expression type comparison. (#6734)
This is a bug introduced by the Rusty AIL migration.
2026-07-29 00:05:07 -07:00
Fish
659f3d7f5d
CCodeGen: Fix the display of negative offsets. (#6730)
* CCodeGen: Fix the display of negative offsets.

* Fix a test case.
2026-07-28 22:29:12 -07:00
Audrey Dutcher
4462c849b9
RegionIdentifier: typecheck post-regionoverlay (#6664) 2026-07-28 19:52:00 -07:00
Fish
22613f4a0a
CompleteCallingConventions: fail loudly when all workers die. (#6718) 2026-07-27 18:20:59 -07:00
Fish
2fffb71f86
state_plugins: Map the heap region lazily and grow it on demand. (#6715) 2026-07-27 18:20:07 -07:00
Fish
55f059982b
UltraPage: Make the symbolic map an actual bitmap. (#6714) 2026-07-27 17:35:52 -07:00
Fish
1a5eedf622
Decompiler: Speed up AIL block simplifications. (#6712)
- Make SPropagator, SRDA, and BlockSimplifier normal classes instead of Analysis classes.

- Share peephole optimizer instances across BlockSimplifiers.

- BlockSimplifier: Skip unnecessary peephole passes; avoid block-level comparisons for fixpoint determination.

- Add a runtime-only peephole_optimized flag to AIL statements so we skip running peephole optimizations on already optimized statements.
2026-07-27 16:34:10 -07:00
Fish
2080c15a26
RemoveRedundantBitmasks: Fix an in-place expression update. (#6722) 2026-07-27 15:53:53 -07:00
dependabot[bot]
3e2e4b90fc
ci: bump taiki-e/install-action from 2.84.0 to 2.85.2 (#6719)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.84.0 to 2.85.2.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](a6b2e2dcd8...41049aa566)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.85.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 13:13:52 -07:00
dependabot[bot]
59400706f8
ci: bump astral-sh/setup-uv from 8.3.2 to 9.0.0 (#6720)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.2 to 9.0.0.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](11f9893b08...c771a70e62)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 9.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 13:13:35 -07:00
Fish
bc72b9e1a6
MCP: Protect stdio from forked workers. (#6717)
Some checks failed
CI / Rust Check (push) Failing after 6s
CI / rust_test (ubuntu-latest) (push) Failing after 9s
Test with coverage / Build (push) Failing after 8s
Test with coverage / Test (push) Has been skipped
Test with coverage / Test-1 (push) Has been skipped
Test with coverage / Test-2 (push) Has been skipped
Test with coverage / Test-3 (push) Has been skipped
Test with coverage / Test-4 (push) Has been skipped
Test with coverage / Test-5 (push) Has been skipped
Test with coverage / Test-6 (push) Has been skipped
Test with coverage / Test-7 (push) Has been skipped
Test with coverage / Test-8 (push) Has been skipped
Test with coverage / Test-9 (push) Has been skipped
Test with coverage / Test Rust packages (push) Failing after 6s
Test with coverage / Report (push) Has been skipped
CI / ci (push) Has been cancelled
CI / Test installation (macos-26, py3.12) (push) Has been cancelled
CI / Test installation (ubuntu-24.04, py3.14) (push) Has been cancelled
CI / Test installation (windows-2025, py3.12) (push) Has been cancelled
CI / rust_test (macos-latest) (push) Has been cancelled
CI / rust_test (windows-latest) (push) Has been cancelled
2026-07-27 03:01:56 -07:00
Fish
651d9cccc2
Tests: Speed up decompiler-related test cases (strike 1). (#6713) 2026-07-27 01:56:24 -07:00
Fish
9c1fb2367e
StructuringOptimizationPass: Cache structurability across passes. (#6711) 2026-07-27 00:03:32 -07:00
Quintin Kong
e4ff240001
x86/amd64: fix UMUL CF/OF flags (widen before multiply) (#6703)
Fixes #6067

pc_actions_UMUL computed the product at operand width, so the extracted
"high half" was always zero and CF/OF (OF = CF) were constantly 0:

    lo = (cc_dep1 * cc_dep2)[nbits-1:0]   # truncated to nbits
    hi = (lo >> nbits)[nbits-1:0]         # lo is nbits wide -> always 0

The sibling pc_actions_SMUL is correct because it widens first
(sign_extend). Mirror it with zero_extend: multiply the operands widened to
2*nbits and take the high half. For CC_OP_MUL{B,W,L,Q}, CF = OF =
(high half != 0) per the Intel SDM. This is why `imul` reported CF/OF
correctly while `mul` did not (issue #6067: `mul %ebx` left CF clear).
2026-07-26 22:32:25 -07:00
Fish
6d5860d0ba
COWDict: Faster chain walks. (#6710) 2026-07-26 22:20:20 -07:00
Fish
3b41a92e29
SimpleSolver: Hash memoization. (#6709) 2026-07-26 22:10:26 -07:00
Fish
83f364dc01
Memoize C++ prototype parsing. (#6708) 2026-07-26 21:33:20 -07:00
Fish
787c2c7d8e
SimpleSolver: Memoize least common ancestors on type lattices. (#6707) 2026-07-26 17:17:09 -07:00
Fish
b37cca0101
VRA: Register the Reference stack variable against its atom. (#6705) 2026-07-26 07:58:06 -07:00
Fish
fc7ff8e62c
CFGFast: Linear scan heuristics for monotonic byte ramps and floats. (#6701) 2026-07-26 00:35:43 -07:00
Yan Shoshitaishvili
83d0061e12
Calling conventions: ignore stack canary comparisons as returns (#6699)
* Calling conventions: ignore stack canary comparisons as returns
2026-07-24 16:34:21 -07:00
Yan Shoshitaishvili
f74d1c5c1e
Typehoon: index subtype constraint components (#6696)
* Typehoon: index subtype constraint components

* Refactor the code to eliminate weird terminology.

* Fix test cases.

---------

Co-authored-by: Fish <fishw@asu.edu>
2026-07-24 16:26:47 -07:00
Max Ambaum
be9c801b16
Add __sprintf_chk as a simprocedure (#6698)
* Add __sprintf_chk as a simprocedure

http://refspecs.linux-foundation.org/LSB_4.0.0/LSB-Core-generic/LSB-Core-generic/libc---sprintf-chk-1.html
2026-07-24 14:42:53 -07:00
Fish
3efd1ec6db
SimLibrary/SimSyscallLibrary: Treat None prototypes as absent. (#6673) 2026-07-24 11:29:58 -07:00
Ati Priya
db21fb0fee
Rewrite the amd64 CondO/CondNO ccall (#6693)
* Rewrite the amd64 CondO/CondNO ccall family

amd64g_calculate_condition with cond CondO/CondNO had no rewrite arm at
all, so every jo/jno/seto/cmovno site leaked into the decompilation as an
uncompilable _ccall(0|1, cc_op, ...).

Add arms for the cc_op families that define OF:

  LOGIC{B,W,L,Q}  and/or/xor always clear OF -> constant 0 / 1
  ADD{B,W,L,Q}    -> __OFADD__(dep_1, dep_2)
  SUB{B,W,L,Q}    -> __OFSUB__(dep_1, dep_2)
  UMUL{B,W,L,Q}   -> __OFUMUL__(dep_1, dep_2)
  SMUL{B,W,L,Q}   -> __OFSMUL__(dep_1, dep_2)
  INC{B,W,L,Q}    result == signed minimum
  DEC{B,W,L,Q}    result == signed maximum
  COPY            test the stored OF bit

The overflow helpers follow the existing __CFADD__ arm: a named usercall
whose operands carry the operation width. CondNO reuses the same helper
and compares it against zero.

Unsigned multiply overflow is defined as "the high half of the full
2N-bit product is nonzero", i.e. the product does not fit in N unsigned
bits. Note this is NOT the threshold used by the x86 rewriter, which
compares the product against 1 << (N - 1) -- that is the signed
threshold, half the correct unsigned one, and it reports overflow for
every product in [2^(N-1), 2^N - 1] even though those fit. At 8 bits it
misclassifies 820 of 65536 operand pairs, all false positives.

pc_actions_UMUL in the VEX ccall helpers is itself wrong here: it
multiplies two N-bit values without widening, so its `>> nbits` is
always zero and its CF/OF do not agree with the hardware. The rewrite
arm follows the hardware and pc_actions_SMUL's (correct) structure
instead; fixing the helper is left alone.

Every arm was checked exhaustively at 8 bits against
pc_calculate_condition, and the ADD/SUB/UMUL/SMUL arms additionally
against real setcc results.

* Drop the synthetic CondO fixture test

The real gzip and file fixtures already cover the CondO arms; a purpose
built binary added nothing that the unit tests do not already check.

* Cover the CondNO overflow path with a real binary

tar's argp helper guards a multiply with 'mul %rbp; jno', exercising the
CondNO side of UMULQ that gzip and file do not reach. Other cc_op
families still leak a ccall in that function, so only the OF conditions
are asserted.

* Cover the CondO overflow arms with three more real binaries

coreutils cat, grep and zlib's minigzip each carry the xalloc /
__builtin_mul_overflow idiom, between them exercising CondO against
ADDQ, SMULQ and UMULQ across three separate projects. Verified symbols
and addresses are cited on each test.

* Correct the tar overflow test to CondO

The jno there is canonicalized into CondO with an inverted branch, so the
ccall reaching the rewriter carries cond 0, not cond 1. The negation seen
in the output is the structurer's, not the condition's.
2026-07-24 11:28:39 -07:00
Fish
91cc026062
AILVexLifter: Fix libVEX overread by padding in convert_from_lift. (#6686) 2026-07-24 01:43:20 -07:00
Yan Shoshitaishvili
90062a9914
Support native AIL Abs expressions in light engines (#6683)
* Fix AIL Abs unary operation dispatch

* Fix Abs regression test lint
2026-07-24 00:43:03 -07:00
Yan Shoshitaishvili
fe434a049b
Decompiler: reject float constants in string simplifiers (#6682)
* Decompiler: reject float constants in string simplifiers

* Tests: allow private simplifier coverage
2026-07-24 00:28:36 -07:00
Yan Shoshitaishvili
208ec719a6
Decompiler: disambiguate Extract condition placeholders (#6677) 2026-07-23 22:33:21 -07:00
Yan Shoshitaishvili
8b98c5ad6c
Keep eager evaluation integer-only (#6681) 2026-07-23 22:30:15 -07:00
Yan Shoshitaishvili
2dd6cb393b
Typehoon: preserve standard SimTypeNum types (#6678) 2026-07-23 21:15:18 -07:00
Yan Shoshitaishvili
cf54c35b9b
AIL: handle HAddV operations (#6680)
* AIL: handle HAddV operations

* Fix HAddV CI diagnostics
2026-07-23 21:10:10 -07:00
Vedant Soni
1808e7fadd
Migrate rust demangling to pydemumble (#6663)
* use pydemumble to demangle rust symbols

* update tests

* remove rust_demangle and unpin pydemumble from pyproject

* pin pydemumble to latest version
2026-07-23 17:32:00 -07:00
Michał Kowalczyk
c13863214c
docs: Fix dangling links (#6533) 2026-07-23 17:31:25 -07:00
Fish
75500dd271
VRA: Copy over integer signedness to type constraints during Convert. (#6672)
* VRA: Copy over integer signedness to type constraints during Convert.

* Adjust a test case.
2026-07-23 15:57:26 -07:00
Fish
85d77f0301
RemoveRedundantShifts: Retain sign extensions. (#6671)
`(x << N) >> N` was rewritten into a Convert-of-Convert pair whose outer Convert
zero-extended for BOTH logical (Shr) and arithmetic (Sar) right shifts. For Sar
this is unsound: the idiom sign-extends the low (M-N) bits, but the zero-extending
Convert rendered as a bitmask, so e.g. `(int)(x << 20) >> 20` decompiled to
`x & 0xfff` (and the 64-bit twin to `x & 0xffffffffff`), which drops the
sign bit.
2026-07-23 14:43:45 -07:00
Fish
0849ddb03c
CFGFast: Tolerate leading null bytes during string scanning. (#6670)
* CFGFast: Tolerate a single leading null byte when scanning for strings.

* CFGFast: Scan for mixed pointers in high-based images during complete scanning.

* Tests: Add a regression test for data detection in a PE32 with data tables in .text.
2026-07-23 14:43:32 -07:00
Fish
8dc3772762
tests: CFG resolves delay-load import calls to named imports. (#6666) 2026-07-23 14:16:54 -07:00
Fish
8608d976e3
Decompiler: Fix re-rendering of decompilation caches reloaded from angrDb. (#6669)
* Decompiler: Fix re-rendering of decompilation caches reloaded from angrdb.

Re-rendering a deserialized codegen (as angr-management does on display/edit)
dropped variable declarations and string constants and rendered slightly different
C, because several pieces of state were not restored:

- VariableManagerInternal never serialized variable_to_types / variables_with_
  manual_types, so get_variable_type() returned None and all locals rendered as int.
- parse_codegen did not attach the project, and left display options that serialize
  as None (e.g. max_str_len) unset. Attach project and initialize display options
  from the codegen constructor defaults.
- CConstant string references lost MemoryData.content (not serialized); re-read it
  from the loader at parse time so strings render as strings, not raw addresses.
- regenerate_text() now refreshes CFunction.unified_local_vars from the (restored or
  updated) variable manager so declarations reflect current types.
- CBinaryOp._cstyle_null_cmp is rebuilt from the codegen flag in set_codegen,
  restoring !x vs x == 0.
- Compound-assignment folding (x += 1) compared unified variables by identity; use
  == so it works across deserialized variables that are equal but not the same
  object.

Adds an end-to-end test (1after909::doit) asserting a reloaded cache re-renders
byte-identically.

* Lint code.
2026-07-22 05:01:09 -07:00
Fish
2eadec31c6
Decompiler: Tolerate peephole optimizations not importable at parse time. (#6668)
* Decompiler: Tolerate peephole optimizations not importable at parse time.

* Lint code.
2026-07-22 03:56:06 -07:00
Fish
2f8266bdda
CFGFast: Constant-fold AMD64 PE IAT calls. (#6667)
* CFGFast: constant-fold AMD64 PE IAT jumps/calls, bypassing the resolver

* CFGFast: fold delay-load IAT calls too, matching MemoryLoadResolver

* tests: update lwip xrefs for corrected RO-region ordering
2026-07-22 03:55:07 -07:00
Fish
02c374b5b4
DecompilationCache: Serialization support. (#6624)
Also,

- Refactored variable_kb into kb.dec_variables.

- Spill decompilation cache into RuntimeDb.

- Save decompilation cache into angrDb. Decompilation results can be preserved across runs.

- No longer check in _pb2.py files; they are generated during build.
2026-07-22 03:03:40 -07:00
Ati Priya
e8858b82cd
typehoon: type pointer-to-array locals as pointers (#6620)
* typehoon: type function-scope pointer-to-array locals as element pointers (T*) so they render as pointers, not arrays

c_repr drops the "*" for pointer-to-array, so such locals were declared
as arrays and assignments to them were invalid C. Flatten T (*)[N] to
T * for function-scope variables; globals and plain arrays unchanged.

* tests: accept element-pointer rendering in reverting-switch-lowering test

Locals typed as element pointers render "ptr = p + 1;" instead of
"ptr = &p[1];"; both are equivalent. Accept either form.
2026-07-22 02:28:25 -07:00
Yan Shoshitaishvili
bcf5902d07
Keep entry jumps inside sized function symbols (#6651)
* Keep entry jumps within sized function symbols

* Narrow CFG regression symbol type

* Use committed binary for CFG entry-jump regression
2026-07-22 01:52:32 -07:00
Yan Shoshitaishvili
21cea3815f
Refine C++ symbol prototypes with machine ABI facts (#6652)
* Refine C++ symbol prototypes with machine ABI facts

* Preserve explicit C++ calling conventions

* Use prebuilt C++ calling-convention fixture
2026-07-22 01:50:12 -07:00
Fish
1102d5fc47
RustSimTypeInt: Include size in equality and hashing; fix copy() dropping size (#6659)
Fixes #6625.
2026-07-22 01:48:02 -07:00
volodya
95fa7ea003
utils/graph: answer subgraph_between_nodes reachability in one pass (#6662)
subgraph_between_nodes() copied the whole graph and then ran a fresh
networkx.has_path() search for every (candidate successor, frontier node)
pair, so its worst case was O(candidate_edges * frontier_nodes * (V + E)).
Negative queries are the expensive ones: a successor that cannot reach any
frontier node forces a full traversal of everything reachable from it, once
per frontier node.

RegionIdentifier._find_initial_loop_nodes() calls this for every loop it
recovers, so a loop head whose successors lead into a large region that only
returns to the head (which the function's own "remove all incoming edges of
the source" step makes unable to reach any latch) makes loop recovery
quadratic. On a 1.1k-node AArch64 CFG built to have that shape, decompilation
spends 40.5s of 52.5s inside 608,847 has_path() calls.

Replace the repeated searches with a single reverse multi-source BFS from the
frontier that stops at the source; membership in the resulting set answers
every reachability question the forward walk asks. Stopping the reverse walk
at the source is exactly equivalent to deleting all incoming edges of the
source, so the graph copy is no longer needed either. Also peel dead leaves
with a degree worklist instead of rescanning all nodes after each removal.

Results are unchanged, including node/edge insertion order and edge
attributes: 6,000 randomized fixed-seed comparisons (3,000 graphs x both
include_frontier modes) against the previous implementation are identical, as
is the decompiler output on the CFG above (52.5s -> 12.5s end to end, 0
has_path() calls).

One behaviour change is deliberate: frontier is now turned into a set before
the "source not in graph or any(node not in graph for node in frontier)"
check rather than after it. In the old order an iterator argument was consumed
by that check, so the subsequent set(frontier) was empty and the function
silently sliced with no frontier at all. Every in-tree caller passes a list or
a set, so no in-tree behaviour changes.
2026-07-22 01:41:36 -07:00
Ati Priya
26736ec096
decompiler: ExpressionNarrower over-narrows vvars used as Insert bases (#6649)
EffectiveSizeExtractor skipped the base operand of an Insert, so a use as
an Insert base contributed no width requirement in AILSimplifier's
narrowing pass. A register vvar whose remaining uses were narrow (e.g. an
ah-style byte Extract) was then narrowed below the Insert base width and
zero-extended back at the use site, destroying every preserved byte of the
base and misplacing the extracted byte:

    v1 = a0->field_10;                        // v1 narrowed to char
    a0->field_10 = _INSERT(v1, 1, v1 & 239);  // upper 3 bytes zeroed,
                                              // wrong byte masked

for what is really a full-width read-modify-write (field_10 &= 0xffffefff).

Walk the Insert base in EffectiveSizeExtractor so it is recorded as a
full-width use: every byte outside the inserted range is preserved into
the result, so the base can never be narrowed below its own width.
2026-07-20 23:33:10 -07:00
Kevin Phoenix
0c87229837
Pass new ret_expr during construction rather than mutating (#6642)
* Pass new ret_expr during construction rather than mutating

* More improvements.

---------

Co-authored-by: Fish <fishw@asu.edu>
2026-07-20 23:22:14 -07:00
Ati Priya
8ad0dc91aa
Decompiler: Recover CondNL (jge) over SUB and LOGIC in the amd64 ccall rewriter (#6657) 2026-07-20 23:19:14 -07:00
Vedant Soni
a3c8d835c7
RustStructuredCodeGenerator : align __init__ with CStructuredCodeGenerator.__init__ (#6650)
* Align RustStructuredCodeGenerator.__init__ with CStructuredCodeGenerator

* type fix
2026-07-20 23:17:44 -07:00
Yan Shoshitaishvili
0fcf7c050b
Copy stateful address concretization strategies on state fork (#6653)
* Copy stateful address concretization strategies

* Fix address strategy test typing
2026-07-20 23:17:11 -07:00
Yan Shoshitaishvili
dd1cefa642
Handle case-insensitive P-code memory space names in AIL conversion (#6654)
* Handle uppercase P-code memory spaces

* Fix P-code regression test typing
2026-07-20 23:15:12 -07:00
Kevin Phoenix
d099a933bf
Remove ConcreteBackerMixin (#6643) 2026-07-20 11:00:28 -07:00
pre-commit-ci[bot]
2c6c986f66
[pre-commit.ci] pre-commit autoupdate (#6644)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.21 → v0.15.22](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.21...v0.15.22)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-20 10:49:02 -07:00
dependabot[bot]
7d74ec8924
ci: bump actions/setup-python from 6.3.0 to 7.0.0 (#6637)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6.3.0 to 7.0.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](ece7cb06ca...5fda3b95a4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-20 10:08:44 -07:00
dependabot[bot]
41baf181f3
rust: bump regex from 1.12.2 to 1.13.1 (#6640)
Bumps [regex](https://github.com/rust-lang/regex) from 1.12.2 to 1.13.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.12.2...1.13.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-version: 1.13.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-20 10:08:14 -07:00
dependabot[bot]
ada9fcd585
rust: bump serde from 1.0.228 to 1.0.229 (#6639)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.228 to 1.0.229.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.228...v1.0.229)

---
updated-dependencies:
- dependency-name: serde
  dependency-version: 1.0.229
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-20 10:08:07 -07:00
dependabot[bot]
635ddcd134
ci: bump taiki-e/install-action from 2.83.2 to 2.84.0 (#6636)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.83.2 to 2.84.0.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](43aecc8d72...a6b2e2dcd8)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.84.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-20 10:07:54 -07:00
dependabot[bot]
e9d795193a
ci: bump actions/checkout from 7.0.0 to 7.0.1 (#6638)
Bumps [actions/checkout](https://github.com/actions/checkout) from 7.0.0 to 7.0.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](9c091bb21b...3d3c42e5aa)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-20 10:07:39 -07:00
Kevin Phoenix
d145bd41fd
Remove global condition from SimState (#6641) 2026-07-20 10:06:54 -07:00
Ati Priya
3984a816b6
Fix bswap32 intrinsic name: __buildin_ -> __builtin_ (#6635)
gcc has no __buildin_bswap32, so the emitted call never resolved.
2026-07-20 02:12:23 -07:00
Kevin Phoenix
1944e72e90
Remove widen() from state api (#6632) 2026-07-17 13:23:53 -07:00
Kevin Phoenix
bd58a30e6e
Update installation CI run (#6631) 2026-07-17 11:50:28 -07:00
Kevin Phoenix
a6f88cd76f
Move STRONGREF_STATE logic entirely within history plugin (#6629) 2026-07-17 11:15:50 -07:00
Kevin Phoenix
c213b234b3
Pin a recent pydantic-ai (#6630) 2026-07-17 11:09:09 -07:00
Kevin Phoenix
352d642363
identifier: Deduplicate 22 copies of rand_str into shared helpers (#6628) 2026-07-17 10:41:08 -07:00
Vedant Soni
07115bb2ad
RustStructuredCodeGenerator: Make indent_delta configurable (#6626)
* add indent_size kwarg to rustcodegen

* make indent_delta variable
2026-07-17 02:06:11 -07:00
Fish
d9071a160e
Decompiler: Fix "unsupported instructions" when rbp is used as GPR. (#6627)
* VariableRecoveryFast: Do not seed bp with a stack address when bp is a GPR.

* RegisterSaveAreaSimplifierAdvanced: Fix extern check and handle shrink-wrapped spills.
2026-07-16 17:25:45 -07:00
angr-bot
f70bd5f0e1 Update version to 9.3.1.dev0 [ci skip] 2026-07-15 10:17:19 +00:00
Fish
656890235f
SwitchDefaultCaseDuplicator: handle cmov-split predecessor blocks. (#6622)
When duplicating the default-case node for an unexpected predecessor,
the predecessor address comes from the function-level CFG graph, but
the edge removal happens on the AIL graph. If clinic broke the
predecessor block into multiple nodes (e.g., at a cmov), the AIL block
at the CFG predecessor address no longer has a direct edge to the
default-case node, and remove_edge raised a NetworkXError.

Fix: Search the default-case node's AIL predecessors for the fragment that
falls within the CFG predecessor block's address range, and skip with a
warning if none is found.
2026-07-15 02:13:00 -07:00
Fish
7b24f31e6d Pin pydemumble to v0.0.1. 2026-07-15 02:01:02 -07:00
Fish
77ea846527
angrDb: Direct load and store of serialized bytes for CFG and functions. (#6610)
* Move serialized functions directly into LMDB on load.

* Serialize the callgraph instead of rebuilding it on load.

* Move serialized CFG nodes directly into LMDB on angrdb load.

* Copy clean spilled functions directly from LMDB on dump.

* Do not serialize empty variable managers.

* Serialize CFG edges at the key level to avoid node materialization.

* Bulk-insert function and variable rows via SQLAlchemy Core.

* Skip the CFGNode.function_address fill when node records carry it.

* struct-pack spilled edge data instead of protobuf.

* Avoid O(n^2) full-LRU-order copy in spilling eviction.

* Byte-copy clean spilled CFG nodes when serializing a CFG model.

* Lint code.
2026-07-15 01:40:46 -07:00
Fish
654fbcea8d
AIL: Port the AIL VEX lifter to Rust. (#6608)
* port the VEX->AIL converter and Manager to Rust

* allow a non-constant rounding mode on Convert/BinaryOp

* construct AilExpression/AilStatement natively in the converter

* resolve libpyvex symbols on Windows

* classify name-only VEX ops; guarantee ins_addr on every stmt

* take the Manager as a typed pyclass handle in the converter

* Lint code.

* ailment.pyi: add Manager and VEXIRSBConverter stubs
2026-07-15 01:23:34 -07:00
Fish
84684c9861
Typehoon: Improve determinism. (#6621)
* Typehoon: Hash type-system objects by a stable type tag instead of class identity.

* Typehoon: Hash Variance/ConstraintGraphTag/FORGOTTEN enums by value.

* Decompiler/codegen: Emit local and extern struct typedefs in a stable, rename-independent order.

* Decompiler/codegen: Break struct layout ties by a rename-proof definition order.

* sim_type: Serialize SimStruct._def_order in the JSON schema.
2026-07-15 01:21:37 -07:00
Ati Priya
36587c513a
FunctionParser: Serialize Function._call_sites. (#6613)
* FunctionParser: Serialize Function._call_sites.

serialize stores the transition graph but not Function._call_sites, and
parse_from_cmsg never repopulated it, so functions loaded back from LMDB
(SpillingFunctionDict eviction, angrdb) returned nothing from
get_call_sites(), get_call_target(), and get_call_return().

_call_sites cannot be rebuilt from transition graph edges. Store the
mapping in a new repeated CallSite field on the Function message and
restore it in parse_from_cmsg.

* lint

* Fix pyright errors in FunctionParser.

* Fix pyright error in LMDB function manager test.
2026-07-14 17:15:50 -07:00
pre-commit-ci[bot]
8340441183
[pre-commit.ci] pre-commit autoupdate (#6618)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.20 → v0.15.21](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.20...v0.15.21)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-13 13:39:06 -07:00
dependabot[bot]
aedd30a80a
rust: bump rustc-hash from 2.1.1 to 2.1.3 (#6617)
Bumps [rustc-hash](https://github.com/rust-lang/rustc-hash) from 2.1.1 to 2.1.3.
- [Changelog](https://github.com/rust-lang/rustc-hash/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/rustc-hash/compare/v2.1.1...v2.1.3)

---
updated-dependencies:
- dependency-name: rustc-hash
  dependency-version: 2.1.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-13 10:05:38 -07:00
dependabot[bot]
c6fb169176
ci: bump astral-sh/setup-uv from 8.3.0 to 8.3.2 (#6615)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.0 to 8.3.2.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](d31148d669...11f9893b08)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-13 10:02:33 -07:00
dependabot[bot]
435434da84
ci: bump taiki-e/install-action from 2.82.9 to 2.83.2 (#6614)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.82.9 to 2.83.2.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](4684b84056...43aecc8d72)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.83.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-13 10:02:17 -07:00
Fish
fb0dbcfe15
Analysis: Throttle _release_gil sleep frequency. (#6612) 2026-07-11 13:02:44 -07:00
Fish
84ea4412cc
CFGFast: Throttle progress notifications. (#6611) 2026-07-11 13:02:25 -07:00
Fish
132e69254c
ccall-rewriter: Fix CondBE/CondB SUB ccall size mismatch. (#6609) 2026-07-11 02:40:26 -07:00
Fish
8088c73a09
Speed up multiple rust-decompiler-specific analyses. (#6607)
* rust: make FlirtSigPropagation avoid loading Function objects.

* rust: optimize CleanupFunctionIdentification; use deque for FIFO queues.

* rust: cache demangle + avoid loading Functions in TypeDBLoader/RustSymbolRecovery.

* functions: inline the per-edge jumpkind lookup in FunctionParser.

* flirt: meta-only loads where only meta fields are used.

* Tweaks and adjustments.

* Adjust SootFunction.
2026-07-11 02:35:03 -07:00
Fish
bdf1fe3764
SimplifyBitwiseInserts: Support cases where pb2x is wider than pb2. (#6606) 2026-07-10 21:29:22 -07:00
Fish
44a5b88116
Ssailification: Add defs for addr-ref stack slots with no defs. (#6605) 2026-07-10 14:04:55 -07:00
Fish
d5575a98aa
Ssailification: Generate phi for vars with narrower same-offset partial defs. (#6604) 2026-07-10 13:05:19 -07:00
Fish
c34abbed75
Ssailification: Track subregisters in reg blackout. (#6603) 2026-07-10 11:05:11 -07:00
Fish
b17f62eb35
Fix pickling of Projects with active RtDb. (#6602)
* Fix pickling of Projects with active RtDb.

* Lint code.
2026-07-10 01:25:55 -07:00
Fish
278f0914a5
Speed up test cases by building scoped CFGs. (#6601)
* Speed up test cases by building scoped CFGs.

* Scope CFG/CCC for more test cases.
2026-07-10 01:17:46 -07:00
Fish
c03a6512d7
Work around rust-demangler crash. Fix #6598. (#6600) 2026-07-09 20:52:15 -07:00
Fish
aecf13c7b1
ailment.pyi: Give AIL marker classes exact static types. (#6597)
* ailment.pyi: Give AIL marker classes exact static types.

* Trim multi-variant accessor docstrings to the per-class variant
2026-07-09 03:49:53 -07:00
Fish
f9c21658cc
RuntimeDb: Cleanup at process exit. (#6596)
CPython does not guarantee that __del__ runs at interpreter shutdown, so
RuntimeDb.cleanup() could be skipped, leaving *_angr_rtdb directories on disk.
This PR introducts an atexit hook that cleans up all live RuntimeDb instances.
2026-07-09 01:26:21 -07:00
Fish
e5c13a95ef
RemoveRedundantBitmasks: Eliminate an expr operand setter. (#6595) 2026-07-09 01:25:28 -07:00
Fish
9413721e33
Ssailification: Assume all arg regs are used at uncertain-arg-count call sites. (#6594)
* SRDAView: Return None when multiple reg vvars reach a program point.

* Ssailification: Assume all arg regs are used at uncertain-arg-count call sites.

* Add a regression test.
2026-07-08 17:34:42 -07:00
Fish
55f5a6a87b
should_use_hex: Remove value saturation rule. (#6593)
* should_use_hex: Remove value saturation rule.

* Fix test cases. Introduce two new rules.
2026-07-08 17:34:26 -07:00
Fish
9b6c200b10
Ssailification: Do not resize fully-read args. (#6592)
* expression_narrower: treat Extract offsets as bytes, not bits

* Ssailification: Do not resize fully-read args.
2026-07-08 13:38:11 -07:00
Kevin Phoenix
faafe65ff3
native/angr: add package metadata required by workspace cargo lints (#6591)
The clarirs workspace lints enable clippy::cargo, and CI runs clippy with
-D warnings, so the angr crate's missing description/license/readme/
keywords/categories/repository metadata failed the Rust Check job.
Inherit the shared fields from workspace.package.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 11:42:51 -07:00
Fish
783932a24c
Const: Fix sign_bit for wide values. (#6590) 2026-07-08 11:17:59 -07:00
Fish
f5f6667a77
StackPointerOffset: Wrap around offset according to bits. (#6589)
* StackPointerOffset: Wrap around offset according to bits.

* Format the code.

* Add test cases.
2026-07-08 10:38:03 -07:00
Fish
9235f7fd27
Migrate AIL classes to Rust (#5967)
This PR migrates AIL classes (Block, Statement, Expression, etc.) to Rust. Key changes include:

- Block, Statement, and Expression are native Rust objects. Every access to their properties will create a new Python class object. Therefore, `expr.dst is expr.dst` will always evaluate to False. You'll want to do `expr.dst == expr.dst` instead.

- However, keep in mind that `expr_0 == expr_1` can be expensive because equality checks may potentially go over two expression trees and compare every single node. In this case, you may want to resort to `expr_0.idx == expr_1.idx`. There are still a few places where `idx` is reused; we plan to remove all such cases and guarantee the uniqueness of `idx` for all Statements and Expressions during a single decompilation run.

- `type(expr) is Const` no longer works. You must use `isinstance(expr, Const)` instead. `isinstance(..., ExpressionCls/StatementCls)` is also more expensive than before due to the use of custom meta classes.

- New changes to AIL requires rebuilding the Rust component to land. You can do `python setup.py build_rust --inplace --release` to rebuild the angr Rust component in-place in an editable install.

---------

Co-authored-by: Kevin Phoenix <kevin@kphoenix.us>
2026-07-08 02:10:07 -07:00
Fish
ff0dafb6e5 Update version to 9.3.0.dev0 [ci skip] 2026-07-08 01:03:56 -07:00
angr-bot
b7a375ae1a Update version to 9.2.224.dev0 [ci skip] 2026-07-08 07:55:26 +00:00
Fish
c151e8b2b6
RegionOverlay: Keep extra-edge adjacency for view-only nodes leaving the successor set. (#6585)
* RegionOverlay: Keep extra-edge adjacency for view-only nodes leaving the successor set.

absorb_successor_into() re-attaches an absorbed successor's out-edges to a member
node as view-only extra edges (_extra_full_edges). The targets of these edges stay
in the with-successors view's node set even when structuring later removes their
last crossing edge, at which point they are no longer in successor_nodes().

_iter_view_in_edges() and _iter_view_out_edges() early-returned for non-member
nodes not in successor_nodes(), skipping the _extra_full_edges scan. For such a
view-only node the extra edge remained visible from its member source's
out-adjacency while the node's own in-adjacency came up empty: the view's pred and
succ adjacencies went asymmetric.

Fix: for non-member view nodes, run the successor-derived adjacency only when the
node is in successor_nodes(), but always scan _extra_full_edges, so every edge in
the view is visible symmetrically from both endpoints.

* Add a test case.
2026-07-07 14:59:21 -07:00
Fish
79ec25dc24
FunctionParser: convert previous_names protobuf field to a plain list. (#6584) 2026-07-07 14:21:32 -07:00
Fish
18d7fff834
engines: Handle QSubV. (#6583) 2026-07-07 14:21:00 -07:00
Fish
2d96249533
format_parser: Fix float-len-mod specs and per-class spec-table cache. (#6582)
* format_parser: Fix float-len-mod specs and per-class spec-table cache.

* Lint code.
2026-07-07 14:20:38 -07:00
Fish
f30a48fbf3
FunctionHandler: Fix crash on implicit-outparam return values. Close #6536. (#6581) 2026-07-07 02:46:58 -07:00
Fish
222a01df43
Phoenix: Speed up last-resort refinement. (#6580)
- Answer dominance queries in O(1) through Euler-tour intervals, which are
computed once per run from the dominator tree, and compute the DAG check
once.

- RegionOverlay: Cache successor-node adjacency in _OverlayAdjAtlas.
2026-07-07 01:17:18 -07:00
Fish
db30651e60
LoweredSwitchSimplifier: Do not add head edges for copied case targets. (#6579) 2026-07-07 01:16:59 -07:00
Fish
3a6f130d99
CCodeGen: Render leftover IncompleteSwitchCaseHeadStatement as if-gotos. (#6578) 2026-07-06 22:40:17 -07:00
Fish
75c51bab29
RegionOverlay: detach_edge removes view-only absorbed-successor edges. (#6577)
* RegionOverlay: detach_edge removes view-only absorbed-successor edges.

Edges re-attached by absorb_successor_into() live only in _extra_full_edges
and have no shared-graph counterpart, so detach_edge() silently ignored
them. This causes last-resort refinement in Phoenix to not be able to virtualized
such an edge and believe the edge was virtualized, causing forever loops.

* Phoenix: Guard last-resort refinement against no-op edge virtualization.
2026-07-06 22:38:56 -07:00
Fish
82b2cb9c39
Phoenix: Fix switch-case structuring absorbing heads and more. (#6576)
* Phoenix: Fix switch-case structuring absorbing heads and more.

- _switch_build_cases: when a jump table entry targets the switch head
  itself, emit a goto-case instead of incorrectly absorbing the head node.

- _match_acyclic_switch_cases_address_loaded_from_memory: bail when the
  jump table dispatch node has predecessors other than the switch head;
  absorbing it would disconnect those predecessors (jump table dispatch
  with multiple head nodes).

- Bail when node_a contains an already-structured SwitchCaseNode anywhere
  in its sequence, not just as the last node.

- Remove the switch head from switch_case_known_heads on permanent bails
  (dispatch already structured into a switch-case node). Leaving it marked
  prevented the sequence/ITE matchers from ever structuring the head.

- _refine_cyclic_core: select the loop successor before the
  dangling-node check. At the root region (which has no parent to dissolve
  into), exempt the successor from the check and re-attach other dangling
  sink exit nodes behind a sink successor: they stay reachable through the
  goto that their virtualized edge becomes, so no code is lost.

* tests: Add regression tests for switch-case structuring crashes.

* Lint code.
2026-07-06 22:37:11 -07:00
Fish
681f0b7802
insert_node: Support IncompleteSwitchCaseNode. (#6575) 2026-07-06 22:35:31 -07:00
Fish
568666c1bb
SSA/VRA: Handle ComboRegs. (#6574)
* Clinic: do not create SimComboRegisterVariable for location-less struct args.

* Clinic: actually populate reg_vvars for combo-register argument vvars.

* Ssailification: support combo-register arguments in the initial rewriting state.

* Clinic: rewrite combo-register argument references before variable recovery.

* Variable recovery: handle combo-register parameter vvars in assignments.

* Tests: decompilation of by-value struct arguments passed in registers.
2026-07-06 22:35:13 -07:00
dependabot[bot]
355d1ff77a
ci: bump astral-sh/setup-uv from 8.2.0 to 8.3.0 (#6571)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.2.0 to 8.3.0.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](fac544c07d...d31148d669)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-06 17:33:31 -07:00
dependabot[bot]
bb85e2318c
ci: bump taiki-e/install-action from 2.82.6 to 2.82.9 (#6570)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.82.6 to 2.82.9.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](9bcaee1dca...4684b84056)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.82.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-06 17:33:16 -07:00
Fish
a78382a0cc
ExprNarrowing: Walks each statement at most once to extract effective sizes. (#6573) 2026-07-06 15:40:37 -07:00
Fish
4d526287d5
Fix cache misses in DirectedGraphHelper. (#6569)
* Fix cache misses in DirectedGraphHelper.

- Fix incomplete cache generation after cyclic refinement.

- Fix cache update errors in Phoenix.

- Warn and re-generate the cache if a miss is detected.

* Add test cases.

* Lint code.

* Type check.
2026-07-06 14:55:38 -07:00
Fish
af48f5b28d
LoweredSwitchSimplifier: Do not visit visited nodes. (#6572)
* LoweredSwitchSimplifier: Do not visit visited nodes.

* Type check.
2026-07-06 14:54:15 -07:00
Fish
97f25df24b
CCodeGen: Make pseudocode indentation width adjustable. (#6568) 2026-07-03 12:59:54 -07:00
Ati Priya
1a40a2e720
TypeConstant: Replace by IDs (#6555) 2026-07-03 12:59:22 -07:00
Fish
13e07a3472
CCodeGen: Only display used structs. (#6567) 2026-07-02 19:14:54 -07:00
Fish
869d91604b
TypeTranslator: Structurally canonicalize inferred structs. (#6565) 2026-07-02 11:37:16 -07:00
Fish
b403253296
Fix Win64 stack args recovery; preserve arg regs across __chkstk. (#6564)
* Fix Win64 stack args recovery; preserve arg regs across __chkstk.

* Check for existence of func addr.

* Type check.
2026-07-02 01:45:35 -07:00
Fish
0707412e53
CCodeGen: Decay array-typed operands to pointers; avoid &array. (#6563) 2026-07-01 21:59:14 -07:00
Ati Priya
70eccd64d4
Decompiler: fix OptimizedDivisionSimplifier (#6559)
* decompiler: fix OptimizedDivisionSimplifier returning an over-wide replacement

optimize()'s keep-size guard only narrowed (r.bits < expr.bits); a wider
replacement -- a 64-bit Div for the 32-bit Convert of gcc's magic-multiply
x/k / x%k idiom -- was returned unchanged and tripped 'assert expr.bits ==
r.bits' in the peephole walker, leaving empty codegen. Reconcile widths in both
directions. Adds a regression test (divisors 5, 3, 10000) to TestPeepholeOptimizations.

* lint
2026-07-01 21:27:06 -07:00
Fish
24dd62cd81
RtDb: Reopen dbs after increasing map sizes. (#6562)
@rhelmot managed to trigger "Database handle belongs to another
environment" in nixOS + CPython 3.13.13. This commit attempts to fix
this problem.
2026-07-01 19:33:54 -07:00
Fish
95ac7f14ee
WinStackCanarySimplifier: Support cases when fail to remove rbp. (#6561) 2026-07-01 16:32:51 -07:00
Fish
7879452057
Decompiler: Make semvar naming less aggressive. (#6558) 2026-07-01 13:34:23 -07:00
Fish
eb5aff20ed
SRDAView: Make observe() aware of call-clobbered regs. (#6560)
Also add missing CALLEE_SAVED_REGS to SimCCMicrosoftAMD64.
2026-07-01 13:27:10 -07:00
Fish
a152083983
Clinic: Discover vvars for stack args to avoid over-elimination. (#6557)
* Clinic: Discover vvars for stack args to avoid over-elimination.

* Lint code.
2026-06-30 13:33:21 -07:00
Fish
d263ea9cdc
RustDec: Refactor tests. (#6553)
* RustDec: Refactor tests.

* Fix a test case.
2026-06-29 17:58:53 -07:00
Fish
3461107c1e
PeepholeOpt: (A - C1) cmp C2 => A cmp (C1+C2) for CmpEQ/CmpNE. (#6554)
* PeepholeOpt: (A - C1) cmp C2 => A cmp (C1+C2) for CmpEQ/CmpNE.

* Type check.
2026-06-29 17:58:32 -07:00
Kevin Phoenix
621500031c
Upgrade rust toolchain to 1.96 (#6552) 2026-06-29 17:13:42 -07:00
Fish
e2b9d771dd
Decompiler: Unify hex-display rules. (#6547)
* Decompiler: Unify hex-display rules.

* Lint code.

* Update a test case.
2026-06-29 16:55:24 -07:00
Fish
6f0afb39a3
Reduce TestCFGFastPEMsvcEH to one test case. (#6551) 2026-06-29 16:54:30 -07:00
Fish
d5515757b5
VEXConverter: Handle int3 and int 0x2c. (#6549) 2026-06-29 16:30:25 -07:00
Fish
03d0aac8bb
WinStackCanarySimplifier: Support non-consecutive security-cookie init stmts. (#6548) 2026-06-29 16:16:09 -07:00
Fish
97f26eb601
CFGFast: Propagate func info to jump thunks. Clinic: Fix missing args for cfguard-indirect calls. (#6544) 2026-06-29 15:45:42 -07:00
Fish
7f5371f337
CallingConventions._classify: Return INTEGER for opaque C++ classes. (#6546) 2026-06-29 15:45:12 -07:00
Fish
fd3925649d
AILSimplifier: Eliminate dead VEX ccall cycles. (#6545)
Extend elimination to tmp vvars because ccall destination vvars are not
always simplified to registers.
2026-06-29 15:19:06 -07:00
pre-commit-ci[bot]
44bb07f4e0
[pre-commit.ci] pre-commit autoupdate (#6543)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.18 → v0.15.20](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.18...v0.15.20)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-29 11:22:29 -07:00
dependabot[bot]
4d75178116
ci: bump actions-rust-lang/setup-rust-toolchain from 1.16.1 to 1.17.0 (#6540)
Bumps [actions-rust-lang/setup-rust-toolchain](https://github.com/actions-rust-lang/setup-rust-toolchain) from 1.16.1 to 1.17.0.
- [Release notes](https://github.com/actions-rust-lang/setup-rust-toolchain/releases)
- [Changelog](https://github.com/actions-rust-lang/setup-rust-toolchain/blob/main/CHANGELOG.md)
- [Commits](46268bd060...166cdcfd11)

---
updated-dependencies:
- dependency-name: actions-rust-lang/setup-rust-toolchain
  dependency-version: 1.17.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-29 10:41:27 -07:00
dependabot[bot]
b57fe2d85e
ci: bump taiki-e/install-action from 2.82.2 to 2.82.6 (#6542)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.82.2 to 2.82.6.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](9e1e5806d4...9bcaee1dca)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.82.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-29 09:54:18 -07:00
dependabot[bot]
989d583fc9
ci: bump actions/setup-python from 6.2.0 to 6.3.0 (#6541)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6.2.0 to 6.3.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](a309ff8b42...ece7cb06ca)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-version: 6.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-29 09:54:09 -07:00
Vedant Soni
ca61720a0f
Block: _using_pcode_engine returns false when project is none (#6538) 2026-06-25 14:06:54 -07:00
Kevin Phoenix
b61879f749
angr/decompiler: rewrite shifted-equality back to a masked compare (#6534)
* angr/decompiler: rewrite shifted-equality back to a masked compare

Add a CmpMaskedShift peephole that rewrites an equality comparison against
a right-shifted value back into a masked comparison:

    (x >> n) == c                ==>  (x & mask) == (c << n)
    Convert(N->M, x >> n) == c   ==>  (x & mask) == (c << n)

where ``mask`` selects the compared bits ``[n, n+width-1]`` of ``x``. This
is the inverse of a simplification that can turn a masked compare that
clears the low bits into the shift/extract form; the masked compare is the
clearer form in decompiled output (e.g. ``(*p & 0xfffffff0) == 0x184d2a50``
for an aligned-pointer / bit-field check), so restore it.

The rewrite is value-equivalent. Only logical right shifts and unsigned
truncating Converts are handled, only ``n > 0`` (an ``n == 0`` low-mask
already reads as a cast), and only when the compared constant fits the
compared width.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-23 17:47:45 -07:00
dependabot[bot]
8673087edf
ci: bump actions/cache from 5 to 6 (#6535)
Bumps [actions/cache](https://github.com/actions/cache) from 5 to 6.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-23 16:41:54 -07:00
Kevin Phoenix
75ea02d733
angr/rust: recover if-let from truthiness-style enum discriminant checks (#6508)
* angr/rust: recover if-let from truthiness-style enum discriminant checks

extract_scrutinee_and_discriminant only recognized CmpEQ/CmpNE and sign-bit
discriminant forms. Compilers also emit an enum discriminant test as a
boolean-truncating conversion: Conv(N->1, disc) (rendered "if result as i8")
meaning disc != 0, and Not(Conv(N->1, disc)) meaning disc == 0. In that form
the outer Result's Ok/Err check rendered as "if !(v as i8)" instead of
"if let Ok(...)". Add truthiness handling; both call sites already gate on an
Option/Result-typed scrutinee.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* tests: cover truthiness-style discriminant extraction

Unit tests for PrePatternMatchSimplifier.extract_scrutinee_and_discriminant
on the boolean-truncating-conversion discriminant forms: `Conv(N->1, disc)`
(disc != 0) and `Not(Conv(N->1, disc))` (disc == 0), including the
LogicalAnd leftover passthrough and a negative case ensuring a wider
truncation is not mistaken for a truthiness test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 16:40:42 -07:00
Kevin Phoenix
f489587fbf
tests: compare RDA definitions order-insensitively (#6510)
* tests: compare RDA definitions order-insensitively

The order in which definitions are extracted follows AST annotation
iteration order, an implementation detail that is not guaranteed stable.
Compare the per-address definition lists order-insensitively (assertCountEqual)
instead of relying on list order, walking the result structure explicitly so a
mismatch points at the offending observation/address.

Depends on the hash-recompute-on-unpickle fix: the pickled groundtruth caches
stale per-process hashes, so a hash-based comparison only holds once those are
recomputed on load.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-23 16:38:56 -07:00
angr-bot
b221043a64 Update version to 9.2.223.dev0 [ci skip] 2026-06-23 23:21:50 +00:00
Fish
8c0fa9f915
Move pure data to angr-data. (#6530)
* Move pure data to angr-data.

* Update the angr-data versioning scheme.

* Adjust rust typedb path.

* Add a dependabot task for angr-data.
2026-06-23 16:00:58 -07:00
Fish
8309c06858
Phoenix: Faster unstructured jumptable heads and dispatch nodes check. (#6517)
* Phoenix: Faster unstructured jumptable heads and dispatch nodes check.

* Oops.

* Clear the switch-case cache.
2026-06-23 15:34:05 -07:00
Kevin Phoenix
fbcb6f3fbc
angr: wrap strided-interval annotation bounds to unsigned at construction (#6531) 2026-06-23 15:06:44 -07:00
Kevin Phoenix
ee57104468
angr/decompiler: Implement __eq__ and __hash__ for AILExprIdAnnotation (#6532) 2026-06-23 15:04:24 -07:00
Vedant Soni
8cdf310ac7
handle rust_demangler unabletodemangle errors (#6512) 2026-06-23 11:35:57 -07:00
Michał Kowalczyk
1f0d6fa182
docs: Fix a python snippet and reST formatting (#6527) 2026-06-23 11:35:06 -07:00
Fish
400a2c0bf5
Ssailification: Delta-merge COW maps over unshared layers. (#6528) 2026-06-22 19:47:52 -07:00
Kevin Phoenix
ed0f07b872
angr/decompiler: handle Extract(hi, lo, x) with lo != 0 in condition processor (#6507)
_convert_extract only handled lo == 0, raising NotImplementedError
otherwise ("will be implemented once encountered"). Translate the general
case to a logical right shift by lo followed by a truncating Convert (the
lo == 0 path is the no-shift special case), so condition processing no
longer fails on Extract conditions whose low bit is nonzero.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 18:09:13 -07:00
Fish
f5d29a79b5
Supergraph: Contract super-graphs using a worklist. (#6525) 2026-06-22 15:06:13 -07:00
Fish
6646a7fc07
Phoenix: Reduce node sorting and graph traversal. (#6514)
* Phoenix: Reduce node sorting and graph traversal.

- Introduce a DirectedGraphHelper to keep track of node order and
post-order graph traversal sequence when graphs change. This way we
significantly reduce the number of calls to
RegionOverlayGraph.successors().

- Fix an issue that leads to pre-maturely structuring a switch-case.

* Adjust the location where DGH is initialized.

* Lint and type checking.

* Oops.

* More fixes.

* Lint code.

* Fix linked list node replacement.

* Fix incorrect natural loop creation rules.

* DGH: Add a resilience.

* Update logic.
2026-06-22 13:49:20 -07:00
Fish
c37fece6c0
SRDAView.observe: Use graph dominance (#6521) 2026-06-22 13:48:36 -07:00
Fish
e6d1c54897
Make structuring use no-copy region-overlay graph views (#6520) 2026-06-22 11:29:36 -07:00
pre-commit-ci[bot]
ed32d4e4d9
[pre-commit.ci] pre-commit autoupdate (#6524)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.17 → v0.15.18](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.17...v0.15.18)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-22 10:40:26 -07:00
dependabot[bot]
6837318a8a
ci: bump actions/checkout from 6.0.3 to 7.0.0 (#6522)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.3 to 7.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](df4cb1c069...9c091bb21b)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-22 10:18:40 -07:00
dependabot[bot]
0a4a3dc2ff
ci: bump taiki-e/install-action from 2.81.10 to 2.82.2 (#6523)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.81.10 to 2.82.2.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](7a79fe8c3a...9e1e5806d4)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.82.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-22 10:18:11 -07:00
Fish
a8cb8ee063
ReturnDupLow: Reduce max-func-blocks to 300. (#6519) 2026-06-22 01:23:32 -07:00
Fish
7eb405ffe2
Phoenix: Avoid has_path() when determining loop body nodes. (#6515) 2026-06-22 00:52:50 -07:00
Kevin Phoenix
63c05f1d41
SlottedMemory: Avoid unnecessary Extract/Concat in Load/Store ops (#6513) 2026-06-18 13:56:10 -07:00
Kevin Phoenix
1b5ef20505
knowledge_plugins, code_location: recompute hashes after unpickling (#6511)
* knowledge_plugins, code_location: recompute hashes after unpickling

Atom, Definition, and CodeLocation cache their hash in a `_hash` slot. The
default pickling persists that slot, but the hash folds in per-process-salted
hashes (e.g. of register-name strings), so a value pickled in one process is
stale when unpickled in another -- equal objects then hash differently, which
breaks sets/dicts (and assertCountEqual) rebuilt from a pickle.

Add `__getstate__`/`__setstate__` that drop `_hash` from the pickled state and
reset it to None on load, so it is recomputed lazily in-process. `__dict__`
and all slots are otherwise preserved, and the existing default-format
pickles still load.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Disable protected-member for hash checks

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 12:06:45 -07:00
Kevin Phoenix
d4a3f058e0
tests: sum floats sequentially in run_manyfloatsum_symbolic (#6509)
Since Python 3.12, builtin sum() uses Neumaier compensated summation for
floats, which is more accurate than the program's naive left-to-right
addition. The test constrains the program's sequential FP sum to equal 27.7
exactly, then asserted sum(args) == 27.7 — which additionally demands that
the model have no intermediate rounding error under compensated summation,
something the constraint does not guarantee. Sum with functools.reduce to
mirror the program's sequential addition.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 17:25:37 -07:00
Fish
6e835f930e
CFGFast: Repair guessed jumptable targets; remove overlapped ones. (#6503)
* CFGFast: Repair guessed jumptable targets; remove overlapped ones.

- Track if jumptable targets are guessed or not.
- For guessed jumptable targets, stop at the next referenced data
location or the next jumptable address.

* Increase the default limits.
2026-06-16 03:33:30 -07:00
Fish
55aca77f4e
LightEngine: Fix an off-by-one error in matching handlers. (#6504)
* LightEngine: Fix an off-by-one error in matching handlers.

* Lint code.
2026-06-16 03:33:11 -07:00
Fish
fd35f3b23e
Clinic: Skip diamond rewriting if there are too many cases. (#6505)
This is critical so that we do not rewrite thousands of cmovnz
instructions into diamond shapes in potentially adversarial functions.
2026-06-16 03:32:25 -07:00
Fish
735049283a
Structuring: Introduce RegionOverlay and RegionOverlayGraph. (#6492) 2026-06-16 01:25:58 -07:00
Fish
608da273df
RegionIdentifier: Force default node into loop body if the loop head is also a switch comp node. (#6493)
Test binary:
cb30d69b24245bf2ecdc9e7f53bbad19159999970b6d82c0c00c7d32d9e37aa4 (ls),
test function: 0x414cb0, switch-case addr: 0x414d33.
2026-06-15 18:52:18 -07:00
Fish
95d09fc7cd
SPropagator: Fuse VVar/Tmp uses&defs collection into a single pass. (#6501)
* SPropagator: Fuse VVar/Tmp uses&defs collection into a single pass.

* Type check.
2026-06-15 17:13:48 -07:00
Fish
2c6e2ceb0f
SRDAView: Speed up observe(). (#6502)
* SRDAView: Speed up observe().

- Cache the quasi-topological traversal order for function graphs.

- Avoid deep copies that are unnecessary.

* Type check.
2026-06-15 17:13:33 -07:00
Fish
f05998cc75
Decompiler: Use a singleton walker for _has_call_exprs. (#6500) 2026-06-15 15:04:14 -07:00
Fish
dcd5841017
SSA Traversal: Cache cowdict lookup in merge(). (#6499) 2026-06-15 15:03:48 -07:00
pre-commit-ci[bot]
b4e90bbe13
[pre-commit.ci] pre-commit autoupdate (#6498)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.16 → v0.15.17](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.16...v0.15.17)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-15 12:48:08 -07:00
dependabot[bot]
0f7a37a859
ci: bump EnricoMi/publish-unit-test-result-action from 2.23.0 to 2.24.0 (#6495)
Bumps [EnricoMi/publish-unit-test-result-action](https://github.com/enricomi/publish-unit-test-result-action) from 2.23.0 to 2.24.0.
- [Release notes](https://github.com/enricomi/publish-unit-test-result-action/releases)
- [Commits](c950f6fb44...d0a4676d0e)

---
updated-dependencies:
- dependency-name: EnricoMi/publish-unit-test-result-action
  dependency-version: 2.24.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-15 10:00:58 -07:00
dependabot[bot]
ac641bf34c
ci: bump taiki-e/install-action from 2.81.8 to 2.81.10 (#6496)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.81.8 to 2.81.10.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](0631aa6515...7a79fe8c3a)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.81.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-15 10:00:41 -07:00
dependabot[bot]
2e2a62b277
rust: bump pyo3 from 0.28.3 to 0.29.0 (#6494)
Bumps [pyo3](https://github.com/pyo3/pyo3) from 0.28.3 to 0.29.0.
- [Release notes](https://github.com/pyo3/pyo3/releases)
- [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pyo3/pyo3/compare/v0.28.3...v0.29.0)

---
updated-dependencies:
- dependency-name: pyo3
  dependency-version: 0.29.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-12 13:19:44 -07:00
Fish
9b35f7610a
AIL: Track Let.variant and Macro.returnty in VariableMap (#6490)
* AIL: Track Let.variant and Macro.returnty in VariableMap

* Lint and type check.
2026-06-11 18:24:02 -07:00
Fish
b9382a3ff3
SimCCMicrosoftFastcall: Implement next_arg. (#6491) 2026-06-10 03:19:34 -07:00
Kevin Phoenix
4cc8874b71
Fix misspelled relocatable property on Annotation subclasses (#6486)
claripy's Annotation property is `relocatable`; MultiwriteAnnotation and
AILExprIdAnnotation both define `relocateable`, a dead override, so the
base-class default (relocatable=False) silently applied.

For MultiwriteAnnotation (intended relocatable=True) this means the
annotation is dropped whenever an annotated address AST is simplified,
so the multiwrite concretization filter loses its tag (fgets/gets/
memchr/strchr annotate end addresses with it). For AILExprIdAnnotation
(intended relocatable=False) the intended value matches the default, so
there is no behavior change; the override just becomes effective.
2026-06-09 16:13:40 -07:00
Fish
2ea863abc8
AILSimplifier: incrementally update SRDA in dead-assignment removal (#6488)
* AILSimplifier: incrementally update reaching definitions in dead-assignment removal

* Lint and type check.
2026-06-09 16:13:01 -07:00
Fish
0b45791efc
AILSimplifier: Don't fold calls with write side effects. (#6487)
_fold_call_exprs folds a call whose return value is used exactly once
into that use site and removes the original call statement. This is
incorrect when the call may write via a pointer argument: when they have
"extra_defs" operands, e.g. a call that fills a stack buffer. Folding
moves the call and its side-effect write to the return-value use site,
leaving every other read of the written-through vvar referencing a value
that is now never written.
2026-06-09 14:53:46 -07:00
Fish
2985fffc55
MultiNode: Cache the hash. (#6485) 2026-06-09 03:45:58 -07:00
Fish
a544e59a63
VariableMap: Track Call prototype and calling_convention. (#6482)
* VariableMap: Track Call prototype and calling_convention.

* Lint code.

* Fix two _handle_stmt_SideEffectStatement.
2026-06-09 02:59:11 -07:00
Fish
0158efaf49
AILSimplifier: Expose changed blocks. (#6484)
Also only simplify changed blocks in BlockSimplifier, reducing useless graph and block traversals.
2026-06-09 02:17:41 -07:00
pre-commit-ci[bot]
e0d145d273
[pre-commit.ci] pre-commit autoupdate (#6480)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.15 → v0.15.16](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.15...v0.15.16)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-08 11:46:34 -07:00
Fish
952cd78c35
Analysis: Skip attribute extraction without telemetry. (#6479)
The AnalysisFactory wrapper that runs on every analysis instantiation
bound the call signature and walked all arguments to record them as
OpenTelemetry span attributes. This is pure overhead when no spans
are being recorded.

Also compute AnalysisFactory.__sig lazily.
2026-06-08 11:07:49 -07:00
dependabot[bot]
4f9c9a41ae
ci: bump taiki-e/install-action from 2.81.2 to 2.81.8 (#6478)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.81.2 to 2.81.8.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](6887963ccf...0631aa6515)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.81.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-08 10:46:52 -07:00
dependabot[bot]
579601e2d1
ci: bump codecov/codecov-action from 6.0.1 to 7.0.0 (#6476)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 6.0.1 to 7.0.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](e79a6962e0...fb8b3582c8)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-08 09:33:00 -07:00
dependabot[bot]
b63cd08005
ci: bump astral-sh/setup-uv from 8.1.0 to 8.2.0 (#6477)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.1.0 to 8.2.0.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](08807647e7...fac544c07d)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-08 09:32:45 -07:00
Fish
84da18b58c
Make AILBlockWalker construction cheaper. (#6475)
* Make AILBlockWalker construction cheaper.

* Rename _handle_CallExpr to _handle_Call.

* Lint code.

* Tweak
2026-06-08 01:20:55 -07:00
Fish
41167d1ce4
BlockSimplifier: Reuse the peephole expr walker. (#6473) 2026-06-08 01:20:35 -07:00
Fish
3508180d9d
SPropogator: Reduce the number of AILBlockWalker instances. (#6474)
* SPropogator: Reduce the number of AILBlockWalker instances.

Also introduced AILWhitelistExprTypeWalker and a Decompiler-specific TLS
cache for walker instances that are used repeatedly during
decompilation.

* Lint decompiler.py.
2026-06-08 00:35:29 -07:00
Fish
14f0650d3b
Decompiler: Use VariableMap to track Atom-Variable mapping. (#6470) 2026-06-05 16:28:58 -07:00
Fish
c6e07071d0
ConstPropReverter: Run RDA on the AIL graph for speed. (#6472) 2026-06-05 03:07:38 -07:00
Fish
e439fd5eb6
SSA Traversal: Fix a bug where extern reg defs may be updated to offsets of stack defs. (#6471) 2026-06-05 01:18:46 -07:00
Kevin Phoenix
01bb4c1375
Use abi3 for wheels (#6469) 2026-06-04 12:11:06 -07:00
Kevin Phoenix
6f20ff69c5
Fix many delayed imports (#6458)
* Fix many delayed imports

* Add import paths for readability
2026-06-04 11:33:16 -07:00
Brian Caswell
963be2bd00
spiller_db: replace deprecated datetime.utcnow() default (#6468)
`datetime.datetime.utcnow` is deprecated in Python 3.12+ and emits
`DeprecationWarning` each time it's called. SQLAlchemy invokes it on
every `PickledState` insert when the `Spiller` exploration technique
is in use, so any spiller run with sqlalchemy installed accumulates
warnings (and crashes under `-W error::DeprecationWarning` /
pytest `filterwarnings = ["error"]`).

Switch to a timezone-aware `datetime.datetime.now(datetime.UTC)`
callable. `datetime.UTC` is available unconditionally because the
project requires Python >= 3.12 (pyproject.toml).

Co-authored-by: demoray <demoray@users.noreply.github.com>
2026-06-04 10:41:55 -07:00
Brian Caswell
034265e82f
JumpTableResolver: stop using deprecated BP(condition=...) (#6467)
`BP.__init__` deprecates the `condition=` kwarg, but
`JumpTableResolver._instrument_statements` was still passing it at five
call sites. Every `CFGFast` run that hit a jumptable emitted a
`DeprecationWarning`, and consumers that promote that warning to an
error (pytest `filterwarnings = ["error"]`, `-W error::DeprecationWarning`)
crashed on the first jumptable.

Wrap each hook in a small gating closure that checks the same
scratch/statement predicate the old `condition=` lambda checked, and
drop `condition=` from all five `BP(...)` calls. Behavior is unchanged:
per-function decompilation of `tests/x86_64/cfg_switches` is
byte-identical before and after, and `tests/analyses/cfg/test_jumptables.py`
still passes (25 tests, 24 subtests).

Fixes #6466.

Co-authored-by: demoray <demoray@users.noreply.github.com>
2026-06-04 09:52:09 -07:00
Kevin Phoenix
7fd4ac3bf9
docs: Auto-generate the API reference via autosummary (#6460)
Replace the hand-maintained ``api.rst`` listing with a single
``.. autosummary:: :recursive:`` entry point, and add an autosummary
template under ``_templates/autosummary/module.rst`` that recursively
emits a page per submodule. The output directory ``docs/api/`` is
gitignored.

The template uses ``.. rubric:: Submodules`` instead of a section
heading so the autogenerated pages don't all register a duplicate
"Submodules" autosectionlabel.

Adjustments to ``conf.py`` to make the auto-generated pages readable:

- ``autodoc_class_signature = "separated"`` so the class declaration line
  does not duplicate the ``__init__`` parameter list.
- ``skip_inherited_undocumented`` (autodoc-skip-member): when a subclass
  overrides a method without its own docstring, do not document the
  override on the subclass page; the parent's "Bases:" link already
  points readers to the canonical entry. Avoids duplicating the parent's
  full docstring on every subclass.
- ``_patch_directive_header_for_reexports`` (monkey-patch of
  ``_directive_header_lines``): for re-exported symbols (e.g.
  ``angr.SimState`` re-exporting ``angr.sim_state.SimState``), inject
  ``:no-index:`` so the entry still renders on the parent page but the
  canonical defining-module page is the unambiguous cross-reference
  target.
2026-06-04 08:32:27 -07:00
Fish
3f600adc46
ConstPropReverter: Remove O(N^2) graph traversal. (#6465)
* ConstPropReverter: Remove O(N^2) graph traversal.

* Fixes.
2026-06-03 22:24:43 -07:00
angr-bot
06b287bbc8 Update version to 9.2.222.dev0 [ci skip] 2026-06-03 23:35:59 +00:00
Fish
cd2e5cb89e
CCodeGen: Fix assertion error when MemoryData.content is empty. (#6464) 2026-06-03 15:09:59 -07:00
Fish
0b439f05d4
SPropagator: Fix two cases where Expression.idx is None. (#6463)
Also make SPropagator._ail_manager non-nullable.
2026-06-03 14:55:25 -07:00
Fish
8d87fcf60b
SPropagator: Fix bad propagation for vvars defined in Assignment.src. (#6462) 2026-06-03 14:55:03 -07:00
Brian Caswell
201c746605
typehoon: stabilize two more order-sensitive set iterations in SimpleSolver (#6439)
Two additional places in simple_solver.py where iteration over a set
leaks process-global TypeVariable.idx values into the solver's output.
Follow-up to #6438.

1. Equivalence-class representative selection (~line 1092): next(iter(...))
   over a set of typevars picks an arbitrary element which becomes the
   equivalence class representative and a dict key downstream. Sort
   candidates by repr.

2. determine(...) over remaining sketches (~line 776): determine iterates
   its tvs argument and mutates self.solution per typevar; the post-loop
   fixup propagates equivalence-class solutions back, so resolution order
   can affect the final solution. Sort the pending set before passing.
2026-06-03 14:40:05 -07:00
Brian Caswell
ca0b66868b
typehoon: sort primitive constraints before folding into sketches (#6438)
Iteration order over the primitive_constraints set depends on
TypeVariable.idx, which is assigned from a process-global monotonic
counter. The same logical typevars get different idx values on each
decompilation in the same process, reshuffling set iteration order.
Sketch.add_constraint folds bounds via join/meet, which is not
order-invariant when alternative types collide on the same node, so
decompiler output for the same function can vary across runs.

Sort by repr before iterating. Within a single solver invocation all
typevars share the same global idx baseline, so relative repr ordering
is stable.
2026-06-03 14:38:29 -07:00
Fish
2c8a510608
CFGFast: Fix repeated string creation attempts. (#6461)
* CFGFast: Fix repeated string creation attempts.

* Increase the default thresholds.
2026-06-03 13:25:09 -07:00
Ahmet Çelikbaş
9236dc51d1
LRU cache eviction addr_type mismatch fix & better LMDB packing strategy (#6435)
* LRU cache eviction addr_type mismatch fix & better LMDB packing strategy
2026-06-03 12:32:29 -07:00
Kevin Phoenix
93ae89296d
docs: Disambiguate narrative cross-references (#6459)
Turn on ``autosectionlabel_prefix_document = True`` so that auto-generated
section labels are prefixed with the document path. This silences the
duplicate-label warnings between narrative pages where the same H1 text
appears in more than one file (e.g. "State Plugins" in
``core-concepts/states.rst`` and ``extending-angr/state_plugins.rst``,
"Core Concepts" in ``core-concepts/index.rst`` and
``core-concepts/toplevel.rst``).

Update the existing bare-label ``:ref:`` links to use the prefixed form
(e.g. ``:ref:`text <core-concepts/loading:Loading a Binary>```), and at
the same time collapse the line-wrapped links that were already brittle.
2026-06-03 12:19:46 -07:00
Fish
0377eca5a6
SPTracker: Improve memory usage. (#6397) 2026-06-03 11:32:56 -07:00
Kevin Phoenix
5b027eec45
Fix all bottom imports (#6457)
* Fix all bottom imports

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-03 10:00:42 -07:00
dependabot[bot]
4e0f720d46
ci: bump actions/checkout from 6.0.2 to 6.0.3 (#6456)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.2 to 6.0.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](de0fac2e45...df4cb1c069)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-02 17:21:58 -07:00
dependabot[bot]
9341f998e7
ci: bump taiki-e/install-action from 2.79.7 to 2.81.2 (#6455)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.79.7 to 2.81.2.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](d9be7d8cda...6887963ccf)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.81.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-02 17:21:03 -07:00
Kevin Phoenix
f939c5b88c
Enable ruff isort rule (#6452)
* Enable ruff isort rule

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-02 14:48:07 -07:00
Vedant Soni
42bb296012
render syscallnode as hook (#6453) 2026-06-02 11:38:07 -07:00
Ati Priya
785459c81f
remove name collisions in semantic naming (#6451)
* remove name collisions

* lint
2026-06-02 11:37:45 -07:00
Michał Kowalczyk
134d36fa19
docs: Fix a broken link in loading.rst (#6454) 2026-06-02 11:35:30 -07:00
angr-bot
81861b79c7 Update version to 9.2.221.dev0 [ci skip] 2026-06-01 19:16:00 +00:00
pre-commit-ci[bot]
df1345ae3d
[pre-commit.ci] pre-commit autoupdate (#6450)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.14 → v0.15.15](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.14...v0.15.15)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-01 11:22:40 -07:00
Fish
6ebff1115f
TestDecompiler: Do not force recover variables. (#6449)
* TestDecompiler: Do not force recover variables.

* Tweak
2026-05-31 23:17:37 -07:00
Fish
1708a9e1d5
GraphVVarMapping: Fix a duplicated vvar assignment. (#6448) 2026-05-31 20:27:40 -07:00
Kevin Phoenix
438820c528
Normalize InspectMixin naming (#6447)
* Normalize InspectMixin naming

* Add docstring

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-05-29 14:00:19 -07:00
Fish
bc92ec1061
Clinic: Implement type joining in callee prototype constraining. (#6446)
* Clinic: Implement type joining in callee prototype constraining.

Also,

- Make callee prototype constraining disabled by default.

- Refactor simple_solver.py so that join and meet are usable without
creating a SimpleSolver instance first.

* Update a test case.
2026-05-28 19:04:35 -07:00
Kevin Phoenix
6ac26cac27
Refactor AIL symbolic execution to store block indexes in state.scratch (#6443)
* Refactor AIL sybolic execution to store block indexes in state.scratch

* Verify states are ail-mode

* Add magic to addr.setter to handle ail address tuples

* Add docs
2026-05-28 18:24:47 -07:00
Audrey Dutcher
f381fa7dc3
AIL symexec: allow the use of a custom callstack class (#6445) 2026-05-28 17:26:14 -07:00
Fish
2f3d35f464
Typehoon: Make tv counter per-function. (#6444)
* Typehoon: Make tv counter per-function.

* Pass TypeVariableManager to VR states.

* Lint and bug fix.
2026-05-27 19:33:55 -07:00
Kevin Phoenix
73e96610da
Remove unused typevars (#6442) 2026-05-27 13:04:27 -07:00
Kevin Phoenix
e609b871af
Remove unimplemented prepare_callsite stub (#6441) 2026-05-27 12:10:48 -07:00
angr-bot
c85cffd3cf Update version to 9.2.220.dev0 [ci skip] 2026-05-26 18:22:48 +00:00
1854 changed files with 58475 additions and 122876 deletions

View file

@ -12,3 +12,11 @@ updates:
interval: "weekly"
commit-message:
prefix: "rust"
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
commit-message:
prefix: "deps"
allow:
- dependency-name: "angr-data"

View file

@ -16,20 +16,26 @@ jobs:
ci:
uses: angr/ci-settings/.github/workflows/angr-ci.yml@master
smoketest:
name: Test installation
installation:
name: Test installation (${{ matrix.environment.os }}, py${{ matrix.environment.python-version }})
strategy:
matrix:
os: [windows-2022, macos-15-intel]
environment:
- os: windows-2025
python-version: 3.12
- os: macos-26
python-version: 3.12
- os: ubuntu-24.04
python-version: 3.14
fail-fast: false
runs-on: ${{ matrix.os }}
runs-on: ${{ matrix.environment.os }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
- uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1
if: startsWith(runner.os, 'windows')
- uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v6
- uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v6
- name: Sync dependencies
run: uv sync -p 3.12
run: uv sync -p ${{ matrix.environment.python-version }}
- name: Collect tests
run: uv run pytest --collect-only tests
@ -37,8 +43,8 @@ jobs:
name: Rust Check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
- uses: actions-rust-lang/setup-rust-toolchain@46268bd060767258de96ed93c1251119784f2ab6 # v1
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
- uses: actions-rust-lang/setup-rust-toolchain@166cdcfd11aee3cb47222f9ddb555ce30ddb9659 # v1
with:
components: clippy, rustfmt
- run: cargo clippy --all-targets --all-features -- -D warnings
@ -50,9 +56,9 @@ jobs:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
- uses: actions-rust-lang/setup-rust-toolchain@46268bd060767258de96ed93c1251119784f2ab6 # v1
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
- uses: actions-rust-lang/setup-rust-toolchain@166cdcfd11aee3cb47222f9ddb555ce30ddb9659 # v1
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v5
with:
python-version: "3.12"
- run: cargo test --release

View file

@ -23,11 +23,11 @@ jobs:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2
- uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v6
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
- uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v6
- name: Restore test durations cache
uses: actions/cache/restore@v5
uses: actions/cache/restore@v6
with:
path: test_durations.json
key: cov-test-durations-${{ github.sha }}
@ -61,7 +61,7 @@ jobs:
tar -xpf $PWD/env.tzst -C /
rm env.tzst
- name: Download test binaries
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with:
repository: angr/binaries
path: binaries
@ -96,13 +96,13 @@ jobs:
name: Test Rust packages
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2
- uses: actions-rust-lang/setup-rust-toolchain@46268bd060767258de96ed93c1251119784f2ab6 # v1
- uses: taiki-e/install-action@d9be7d8cda89035c9c843f78bd44d4f72d8403d4 # v2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
- uses: actions-rust-lang/setup-rust-toolchain@166cdcfd11aee3cb47222f9ddb555ce30ddb9659 # v1
- uses: taiki-e/install-action@6c6fd71fe4fb72c3697d269963d0e15df8adedad # v2
with:
tool: cargo-llvm-cov
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v5
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v5
with:
python-version: "3.12"
- name: Run tests
@ -160,13 +160,13 @@ jobs:
junit=$(find . -type f -name 'junit.xml' | paste -sd, -)
echo "junit=$junit" >> "$GITHUB_OUTPUT"
- name: Upload test coverage to Codecov
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v5
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v5
with:
use_oidc: true
verbose: true
files: ${{ steps.files.outputs.coverage }}
- name: Upload test results to Codecov
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v5
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v5
with:
use_oidc: true
fail_ci_if_error: true
@ -182,12 +182,12 @@ jobs:
path: ./test_durations.json
if-no-files-found: error
- name: Update test durations cache
uses: actions/cache/save@v5
uses: actions/cache/save@v6
with:
path: test_durations.json
key: cov-test-durations-${{ github.sha }}
- name: Publish test results
uses: EnricoMi/publish-unit-test-result-action@c950f6fb443cb5af20a377fd0dfaa78838901040 # v2
uses: EnricoMi/publish-unit-test-result-action@d0a4676d0e0b938bc201470d88276b7c74c712b3 # v2
with:
files: "results/**/junit.xml"
action_fail_on_inconclusive: true

View file

@ -28,15 +28,15 @@ jobs:
runner_id: [1, 2, 3, 4, 5]
fail-fast: false
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with:
path: angr
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with:
repository: angr/binaries
path: binaries
- uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1
- uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v6
- uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v6
- name: Setup Java for pysoot
uses: actions/setup-java@v5
with:
@ -54,14 +54,14 @@ jobs:
runner_id: [1, 2, 3]
fail-fast: false
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with:
path: angr
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with:
repository: angr/binaries
path: binaries
- uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v6
- uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v6
- name: Sync dependencies
run: uv --directory angr sync -p 3.12
- name: Run pytest

4
.gitignore vendored
View file

@ -32,9 +32,11 @@ dist
*.db
*.dylib
docs/_build
docs/api/
uv.lock
target/
.DS_Store
*_angr_rtdb
*_angr_rtdb_*
.claude/
.claude
angr/protos/*_pb2.py

View file

@ -62,7 +62,7 @@ repos:
args: [--py310-plus]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.14
rev: v0.16.1
hooks:
- id: ruff-check
args: [--fix, --exit-non-zero-on-fix]

112
Cargo.lock generated
View file

@ -72,12 +72,16 @@ dependencies = [
"indexmap",
"libafl",
"libafl_bolts",
"num-bigint",
"pcode",
"postcard",
"pyo3",
"rangemap",
"regex",
"rustc-hash",
"send_wrapper",
"serde",
"serde_bytes",
"target-lexicon",
"tempfile",
]
@ -844,6 +848,17 @@ checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9"
dependencies = [
"equivalent",
"hashbrown 0.17.0",
"serde",
"serde_core",
]
[[package]]
name = "inventory"
version = "0.3.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4f0c30c76f2f4ccee3fe55a2435f691ca00c0e4bd87abe4f4a851b1d4dac39b"
dependencies = [
"rustversion",
]
[[package]]
@ -1057,6 +1072,26 @@ dependencies = [
"memoffset",
]
[[package]]
name = "num-bigint"
version = "0.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c89e69e7e0f03bea5ef08013795c25018e101932225a656383bd384495ecc367"
dependencies = [
"num-integer",
"num-traits",
"serde",
]
[[package]]
name = "num-integer"
version = "0.1.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7969661fd2958a5cb096e56c8e1ad0444ac2bbcd0061bd28660485a44879858f"
dependencies = [
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.19"
@ -1222,11 +1257,14 @@ dependencies = [
[[package]]
name = "pyo3"
version = "0.28.3"
version = "0.29.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91fd8e38a3b50ed1167fb981cd6fd60147e091784c427b8f7183a7ee32c31c12"
checksum = "4688ddedf473e32662b9b067670129a8afb8c18e351482c70d62ba4a88171e8b"
dependencies = [
"inventory",
"libc",
"num-bigint",
"num-traits",
"once_cell",
"portable-atomic",
"pyo3-build-config",
@ -1236,18 +1274,18 @@ dependencies = [
[[package]]
name = "pyo3-build-config"
version = "0.28.3"
version = "0.29.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e368e7ddfdeb98c9bca7f8383be1648fd84ab466bf2bc015e94008db6d35611e"
checksum = "f41027e41b4bd03f6e60f9f417fe24a6341a6bb744edd62b6f709f2a52ea30e9"
dependencies = [
"target-lexicon",
]
[[package]]
name = "pyo3-ffi"
version = "0.28.3"
version = "0.29.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f29e10af80b1f7ccaf7f69eace800a03ecd13e883acfacc1e5d0988605f651e"
checksum = "e591a95526fead067432c3b3a33fc74770b87b1e04e73671090d9c2055a2b327"
dependencies = [
"libc",
"pyo3-build-config",
@ -1255,9 +1293,9 @@ dependencies = [
[[package]]
name = "pyo3-macros"
version = "0.28.3"
version = "0.29.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df6e520eff47c45997d2fc7dd8214b25dd1310918bbb2642156ef66a67f29813"
checksum = "73225868fc1cd84eef2c3c230ddb91273bf1de46aeb8a4248da76d32a0924a1c"
dependencies = [
"proc-macro2",
"pyo3-macros-backend",
@ -1267,13 +1305,12 @@ dependencies = [
[[package]]
name = "pyo3-macros-backend"
version = "0.28.3"
version = "0.29.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4cdc218d835738f81c2338f822078af45b4afdf8b2e33cbb5916f108b813acb"
checksum = "571575aa3749fa6216757dd47d2a3e7ef360f329a40f0666a9fbd14889024952"
dependencies = [
"heck",
"proc-macro2",
"pyo3-build-config",
"quote",
"syn 2.0.114",
]
@ -1330,9 +1367,9 @@ dependencies = [
[[package]]
name = "regex"
version = "1.12.2"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843bc0191f75f3e22651ae5f1e72939ab2f72a4bc30fa80a066bd66edefc24d4"
checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d"
dependencies = [
"aho-corasick",
"memchr",
@ -1342,9 +1379,9 @@ dependencies = [
[[package]]
name = "regex-automata"
version = "0.4.13"
version = "0.4.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5276caf25ac86c8d810222b3dbb938e512c55c6831a10f3e6ed1c93b84041f1c"
checksum = "8fcfdb36bda0c880c5931cdc7a2bcdc8ba4556847b9d912bca70bc94708711ad"
dependencies = [
"aho-corasick",
"memchr",
@ -1353,9 +1390,9 @@ dependencies = [
[[package]]
name = "regex-syntax"
version = "0.8.8"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a2d987857b319362043e95f5353c0535c1f58eec5336fdfcf626430af7def58"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "region"
@ -1390,9 +1427,9 @@ checksum = "b50b8869d9fc858ce7266cce0194bd74df58b9d0e3f6df3a9fc8eb470d95c09d"
[[package]]
name = "rustc-hash"
version = "2.1.1"
version = "2.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "357703d41365b4b27c590e3ed91eabb1b663f07c4c084095e60cbed4362dff0d"
checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d"
[[package]]
name = "rustix"
@ -1460,9 +1497,9 @@ checksum = "cd0b0ec5f1c1ca621c432a25813d8d60c88abe6d3e08a3eb9cf37d97a0fe3d73"
[[package]]
name = "serde"
version = "1.0.228"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [
"serde_core",
"serde_derive",
@ -1481,23 +1518,33 @@ dependencies = [
]
[[package]]
name = "serde_core"
version = "1.0.228"
name = "serde_bytes"
version = "0.11.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
checksum = "a5d440709e79d88e51ac01c4b72fc6cb7314017bb7da9eeff678aa94c10e3ea8"
dependencies = [
"serde",
"serde_core",
]
[[package]]
name = "serde_core"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.114",
"syn 3.0.2",
]
[[package]]
@ -1622,6 +1669,17 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a207d6d6a2b7fc470b80443726053f18a2481b7e1eee970597051596567987a3"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "target-lexicon"
version = "0.13.5"

View file

@ -3,4 +3,5 @@ include README.md
include Cargo.toml
include Cargo.lock
graft native
recursive-include angr/procedures/definitions *.json
recursive-include angr/protos *.proto
recursive-exclude angr/protos *_pb2.py

View file

@ -43,11 +43,13 @@ import angr
project = angr.Project("angr-doc/examples/defcamp_r100/r100", auto_load_libs=False)
@project.hook(0x400844)
def print_flag(state):
print("FLAG SHOULD BE:", state.posix.dumps(0))
project.terminate_execution()
project.execute()
```

View file

@ -1,8 +1,7 @@
# pylint: disable=wildcard-import
# pylint: disable=wrong-import-position
from __future__ import annotations
__version__ = "9.2.219"
__version__ = "9.3.3.dev0"
if bytes is str:
raise Exception("""
@ -20,6 +19,7 @@ For more information, see here: https://docs.angr.io/appendix/migration
Good luck!
""")
# isort: off
from .utils.formatting import setup_terminal
setup_terminal()
@ -35,170 +35,172 @@ loggers = Loggers()
del Loggers
del logging
# this must happen first, prior to initializing analyses
# angr.state_plugins and angr.sim_state are mutually dependent: the plugin modules register themselves
# on SimState at import time, while SimState needs the SimStatePlugin base class. Importing the
# state_plugins package to completion here, before anything pulls in sim_state, is the one ordering
# that resolves the cycle.
from . import state_plugins # noqa: F401
# SimProcedure must be bound on the ``angr`` package before the procedures package loads, because the
# hundreds of built-in SimProcedures are defined as ``class Foo(angr.SimProcedure)`` and dereference
# that attribute at class-definition time.
from .sim_procedure import SimProcedure
from .procedures import SIM_PROCEDURES, SimProcedures, SIM_LIBRARIES, SIM_TYPE_COLLECTIONS
from . import sim_options
# isort: on
options = sim_options # alias
# enums
from .state_plugins.inspect import BP_BEFORE, BP_AFTER, BP_BOTH, BP_IPDB, BP_IPYTHON
# other stuff
from .state_plugins.inspect import BP
from .state_plugins import SimStatePlugin
from .project import Project, load_shellcode
from .llm_client import LLMClient
from . import (
analyses,
concretization_strategies,
engines,
exploration_techniques,
knowledge_plugins,
sim_options,
)
from . import sim_manager as manager
from . import sim_type as types
from .analyses import Analysis, register_analysis
from .blade import Blade
from .block import Block
from .calling_conventions import DEFAULT_CC, SYSCALL_CC, PointerWrapper, SimCC, default_cc
from .distributed import Server
from .emulator import Emulator, EmulatorStopReason
from .errors import (
AngrError,
AngrRuntimeError,
AngrValueError,
AngrLifterError,
AngrExitError,
AngrPathError,
AngrVaultError,
PathUnreachableError,
SimulationManagerError,
AngrInvalidArgumentError,
AngrSurveyorError,
AngrAIError,
AngrAnalysisError,
AngrAnnotatedCFGError,
AngrAssemblyError,
AngrBackwardSlicingError,
AngrBladeError,
AngrBladeSimProcError,
AngrAnnotatedCFGError,
AngrBackwardSlicingError,
AngrCallableError,
AngrCallableMultistateError,
AngrSyscallError,
AngrSimOSError,
AngrAssemblyError,
AngrTypeError,
AngrMissingTypeError,
AngrIncongruencyError,
AngrForwardAnalysisError,
AngrSkipJobNotice,
AngrDelayJobNotice,
AngrJobMergingFailureNotice,
AngrJobWideningFailureNotice,
AngrCFGError,
AngrVFGError,
AngrVFGRestartAnalysisNotice,
AngrCorruptDBError,
AngrDataGraphError,
AngrDBError,
AngrDDGError,
AngrLoopAnalysisError,
AngrDecompilationError,
AngrDelayJobNotice,
AngrDirectorError,
AngrError,
AngrExitError,
AngrExplorationTechniqueError,
AngrExplorerError,
AngrDirectorError,
AngrTracerError,
AngrDBError,
AngrCorruptDBError,
AngrForwardAnalysisError,
AngrIncompatibleDBError,
TracerEnvironmentError,
SimError,
SimStateError,
SimMergeError,
SimMemoryError,
SimMemoryMissingError,
AngrIncongruencyError,
AngrInvalidArgumentError,
AngrJobMergingFailureNotice,
AngrJobWideningFailureNotice,
AngrLifterError,
AngrLoopAnalysisError,
AngrMissingTypeError,
AngrNoPluginError,
AngrPathError,
AngrRuntimeError,
AngrSimOSError,
AngrSkipJobNotice,
AngrSurveyorError,
AngrSyscallError,
AngrTracerError,
AngrTypeError,
AngrUnsupportedSyscallError,
AngrValueError,
AngrVaultError,
AngrVFGError,
AngrVFGRestartAnalysisNotice,
PathUnreachableError,
SimAbstractMemoryError,
SimRegionMapError,
SimMemoryLimitError,
SimMemoryAddressError,
SimFastMemoryError,
SimActionError,
SimCCallError,
SimCCError,
SimConcreteBreakpointError,
SimConcreteMemoryError,
SimConcreteRegisterError,
SimEmptyCallStackError,
SimEngineError,
SimError,
SimEventError,
SimPosixError,
SimFilesystemError,
SimSymbolicFilesystemError,
SimException,
SimExpressionError,
SimFastMemoryError,
SimFastPathError,
SimFileError,
SimFilesystemError,
SimHeapError,
SimUnsupportedError,
SimIRSBError,
SimIRSBNoDecodeError,
SimMemoryAddressError,
SimMemoryError,
SimMemoryLimitError,
SimMemoryMissingError,
SimMergeError,
SimMissingTempError,
SimOperationError,
SimPosixError,
SimProcedureArgumentError,
SimProcedureError,
SimRegionMapError,
SimReliftException,
SimSegfaultError,
SimSegfaultException,
SimShadowStackError,
SimSlicerError,
SimSolverError,
SimSolverModeError,
SimSolverOptionError,
SimValueError,
SimUnsatError,
SimOperationError,
UnsupportedIROpError,
SimExpressionError,
UnsupportedIRExprError,
SimCCallError,
UnsupportedCCallError,
SimUninitializedAccessError,
SimStateError,
SimStatementError,
UnsupportedIRStmtError,
UnsupportedDirtyError,
SimMissingTempError,
SimEngineError,
SimIRSBError,
SimStateOptionsError,
SimSymbolicFilesystemError,
SimTranslationError,
SimProcedureError,
SimProcedureArgumentError,
SimShadowStackError,
SimFastPathError,
SimIRSBNoDecodeError,
AngrUnsupportedSyscallError,
UnsupportedSyscallError,
SimReliftException,
SimSlicerError,
SimActionError,
SimCCError,
SimUCManagerError,
SimUCManagerAllocationError,
SimUnicornUnsupport,
SimUCManagerError,
SimulationManagerError,
SimUnicornError,
SimUnicornSymbolic,
SimEmptyCallStackError,
SimStateOptionsError,
SimException,
SimSegfaultException,
SimSegfaultError,
SimUnicornUnsupport,
SimUninitializedAccessError,
SimUnsatError,
SimUnsupportedError,
SimValueError,
SimZeroDivisionException,
AngrNoPluginError,
SimConcreteMemoryError,
SimConcreteRegisterError,
SimConcreteBreakpointError,
AngrDecompilationError,
TracerEnvironmentError,
UnsupportedCCallError,
UnsupportedDirtyError,
UnsupportedIRExprError,
UnsupportedIROpError,
UnsupportedIRStmtError,
UnsupportedNodeTypeError,
AngrAIError,
UnsupportedSyscallError,
)
from .blade import Blade
from .simos import SimOS
from .block import Block
from .sim_manager import SimulationManager
from .analyses import Analysis, register_analysis
from . import analyses
from . import knowledge_plugins
from . import exploration_techniques
from .exploration_techniques import ExplorationTechnique
from . import sim_type as types
from .state_hierarchy import StateHierarchy
from .sim_state import SimState
from . import engines
from .calling_conventions import default_cc, DEFAULT_CC, SYSCALL_CC, PointerWrapper, SimCC
from .storage.file import (
SimFileBase,
SimFile,
SimPackets,
SimFileStream,
SimPacketsStream,
SimFileDescriptor,
SimFileDescriptorDuplex,
)
from .state_plugins.filesystem import SimMount, SimHostFilesystem
from .state_plugins.heap import SimHeapBrk, SimHeapPTMalloc, PTChunk
from . import concretization_strategies
from .distributed import Server
from .knowledge_base import KnowledgeBase
from .llm_client import LLMClient
from .procedures import SIM_LIBRARIES, SIM_PROCEDURES, SIM_TYPE_COLLECTIONS, SimProcedures
from .procedures.definitions import load_external_definitions
from .emulator import Emulator, EmulatorStopReason
# for compatibility reasons
from . import sim_manager as manager
from .project import Project, load_shellcode
from .rust import analyses as rust_analyses
from .rust import knowledge_plugins as rust_knowledge_plugins
from .sim_manager import SimulationManager
from .sim_state import SimState
from .simos import SimOS
from .state_hierarchy import StateHierarchy
from .state_plugins import SimStatePlugin
from .state_plugins.filesystem import SimHostFilesystem, SimMount
from .state_plugins.heap import PTChunk, SimHeapBrk, SimHeapPTMalloc
from .state_plugins.inspect import BP, BP_AFTER, BP_BEFORE, BP_BOTH, BP_IPDB, BP_IPYTHON
from .storage.file import (
SimFile,
SimFileBase,
SimFileDescriptor,
SimFileDescriptorDuplex,
SimFileStream,
SimPackets,
SimPacketsStream,
)
options = sim_options # alias
# now that we have everything loaded, re-grab the list of loggers
loggers.load_all_loggers()

View file

@ -4,19 +4,19 @@ import argparse
import contextlib
import logging
import re
from typing import TYPE_CHECKING
from collections.abc import Generator
from typing import TYPE_CHECKING
from rich import progress as rich_progress
from rich.console import Console
from rich.logging import RichHandler
from rich.syntax import Syntax
from rich.console import Console
from rich.table import Column
import angr
from angr.analyses.decompiler import DECOMPILATION_PRESETS
from angr.analyses.decompiler.decompilation_options import PARAM_TO_OPTION
from angr.analyses.decompiler.structuring import STRUCTURER_CLASSES, DEFAULT_STRUCTURER
from angr.analyses.decompiler.presets import DECOMPILATION_PRESETS
from angr.analyses.decompiler.structuring import DEFAULT_STRUCTURER, STRUCTURER_CLASSES
from angr.utils.formatting import ansi_color_enabled
if TYPE_CHECKING:

View file

@ -1,16 +1,17 @@
from __future__ import annotations
from typing import TYPE_CHECKING
from collections.abc import Callable
from typing import TYPE_CHECKING
from angr.engines.ail import ail_call_state
from angr.errors import AngrCallableError, AngrCallableMultistateError
from .callable import Callable as VEXCallable
if TYPE_CHECKING:
from angr.ailment import Address
from angr.analyses.decompiler.clinic import Clinic
from angr.project import Project
from angr.ailment import Address
from angr.sim_manager import SimulationManager

View file

@ -1,14 +1,16 @@
from __future__ import annotations
import logging
from angr.rustylib.ailment import ExpressionKind
from . import expression, statement
from .block import Block
from . import statement
from . import expression
from .statement import Assignment, Statement
from .expression import Expression, Const, Tmp, Register, UnaryOp, BinaryOp
from .converter_common import Converter
from .manager import Manager
from .block_walker import AILBlockRewriter, AILBlockViewer, AILBlockWalker
from .converter_common import Converter
from .expression import BinaryOp, Const, Expression, Register, Tmp, UnaryOp
from .manager import Manager
from .statement import Assignment, NoOp, Statement
log = logging.getLogger(__name__)
@ -21,18 +23,20 @@ available_converters: set[str] = set()
type Address = tuple[int, int | None]
try:
from .converter_vex import VEXIRSBConverter
import pyvex
from .converter_vex import VEXIRSBConverter
available_converters.add("vex")
except ImportError:
log.debug("Could not import VEXIRSBConverter", exc_info=True)
VEXIRSBConverter = None
try:
from .converter_pcode import PCodeIRSBConverter
from angr.engines import pcode
from .converter_pcode import PCodeIRSBConverter
available_converters.add("pcode")
except ImportError:
log.debug("Could not import PCodeIRSBConverter", exc_info=True)
@ -67,8 +71,10 @@ __all__ = [
"Const",
"Expr",
"Expression",
"ExpressionKind",
"IRSBConverter",
"Manager",
"NoOp",
"PCodeIRSBConverter",
"Register",
"Statement",

38
angr/ailment/_deepcopy.py Normal file
View file

@ -0,0 +1,38 @@
"""Helpers used by the Rust ailment classes' ``__deepcopy__`` implementations.
``copy.deepcopy`` on an AIL tree is satisfied by routing every class through its existing ``deep_copy(manager)``
method, with a private ``_DeepcopyManager`` standing in for ``ailment.Manager``.
"""
from __future__ import annotations
import itertools
class _DeepcopyManager:
"""Minimal stand-in for ``ailment.Manager`` -- supplies fresh atom ids."""
__slots__ = ("atom_ctr", "variable_map")
def __init__(self):
self.atom_ctr = itertools.count(start=10**9)
# No side-container -- ``Rust deep_copy`` will skip the
# ``variable_map.transfer`` step when this attribute is ``None``.
self.variable_map = None
def next_atom(self):
return next(self.atom_ctr)
def deepcopy_via_deep_copy(self, memo):
"""Use ``self.deep_copy(manager)`` to satisfy ``copy.deepcopy``."""
# we don't share substructures during deep_copy; ailment trees are acyclic by construction, so the memo dict is
# unused.
_ = memo
manager = _DeepcopyManager()
return self.deep_copy(manager)
__all__ = [
"deepcopy_via_deep_copy",
]

View file

@ -1,107 +1,7 @@
"""Shim for ``from angr.ailment.block import Block``."""
from __future__ import annotations
from typing import TYPE_CHECKING
from angr.ailment.manager import Manager
from angr.rustylib.ailment import Block # pylint:disable=import-error
if TYPE_CHECKING:
from .statement import Statement
class Block:
"""
Describes an AIL block.
__str__ should be fast because Phoenix uses networkx graph filters, and graph filters may print str(block) in some
cases, e.g.,
https://github.com/networkx/networkx/blob/861718f8aadeed4f742a348f0c437396cacdf180/networkx/classes/coreviews.py#L305
"""
__slots__ = (
"_hash",
"addr",
"idx",
"original_size",
"statements",
)
def __init__(self, addr: int, original_size, statements: list[Statement] | None = None, idx: int | None = None):
self.addr = addr
self.original_size = original_size
self.statements: list[Statement] = [] if statements is None else statements
self.idx = idx
self._hash = None # cached hash value
def copy(self, statements=None):
return Block(
addr=self.addr,
original_size=self.original_size,
statements=self.statements[::] if statements is None else statements,
idx=self.idx,
)
def deep_copy(self, manager: Manager):
return Block(
addr=self.addr,
original_size=self.original_size,
statements=[stmt.deep_copy(manager=manager) for stmt in self.statements],
idx=self.idx,
)
@property
def sort_key(self) -> tuple[int, int, int]:
return (self.addr, 0 if self.idx is None else 1, 0 if self.idx is None else self.idx)
def __lt__(self, other):
return self.sort_key < other.sort_key
def __repr__(self):
if self.idx is None:
return f"<AILBlock {self.addr:#x} of {len(self.statements)} statements>"
return f"<AILBlock {self.addr:#x}.{self.idx} of {len(self.statements)} statements>"
def dbg_repr(self, indent=0):
indent_str = " " * indent
if self.idx is None:
block_str = f"{indent_str}## Block {self.addr:x}\n"
else:
block_str = f"{indent_str}## Block {self.addr:x}.{self.idx}\n"
stmts_str = "\n".join(
[
f"{indent_str}{i:02d} | {stmt.tags.get('ins_addr', 0):#x} | {stmt}"
for i, stmt in enumerate(self.statements)
]
)
block_str += stmts_str + "\n"
return block_str
def pp(self) -> None:
print(self.dbg_repr())
def __str__(self):
if self.idx is None:
return f"<AILBlock {self.addr:#x}>"
return f"<AILBlock {self.addr:#x}.{self.idx}>"
def __eq__(self, other):
return (
type(other) is Block
and self.addr == other.addr
and self.statements == other.statements
and self.idx == other.idx
)
def likes(self, other):
return (
type(other) is Block
and len(self.statements) == len(other.statements)
and all(s1.likes(s2) for s1, s2 in zip(self.statements, other.statements))
)
def clear_hash(self):
self._hash = None
def __hash__(self):
# Changing statements does not change the hash of a block, which allows in-place statement editing
if self._hash is None:
self._hash = hash((Block, self.addr, self.idx))
return self._hash
__all__ = ["Block"]

View file

@ -1,27 +1,76 @@
# pylint:disable=unused-argument,no-self-use
from __future__ import annotations
from abc import abstractmethod
from collections import OrderedDict
from collections.abc import Callable
from abc import abstractmethod
from typing import Any, TypeVar, cast
from typing import Any, cast
from angr.rustylib.ailment import Expression as _RustExpression # pylint:disable=import-error
from angr.rustylib.ailment import Statement as _RustStatement # pylint:disable=import-error
from . import Block
from .expression import (
ITE,
Array,
Atom,
BasePointerOffset,
BinaryOp,
Call,
ComboRegister,
Const,
Convert,
DirtyExpression,
Expression,
Extract,
FunctionLikeMacro,
Insert,
Let,
Load,
Macro,
MultiStatementExpression,
Phi,
Register,
Reinterpret,
RustEnum,
StackBaseOffset,
StringLiteral,
Struct,
Tmp,
UnaryOp,
VEXCCallExpression,
VirtualVariable,
)
from .statement import (
CAS,
Statement,
SideEffectStatement,
ConditionalJump,
Assignment,
Store,
Return,
Jump,
ConditionalJump,
DirtyStatement,
Jump,
Label,
NoOp,
Return,
SideEffectStatement,
Statement,
Store,
WeakAssignment,
)
from .expression import (
_DEFAULT_STMT_HANDLER_TYPES = {
Assignment,
WeakAssignment,
CAS,
SideEffectStatement,
Store,
ConditionalJump,
Jump,
Return,
DirtyStatement,
}
_DEFAULT_EXPR_HANDLER_TYPES = {
Call,
Load,
Expression,
BinaryOp,
UnaryOp,
Convert,
@ -30,77 +79,162 @@ from .expression import (
VEXCCallExpression,
Tmp,
Register,
Const,
ComboRegister,
Reinterpret,
Const,
MultiStatementExpression,
VirtualVariable,
Phi,
Extract,
Insert,
RustEnum,
Struct,
Array,
FunctionLikeMacro,
StringLiteral,
}
def _dispatch_key(obj):
"""Resolve a handler-dict key for ``obj``."""
# Fast path: fat-enum instances are the overwhelming majority of dispatches (e.g., ~4M on ``doit``).
# Dispatch on the cached ``pykind`` instead of ``kind`` to avoid extra Rust-to-C conversion.
t = type(obj)
if t is _RustExpression:
return _EXPR_KIND_TO_MARKER.get(obj.pykind, _RustExpression)
if t is _RustStatement:
return _STMT_KIND_TO_MARKER.get(obj.pykind, _RustStatement)
# Slow path: pure-Python instances may not expose ``kind``.
kind = getattr(obj, "kind", None)
if kind is None:
return type(obj)
return _KIND_TO_MARKER.get(kind, type(obj))
_EXPR_MARKERS = (
Const,
Tmp,
Register,
ComboRegister,
Atom,
VirtualVariable,
Phi,
UnaryOp,
BinaryOp,
Convert,
Reinterpret,
Load,
ITE,
Extract,
Insert,
Call,
DirtyExpression,
VEXCCallExpression,
MultiStatementExpression,
StringLiteral,
Struct,
RustEnum,
Array,
Let,
Macro,
FunctionLikeMacro,
BasePointerOffset,
StackBaseOffset,
)
ExprType = TypeVar("ExprType")
StmtType = TypeVar("StmtType")
BlockType = TypeVar("BlockType")
_STMT_MARKERS = (
Assignment,
WeakAssignment,
Store,
Jump,
ConditionalJump,
SideEffectStatement,
Return,
CAS,
DirtyStatement,
Label,
NoOp,
)
_EXPR_KIND_TO_MARKER: dict = {}
_STMT_KIND_TO_MARKER: dict = {}
_KIND_TO_MARKER: dict = {}
_marker = None
_kind_attr = None
for _marker in _EXPR_MARKERS:
_kind_attr = _marker.__dict__.get("_kind")
if _kind_attr is not None:
_EXPR_KIND_TO_MARKER.setdefault(_kind_attr, _marker)
_KIND_TO_MARKER.setdefault(_kind_attr, _marker)
for _marker in _STMT_MARKERS:
_kind_attr = _marker.__dict__.get("_kind")
if _kind_attr is not None:
_STMT_KIND_TO_MARKER.setdefault(_kind_attr, _marker)
# _KIND_TO_MARKER may have an EK collision here -- skip if so.
if _kind_attr not in _KIND_TO_MARKER:
_KIND_TO_MARKER.setdefault(_kind_attr, _marker)
del _marker, _kind_attr
class AILBlockWalker[ExprType, StmtType, BlockType]:
"""
Walks all statements and expressions of an AIL node and construct arbitrary values based on them.
Note that we lazily initialize self._stmt_handlers and self._expr_handlers when they are accessed. This is to
support the existing pattern of updating stmt/expr handlers in-place after creating a block walker, and is slightly
slower. Overridding handler methods in a new class is the fastest approach.
"""
_default_stmt_funcs: dict[type, Callable]
_default_expr_funcs: dict[type, Callable]
# pykind (int) -> handler shadows of the default tables, for zero-frame
# single-lookup dispatch of Rust nodes that use the default handler set.
_default_stmt_funcs_by_pykind: dict
_default_expr_funcs_by_pykind: dict
def __init__(self, stmt_handlers=None, expr_handlers=None):
_default_stmt_handlers: dict[type, Callable[[int, Any, Block | None], StmtType]] = {
Assignment: self._handle_Assignment,
WeakAssignment: self._handle_WeakAssignment,
CAS: self._handle_CAS,
SideEffectStatement: self._handle_SideEffectStatement,
Store: self._handle_Store,
ConditionalJump: self._handle_ConditionalJump,
Jump: self._handle_Jump,
Return: self._handle_Return,
DirtyStatement: self._handle_DirtyStatement,
self._stmt_handlers: dict[type, Callable[[int, Any, Block | None], StmtType]] | None = stmt_handlers or None
self._expr_handlers: dict[type, Callable[[int, Any, int, Statement | None, Block | None], ExprType]] | None = (
expr_handlers or None
)
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
cls.rebuild_default_handler_funcs()
@classmethod
def rebuild_default_handler_funcs(cls) -> None:
cls._default_stmt_funcs = {t: getattr(cls, f"_handle_{t.__name__}") for t in _DEFAULT_STMT_HANDLER_TYPES}
cls._default_expr_funcs = {t: getattr(cls, f"_handle_{t.__name__}") for t in _DEFAULT_EXPR_HANDLER_TYPES}
cls._default_stmt_funcs_by_pykind = {
k: f for t, f in cls._default_stmt_funcs.items() if (k := t.__dict__.get("_kind")) is not None
}
cls._default_expr_funcs_by_pykind = {
k: f for t, f in cls._default_expr_funcs.items() if (k := t.__dict__.get("_kind")) is not None
}
_default_expr_handlers: dict[type, Callable[[int, Any, int, Statement | None, Block | None], ExprType]] = {
Call: self._handle_CallExpr,
Load: self._handle_Load,
BinaryOp: self._handle_BinaryOp,
UnaryOp: self._handle_UnaryOp,
Convert: self._handle_Convert,
ITE: self._handle_ITE,
DirtyExpression: self._handle_DirtyExpression,
VEXCCallExpression: self._handle_VEXCCallExpression,
Tmp: self._handle_Tmp,
Register: self._handle_Register,
ComboRegister: self._handle_ComboRegister,
Reinterpret: self._handle_Reinterpret,
Const: self._handle_Const,
MultiStatementExpression: self._handle_MultiStatementExpression,
VirtualVariable: self._handle_VirtualVariable,
Phi: self._handle_Phi,
Extract: self._handle_Extract,
Insert: self._handle_Insert,
RustEnum: self._handle_RustEnum,
Struct: self._handle_Struct,
Array: self._handle_Array,
FunctionLikeMacro: self._handle_FunctionLikeMacro,
StringLiteral: self._handle_StringLiteral,
}
@property
def stmt_handlers(self) -> dict[type, Callable[[int, Any, Block | None], StmtType]]:
if self._stmt_handlers is None:
self._stmt_handlers = {t: getattr(self, f"_handle_{t.__name__}") for t in _DEFAULT_STMT_HANDLER_TYPES}
return self._stmt_handlers
self.stmt_handlers: dict[type, Callable[[int, Any, Block | None], StmtType]] = (
stmt_handlers or _default_stmt_handlers
)
self.expr_handlers: dict[type, Callable[[int, Any, int, Statement | None, Block | None], ExprType]] = (
expr_handlers or _default_expr_handlers
)
@stmt_handlers.setter
def stmt_handlers(self, value) -> None:
self._stmt_handlers = value
@property
def expr_handlers(self) -> dict[type, Callable[[int, Any, int, Statement | None, Block | None], ExprType]]:
if self._expr_handlers is None:
self._expr_handlers = {t: getattr(self, f"_handle_{t.__name__}") for t in _DEFAULT_EXPR_HANDLER_TYPES}
return self._expr_handlers
@expr_handlers.setter
def expr_handlers(self, value) -> None:
self._expr_handlers = value
def reset(self) -> None:
"""
Reset per-walk state variables so that this walker can be reused for another walk. Subclasses that updates
state across a walk must override this to clear that state.
"""
def walk(self, block: Block) -> BlockType:
i = 0
@ -128,13 +262,47 @@ class AILBlockWalker[ExprType, StmtType, BlockType]:
return self._handle_expr(0, expr, stmt_idx or 0, stmt, block)
def _handle_stmt(self, stmt_idx: int, stmt: Statement, block: Block | None) -> StmtType:
handler = self.stmt_handlers.get(type(stmt), self._stmt_top)
# Inline the stmt-side dispatch: a Rust statement (the common case, ~1M/decompile) skips the ``_dispatch_key``
# frame and the redundant expr-side check, dispatching on the cached ``pykind`` int directly.
handlers = self._stmt_handlers
if handlers is None:
if type(stmt) is _RustStatement:
func = self._default_stmt_funcs_by_pykind.get(stmt.pykind)
else:
func = self._default_stmt_funcs.get(_dispatch_key(stmt))
if func is None:
return self._stmt_top(stmt_idx, stmt, block)
return func(self, stmt_idx, stmt, block)
if type(stmt) is _RustStatement:
key = _STMT_KIND_TO_MARKER.get(stmt.pykind, _RustStatement)
else:
key = _dispatch_key(stmt)
handler = handlers.get(key)
if handler is None:
return self._stmt_top(stmt_idx, stmt, block)
return handler(stmt_idx, stmt, block)
def _handle_expr(
self, expr_idx: int, expr: Expression, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> ExprType:
handler = self.expr_handlers.get(type(expr), self._top)
# Inline the expr-side dispatch: a Rust expression (the common case) dispatches on the cached ``pykind`` int
# with no ``_dispatch_key`` frame and no redundant stmt-side check.
handlers = self._expr_handlers
if handlers is None:
if type(expr) is _RustExpression:
func = self._default_expr_funcs_by_pykind.get(expr.pykind)
else:
func = self._default_expr_funcs.get(_dispatch_key(expr))
if func is None:
return self._top(expr_idx, expr, stmt_idx, stmt, block)
return func(self, expr_idx, expr, stmt_idx, stmt, block)
if type(expr) is _RustExpression:
key = _EXPR_KIND_TO_MARKER.get(expr.pykind, _RustExpression)
else:
key = _dispatch_key(expr)
handler = handlers.get(key)
if handler is None:
return self._top(expr_idx, expr, stmt_idx, stmt, block)
return handler(expr_idx, expr, stmt_idx, stmt, block)
@abstractmethod
@ -213,7 +381,7 @@ class AILBlockWalker[ExprType, StmtType, BlockType]:
self._handle_expr(0, expr.addr, stmt_idx, stmt, block)
return self._top(expr_idx, expr, stmt_idx, stmt, block)
def _handle_CallExpr(
def _handle_Call(
self, expr_idx: int, expr: Call, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> ExprType:
if not isinstance(expr.target, str):
@ -226,8 +394,9 @@ class AILBlockWalker[ExprType, StmtType, BlockType]:
def _handle_BinaryOp(
self, expr_idx: int, expr: BinaryOp, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> ExprType:
self._handle_expr(0, expr.operands[0], stmt_idx, stmt, block)
self._handle_expr(1, expr.operands[1], stmt_idx, stmt, block)
ops = expr.operands
self._handle_expr(0, ops[0], stmt_idx, stmt, block)
self._handle_expr(1, ops[1], stmt_idx, stmt, block)
return self._top(expr_idx, expr, stmt_idx, stmt, block)
def _handle_UnaryOp(
@ -295,10 +464,12 @@ class AILBlockWalker[ExprType, StmtType, BlockType]:
def _handle_DirtyExpression(
self, expr_idx: int, expr: DirtyExpression, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> ExprType:
for idx, operand in enumerate(expr.operands):
ops = expr.operands
for idx, operand in enumerate(ops):
self._handle_expr(idx, operand, stmt_idx, stmt, block)
if expr.guard is not None:
self._handle_expr(len(expr.operands) + 1, expr.guard, stmt_idx, stmt, block)
guard = expr.guard
if guard is not None:
self._handle_expr(len(ops) + 1, guard, stmt_idx, stmt, block)
return self._top(expr_idx, expr, stmt_idx, stmt, block)
def _handle_VEXCCallExpression(
@ -361,6 +532,10 @@ class AILBlockWalker[ExprType, StmtType, BlockType]:
return self._top(expr_idx, expr, stmt_idx, stmt, block)
# __init_subclass__ only runs for subclasses; build the base class's default handler tables explicitly.
AILBlockWalker.rebuild_default_handler_funcs()
class AILBlockViewer(AILBlockWalker[None, None, None]):
"""
Walks all statements and expressions of an AIL node and do nothing.
@ -372,7 +547,7 @@ class AILBlockViewer(AILBlockWalker[None, None, None]):
def _stmt_top(self, stmt_idx: int, stmt: Statement, block: Block | None):
return None
def _handle_block_end(self, stmt_results: list[StmtType], block: Block):
def _handle_block_end(self, stmt_results: list[None], block: Block):
return None
# Duplicate all handlers for performance...
@ -427,7 +602,7 @@ class AILBlockViewer(AILBlockWalker[None, None, None]):
def _handle_Load(self, expr_idx: int, expr: Load, stmt_idx: int, stmt: Statement | None, block: Block | None):
self._handle_expr(0, expr.addr, stmt_idx, stmt, block)
def _handle_CallExpr(self, expr_idx: int, expr: Call, stmt_idx: int, stmt: Statement | None, block: Block | None):
def _handle_Call(self, expr_idx: int, expr: Call, stmt_idx: int, stmt: Statement | None, block: Block | None):
if not isinstance(expr.target, str):
self._handle_expr(-1, expr.target, stmt_idx, stmt, block)
if expr.args:
@ -437,8 +612,9 @@ class AILBlockViewer(AILBlockWalker[None, None, None]):
def _handle_BinaryOp(
self, expr_idx: int, expr: BinaryOp, stmt_idx: int, stmt: Statement | None, block: Block | None
):
self._handle_expr(0, expr.operands[0], stmt_idx, stmt, block)
self._handle_expr(1, expr.operands[1], stmt_idx, stmt, block)
ops = expr.operands
self._handle_expr(0, ops[0], stmt_idx, stmt, block)
self._handle_expr(1, ops[1], stmt_idx, stmt, block)
def _handle_UnaryOp(self, expr_idx: int, expr: UnaryOp, stmt_idx: int, stmt: Statement | None, block: Block | None):
self._handle_expr(0, expr.operand, stmt_idx, stmt, block)
@ -493,10 +669,12 @@ class AILBlockViewer(AILBlockWalker[None, None, None]):
def _handle_DirtyExpression(
self, expr_idx: int, expr: DirtyExpression, stmt_idx: int, stmt: Statement | None, block: Block | None
):
for idx, operand in enumerate(expr.operands):
ops = expr.operands
for idx, operand in enumerate(ops):
self._handle_expr(idx, operand, stmt_idx, stmt, block)
if expr.guard is not None:
self._handle_expr(len(expr.operands) + 1, expr.guard, stmt_idx, stmt, block)
guard = expr.guard
if guard is not None:
self._handle_expr(len(ops) + 1, guard, stmt_idx, stmt, block)
def _handle_VEXCCallExpression(
self, expr_idx: int, expr: VEXCCallExpression, stmt_idx: int, stmt: Statement | None, block: Block | None
@ -582,58 +760,69 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
#
def _handle_Assignment(self, stmt_idx: int, stmt: Assignment, block: Block | None) -> Statement:
dst = self._handle_expr(0, stmt.dst, stmt_idx, stmt, block)
dst_in = stmt.dst
dst = self._handle_expr(0, dst_in, stmt_idx, stmt, block)
assert isinstance(dst, Atom)
changed = dst is not stmt.dst
changed = dst != dst_in
src = self._handle_expr(1, stmt.src, stmt_idx, stmt, block)
changed |= src is not stmt.src
src_in = stmt.src
src = self._handle_expr(1, src_in, stmt_idx, stmt, block)
changed |= src != src_in
if changed:
return Assignment(stmt.idx, dst, src, **stmt.tags)
return stmt
def _handle_WeakAssignment(self, stmt_idx: int, stmt: WeakAssignment, block: Block | None) -> Statement:
dst = self._handle_expr(0, stmt.dst, stmt_idx, stmt, block)
dst_in = stmt.dst
dst = self._handle_expr(0, dst_in, stmt_idx, stmt, block)
assert isinstance(dst, Atom)
changed = dst is not stmt.dst
changed = dst != dst_in
src = self._handle_expr(1, stmt.src, stmt_idx, stmt, block)
changed |= src is not stmt.src
src_in = stmt.src
src = self._handle_expr(1, src_in, stmt_idx, stmt, block)
changed |= src != src_in
if changed:
return WeakAssignment(stmt.idx, dst, src, **stmt.tags)
return stmt
def _handle_CAS(self, stmt_idx: int, stmt: CAS, block: Block | None) -> Statement:
addr = self._handle_expr(0, stmt.addr, stmt_idx, stmt, block)
changed = addr is not stmt.addr
addr_in = stmt.addr
addr = self._handle_expr(0, addr_in, stmt_idx, stmt, block)
changed = addr != addr_in
data_lo = self._handle_expr(1, stmt.data_lo, stmt_idx, stmt, block)
changed |= data_lo is not stmt.data_lo
data_lo_in = stmt.data_lo
data_lo = self._handle_expr(1, data_lo_in, stmt_idx, stmt, block)
changed |= data_lo != data_lo_in
data_hi = None
if stmt.data_hi is not None:
data_hi = self._handle_expr(2, stmt.data_hi, stmt_idx, stmt, block)
changed |= data_hi is not stmt.data_hi
data_hi_in = stmt.data_hi
if data_hi_in is not None:
data_hi = self._handle_expr(2, data_hi_in, stmt_idx, stmt, block)
changed |= data_hi != data_hi_in
expd_lo = self._handle_expr(3, stmt.expd_lo, stmt_idx, stmt, block)
changed |= expd_lo is not stmt.expd_lo
expd_lo_in = stmt.expd_lo
expd_lo = self._handle_expr(3, expd_lo_in, stmt_idx, stmt, block)
changed |= expd_lo != expd_lo_in
expd_hi = None
if stmt.expd_hi is not None:
expd_hi = self._handle_expr(4, stmt.expd_hi, stmt_idx, stmt, block)
changed |= expd_hi is not stmt.expd_hi
expd_hi_in = stmt.expd_hi
if expd_hi_in is not None:
expd_hi = self._handle_expr(4, expd_hi_in, stmt_idx, stmt, block)
changed |= expd_hi != expd_hi_in
old_lo = self._handle_expr(5, stmt.old_lo, stmt_idx, stmt, block)
old_lo_in = stmt.old_lo
old_lo = self._handle_expr(5, old_lo_in, stmt_idx, stmt, block)
assert isinstance(old_lo, Atom)
changed |= old_lo is not stmt.old_lo
changed |= old_lo != old_lo_in
old_hi = None
if stmt.old_hi is not None:
old_hi = self._handle_expr(6, stmt.old_hi, stmt_idx, stmt, block)
old_hi_in = stmt.old_hi
if old_hi_in is not None:
old_hi = self._handle_expr(6, old_hi_in, stmt_idx, stmt, block)
assert isinstance(old_hi, Atom)
changed |= old_hi is not stmt.old_hi
changed |= old_hi != old_hi_in
if changed:
return CAS(
@ -651,17 +840,23 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
return stmt
def _handle_SideEffectStatement(self, stmt_idx: int, stmt: SideEffectStatement, block: Block | None) -> Statement:
new_expr = self._handle_expr(0, stmt.expr, stmt_idx, stmt, block)
changed = new_expr is not stmt.expr
expr_in = stmt.expr
new_expr = self._handle_expr(0, expr_in, stmt_idx, stmt, block)
changed = new_expr != expr_in
new_ret_expr = None
if stmt.ret_expr is not None:
new_ret_expr = self._handle_expr(-1, stmt.ret_expr, stmt_idx, stmt, block)
if new_ret_expr is not None and new_ret_expr is not stmt.ret_expr:
ret_expr_in = stmt.ret_expr
if ret_expr_in is not None:
new_ret_expr = self._handle_expr(-1, ret_expr_in, stmt_idx, stmt, block)
if new_ret_expr is not None and new_ret_expr != ret_expr_in:
changed = True
if changed:
side_effect_expr: Call = new_expr if isinstance(new_expr, Call) else stmt.expr
# ``FunctionLikeMacro`` is included because it is a Call-shaped expression that may legitimately replace a
# Call inside a SideEffectStatement (e.g. format_macro_simplifier rewrites ``stmt.expr`` from a Call to
# ``format!(...)``). Before the ailment Rust flatten, FunctionLikeMacro inherited from Call via the pyclass
# hierarchy and matched ``isinstance(_, Call)`` automatically; after the flatten the union must be explicit.
side_effect_expr: Call = new_expr if isinstance(new_expr, (Call, FunctionLikeMacro)) else expr_in
return SideEffectStatement(
stmt.idx,
side_effect_expr,
@ -672,14 +867,17 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
return stmt
def _handle_Store(self, stmt_idx: int, stmt: Store, block: Block | None) -> Statement:
addr = self._handle_expr(0, stmt.addr, stmt_idx, stmt, block)
changed = addr is not stmt.addr
addr_in = stmt.addr
addr = self._handle_expr(0, addr_in, stmt_idx, stmt, block)
changed = addr != addr_in
data = self._handle_expr(1, stmt.data, stmt_idx, stmt, block)
changed |= data is not stmt.data
data_in = stmt.data
data = self._handle_expr(1, data_in, stmt_idx, stmt, block)
changed |= data != data_in
guard = None if stmt.guard is None else self._handle_expr(2, stmt.guard, stmt_idx, stmt, block)
changed |= guard is not stmt.guard
guard_in = stmt.guard
guard = None if guard_in is None else self._handle_expr(2, guard_in, stmt_idx, stmt, block)
changed |= guard != guard_in
if changed:
return Store(
@ -689,15 +887,14 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
stmt.size,
stmt.endness,
guard=guard,
variable=stmt.variable,
offset=stmt.offset,
**stmt.tags,
)
return stmt
def _handle_Jump(self, stmt_idx: int, stmt: Jump, block: Block | None) -> Statement:
target = self._handle_expr(0, stmt.target, stmt_idx, stmt, block)
changed = target is not stmt.target
target_in = stmt.target
target = self._handle_expr(0, target_in, stmt_idx, stmt, block)
changed = target != target_in
if changed:
return Jump(
@ -709,18 +906,21 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
return stmt
def _handle_ConditionalJump(self, stmt_idx: int, stmt: ConditionalJump, block: Block | None) -> Statement:
condition = self._handle_expr(0, stmt.condition, stmt_idx, stmt, block)
changed = condition is not stmt.condition
condition_in = stmt.condition
condition = self._handle_expr(0, condition_in, stmt_idx, stmt, block)
changed = condition != condition_in
true_target = None
if stmt.true_target is not None:
true_target = self._handle_expr(1, stmt.true_target, stmt_idx, stmt, block)
changed |= true_target is not stmt.true_target
true_target_in = stmt.true_target
if true_target_in is not None:
true_target = self._handle_expr(1, true_target_in, stmt_idx, stmt, block)
changed |= true_target != true_target_in
false_target = None
if stmt.false_target is not None:
false_target = self._handle_expr(2, stmt.false_target, stmt_idx, stmt, block)
changed |= false_target is not stmt.false_target
false_target_in = stmt.false_target
if false_target_in is not None:
false_target = self._handle_expr(2, false_target_in, stmt_idx, stmt, block)
changed |= false_target != false_target_in
if changed:
return ConditionalJump(
@ -735,20 +935,22 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
return stmt
def _handle_Return(self, stmt_idx: int, stmt: Return, block: Block | None) -> Statement:
if stmt.ret_exprs:
ret_exprs_in = stmt.ret_exprs
if ret_exprs_in:
new_ret_exprs = [
self._handle_expr(idx, expr, stmt_idx, stmt, block) for idx, expr in enumerate(stmt.ret_exprs)
self._handle_expr(idx, expr, stmt_idx, stmt, block) for idx, expr in enumerate(ret_exprs_in)
]
changed = any(old is not new for new, old in zip(new_ret_exprs, stmt.ret_exprs))
changed = any(old != new for new, old in zip(new_ret_exprs, ret_exprs_in))
if changed:
return Return(stmt.idx, new_ret_exprs, **stmt.tags)
return stmt
def _handle_DirtyStatement(self, stmt_idx: int, stmt: DirtyStatement, block: Block | None) -> Statement:
dirty = self._handle_expr(0, stmt.dirty, stmt_idx, stmt, block)
dirty_in = stmt.dirty
dirty = self._handle_expr(0, dirty_in, stmt_idx, stmt, block)
assert isinstance(dirty, DirtyExpression)
changed = dirty is not stmt.dirty
changed = dirty != dirty_in
if changed:
return DirtyStatement(stmt.idx, dirty, **stmt.tags)
@ -758,19 +960,21 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
# Expression handlers
def _handle_expr(self, expr_idx: int, expr: Expression, stmt_idx: int, stmt: Statement | None, block: Block | None):
# reach a fixed point
while True:
for _ in range(16): # limit the number of iterations to avoid infinite loops
result = super()._handle_expr(expr_idx, expr, stmt_idx, stmt, block)
if result is expr:
break
return expr
if isinstance(result, Expression) and isinstance(expr, Expression) and result.likes(expr):
return result
expr = result
return expr
def _handle_Load(
self, expr_idx: int, expr: Load, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
addr = self._handle_expr(0, expr.addr, stmt_idx, stmt, block)
changed = addr is not expr.addr
addr_in = expr.addr
addr = self._handle_expr(0, addr_in, stmt_idx, stmt, block)
changed = addr != addr_in
if changed:
new_expr = expr.copy()
@ -799,21 +1003,23 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
return expr
def _handle_CallExpr(
def _handle_Call(
self, expr_idx: int, expr: Call, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
changed = False
if isinstance(expr.target, str):
new_target = expr.target
target_in = expr.target
if isinstance(target_in, str):
new_target = target_in
else:
new_target = self._handle_expr(-1, expr.target, stmt_idx, stmt, block)
changed |= new_target is not expr.target
new_target = self._handle_expr(-1, target_in, stmt_idx, stmt, block)
changed |= new_target != target_in
args_in = expr.args
new_args = None
if expr.args is not None:
new_args = [self._handle_expr(idx, arg, stmt_idx, stmt, block) for idx, arg in enumerate(expr.args)]
changed |= any(old is not new for new, old in zip(new_args, expr.args))
if args_in is not None:
new_args = [self._handle_expr(idx, arg, stmt_idx, stmt, block) for idx, arg in enumerate(args_in)]
changed |= any(old is not new for new, old in zip(new_args, args_in))
if changed:
expr = expr.copy()
@ -825,11 +1031,13 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_BinaryOp(
self, expr_idx: int, expr: BinaryOp, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
operand_0 = self._handle_expr(0, expr.operands[0], stmt_idx, stmt, block)
changed = operand_0 is not expr.operands[0]
ops = expr.operands
op0_in, op1_in = ops[0], ops[1]
operand_0 = self._handle_expr(0, op0_in, stmt_idx, stmt, block)
changed = operand_0 != op0_in
operand_1 = self._handle_expr(1, expr.operands[1], stmt_idx, stmt, block)
changed |= operand_1 is not expr.operands[1]
operand_1 = self._handle_expr(1, op1_in, stmt_idx, stmt, block)
changed |= operand_1 != op1_in
if changed:
new_expr = expr.copy()
@ -842,8 +1050,9 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_UnaryOp(
self, expr_idx: int, expr: UnaryOp, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
new_operand = self._handle_expr(0, expr.operand, stmt_idx, stmt, block)
changed = new_operand is not expr.operand
operand_in = expr.operand
new_operand = self._handle_expr(0, operand_in, stmt_idx, stmt, block)
changed = new_operand != operand_in
if changed:
new_expr = expr.copy()
@ -854,8 +1063,9 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_Convert(
self, expr_idx: int, expr: Convert, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
new_operand = self._handle_expr(expr_idx, expr.operand, stmt_idx, stmt, block)
changed = new_operand is not expr.operand
operand_in = expr.operand
new_operand = self._handle_expr(expr_idx, operand_in, stmt_idx, stmt, block)
changed = new_operand != operand_in
if changed:
return Convert(expr.idx, expr.from_bits, expr.to_bits, expr.is_signed, new_operand, **expr.tags)
@ -864,8 +1074,9 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_Reinterpret(
self, expr_idx: int, expr: Reinterpret, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
new_operand = self._handle_expr(expr_idx, expr.operand, stmt_idx, stmt, block)
changed = new_operand is not expr.operand
operand_in = expr.operand
new_operand = self._handle_expr(expr_idx, operand_in, stmt_idx, stmt, block)
changed = new_operand != operand_in
if changed:
return Reinterpret(
@ -876,14 +1087,17 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_ITE(
self, expr_idx: int, expr: ITE, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
cond = self._handle_expr(0, expr.cond, stmt_idx, stmt, block)
changed = cond is not expr.cond
cond_in = expr.cond
cond = self._handle_expr(0, cond_in, stmt_idx, stmt, block)
changed = cond != cond_in
iftrue = self._handle_expr(1, expr.iftrue, stmt_idx, stmt, block)
changed |= iftrue is not expr.iftrue
iftrue_in = expr.iftrue
iftrue = self._handle_expr(1, iftrue_in, stmt_idx, stmt, block)
changed |= iftrue != iftrue_in
iffalse = self._handle_expr(2, expr.iffalse, stmt_idx, stmt, block)
changed |= iffalse is not expr.iffalse
iffalse_in = expr.iffalse
iffalse = self._handle_expr(2, iffalse_in, stmt_idx, stmt, block)
changed |= iffalse != iffalse_in
if changed:
new_expr = expr.copy()
@ -901,13 +1115,12 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
super()._handle_Phi(expr_idx, expr, stmt_idx, stmt, block)
return expr
changed = False
src_and_vvars_in = expr.src_and_vvars
src_and_vvars = [
(src, self._handle_expr(idx, vvar, stmt_idx, stmt, block) if vvar is not None else None)
for idx, (src, vvar) in enumerate(expr.src_and_vvars)
for idx, (src, vvar) in enumerate(src_and_vvars_in)
]
changed = any(new is not old for (_, new), (_, old) in zip(src_and_vvars, expr.src_and_vvars))
changed = any(new != old for (_, new), (_, old) in zip(src_and_vvars, src_and_vvars_in))
if changed:
assert all(vvar is None or isinstance(vvar, VirtualVariable) for _, vvar in src_and_vvars)
@ -922,14 +1135,15 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_DirtyExpression(
self, expr_idx: int, expr: DirtyExpression, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
changed = False
new_operands = [self._handle_expr(0, operand, stmt_idx, stmt, block) for operand in expr.operands]
changed = any(new is not old for new, old in zip(new_operands, expr.operands))
operands_in = expr.operands
new_operands = [self._handle_expr(0, operand, stmt_idx, stmt, block) for operand in operands_in]
changed = any(new != old for new, old in zip(new_operands, operands_in))
new_guard = None
if expr.guard is not None:
new_guard = self._handle_expr(2, expr.guard, stmt_idx, stmt, block)
changed |= new_guard is not expr.guard
guard_in = expr.guard
if guard_in is not None:
new_guard = self._handle_expr(2, guard_in, stmt_idx, stmt, block)
changed |= new_guard != guard_in
if changed:
return DirtyExpression(
@ -948,10 +1162,11 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_VEXCCallExpression(
self, expr_idx: int, expr: VEXCCallExpression, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
operands_in = expr.operands
new_operands = [
self._handle_expr(idx, operand, stmt_idx, stmt, block) for idx, operand in enumerate(expr.operands)
self._handle_expr(idx, operand, stmt_idx, stmt, block) for idx, operand in enumerate(operands_in)
]
changed = any(new is not old for new, old in zip(new_operands, expr.operands))
changed = any(new is not old for new, old in zip(new_operands, operands_in))
if changed:
new_expr = expr.copy()
@ -962,11 +1177,13 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_MultiStatementExpression(
self, expr_idx, expr: MultiStatementExpression, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
new_statements = [self._handle_stmt(idx, stmt_, None) for idx, stmt_ in enumerate(expr.stmts)]
changed = any(new is not old for new, old in zip(new_statements, expr.stmts))
stmts_in = expr.stmts
new_statements = [self._handle_stmt(idx, stmt_, None) for idx, stmt_ in enumerate(stmts_in)]
changed = any(new is not old for new, old in zip(new_statements, stmts_in))
new_expr = self._handle_expr(0, expr.expr, stmt_idx, stmt, block)
changed |= new_expr is not expr.expr
expr_in = expr.expr
new_expr = self._handle_expr(0, expr_in, stmt_idx, stmt, block)
changed |= new_expr != expr_in
if changed:
expr_ = expr.copy()
@ -978,10 +1195,12 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_Extract(
self, expr_idx: int, expr: Extract, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
new_base = self._handle_expr(0, expr.base, stmt_idx, stmt, block)
new_offset = self._handle_expr(1, expr.offset, stmt_idx, stmt, block)
base_in = expr.base
offset_in = expr.offset
new_base = self._handle_expr(0, base_in, stmt_idx, stmt, block)
new_offset = self._handle_expr(1, offset_in, stmt_idx, stmt, block)
if new_base is not expr.base or new_offset is not expr.offset:
if new_base != base_in or new_offset != offset_in:
result = expr.copy()
result.base = new_base
result.offset = new_offset
@ -991,11 +1210,14 @@ class AILBlockRewriter(AILBlockWalker[Expression, Statement, Block]):
def _handle_Insert(
self, expr_idx: int, expr: Insert, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
new_base = self._handle_expr(0, expr.base, stmt_idx, stmt, block)
new_offset = self._handle_expr(1, expr.offset, stmt_idx, stmt, block)
new_value = self._handle_expr(2, expr.value, stmt_idx, stmt, block)
base_in = expr.base
offset_in = expr.offset
value_in = expr.value
new_base = self._handle_expr(0, base_in, stmt_idx, stmt, block)
new_offset = self._handle_expr(1, offset_in, stmt_idx, stmt, block)
new_value = self._handle_expr(2, value_in, stmt_idx, stmt, block)
if new_base is not expr.base or new_offset is not expr.offset or new_value is not expr.value:
if new_base != base_in or new_offset != offset_in or new_value != value_in:
result = expr.copy()
result.base = new_base
result.offset = new_offset

View file

@ -1,18 +1,20 @@
from __future__ import annotations
import logging
from pypcode import OpCode, Varnode, PcodeOp
import pypcode
from pypcode import OpCode, PcodeOp, Varnode
from angr.utils.constants import DEFAULT_STATEMENT
from angr.engines.pcode.lifter import IRSB
from angr.utils.constants import DEFAULT_STATEMENT
from .block import Block
from .statement import Statement, Assignment, Store, Jump, ConditionalJump, Return, SideEffectStatement
from .expression import Call, Expression, DirtyExpression, Const, Register, Tmp, UnaryOp, BinaryOp, Load, Convert
from .converter_common import Converter
from .expression import BinaryOp, Call, Const, Convert, DirtyExpression, Expression, Load, Register, Tmp, UnaryOp
# FIXME: Convert, ITE
from .manager import Manager
from .converter_common import Converter
from .statement import Assignment, ConditionalJump, Jump, Return, SideEffectStatement, Statement, Store
log = logging.getLogger(name=__name__)
@ -291,12 +293,11 @@ class PCodeIRSBConverter(Converter):
size = varnode.size * 8
if space_name == "const":
return Const(self._manager.next_atom(), None, varnode.offset, size)
return Const(self._manager.next_atom(), varnode.offset, size)
if space_name == "register":
offset = self._map_register_name(varnode)
return Register(
self._manager.next_atom(),
None,
offset,
size,
reg_name=varnode.getRegisterName(),
@ -314,23 +315,23 @@ class PCodeIRSBConverter(Converter):
assert unique_offset is not None, "Cannot find the source unique variable"
# TODO: Check size
_, ori_tmp_size = self._unique_tracker[unique_offset]
t = Tmp(self._manager.next_atom(), None, unique_offset, ori_tmp_size * 8)
t = Tmp(self._manager.next_atom(), unique_offset, ori_tmp_size * 8)
# FIXME: Asserting BE
right_shift_amount = varnode.offset + varnode.size - (unique_offset + ori_tmp_size)
if right_shift_amount != 0:
t = BinaryOp(
self._manager.next_atom(),
"Shr",
[t, Const(self._manager.next_atom(), None, right_shift_amount * 8, 8)],
[t, Const(self._manager.next_atom(), right_shift_amount * 8, 8)],
False,
ins_addr=self._manager.ins_addr,
)
return Convert(self._manager.next_atom(), t.bits, size, False, t, ins_addr=self._manager.ins_addr)
return Tmp(self._manager.next_atom(), None, offset, size)
if space_name in ["ram", "mem"]:
return Tmp(self._manager.next_atom(), offset, size)
if space_name.lower() in ["ram", "mem"]:
assert not is_write
addr = Const(self._manager.next_atom(), None, varnode.offset, self._manager.arch.bits)
addr = Const(self._manager.next_atom(), varnode.offset, self._manager.arch.bits)
# Note: Load takes bytes, not bits, for size
return Load(
self._manager.next_atom(),
@ -358,8 +359,8 @@ class PCodeIRSBConverter(Converter):
return Assignment(
self._statement_idx, self._convert_varnode(varnode, True), value, ins_addr=self._manager.ins_addr
)
if space_name in ["ram", "mem"]:
addr = Const(self._manager.next_atom(), None, varnode.offset, self._manager.arch.bits)
if space_name.lower() in ["ram", "mem"]:
addr = Const(self._manager.next_atom(), varnode.offset, self._manager.arch.bits)
return Store(
self._statement_idx,
addr,
@ -428,7 +429,7 @@ class PCodeIRSBConverter(Converter):
out = self._current_op.output
inp = self._get_value(self._current_op.inputs[0])
cval = Const(self._manager.next_atom(), None, 0, self._current_op.inputs[0].size * 8)
cval = Const(self._manager.next_atom(), 0, self._current_op.inputs[0].size * 8)
expr = BinaryOp(self._manager.next_atom(), "CmpEQ", [inp, cval], signed=False, ins_addr=self._manager.ins_addr)
@ -498,7 +499,7 @@ class PCodeIRSBConverter(Converter):
# special handling: if the previous statement is a ConditionalJump with a None destination address, then we
# back-patch the previous statement
dest = Const(self._manager.next_atom(), None, dest_addr, self._manager.arch.bits)
dest = Const(self._manager.next_atom(), dest_addr, self._manager.arch.bits)
if self._statements:
last_stmt = self._statements[-1]
if isinstance(last_stmt, ConditionalJump) and last_stmt.false_target is None:
@ -516,14 +517,13 @@ class PCodeIRSBConverter(Converter):
raise NotImplementedError("p-code relative branch not supported yet")
dest_addr = self._current_op.inputs[0].offset
cond = self._get_value(self._current_op.inputs[1])
cval = Const(self._manager.next_atom(), None, 0, cond.bits)
cval = Const(self._manager.next_atom(), 0, cond.bits)
condition = BinaryOp(self._manager.next_atom(), "CmpNE", [cond, cval], signed=False)
dest = Const(self._manager.next_atom(), None, dest_addr, self._manager.arch.bits)
dest = Const(self._manager.next_atom(), dest_addr, self._manager.arch.bits)
if self._irsb._ops[-1] is self._current_op:
# if the cbranch op is the last op, then we need to generate a fallthru target
fallthru = Const(
self._manager.next_atom(),
None,
self._next_ins_addr,
self._manager.arch.bits,
)
@ -562,10 +562,10 @@ class PCodeIRSBConverter(Converter):
ret_expr = (
None
if ret_reg_offset is None
else Register(None, None, ret_reg_offset, self._manager.arch.bits, ins_addr=self._manager.ins_addr)
else Register(None, ret_reg_offset, self._manager.arch.bits, ins_addr=self._manager.ins_addr)
) # ???
if self._irsb.next is not None:
dest = Const(self._manager.next_atom(), None, self._irsb.next.con.value, self._manager.arch.bits)
dest = Const(self._manager.next_atom(), self._irsb.next.con.value, self._manager.arch.bits)
else:
dest = None
call_expr = Call(
@ -590,7 +590,7 @@ class PCodeIRSBConverter(Converter):
Convert a p-code indirect call operation
"""
ret_reg_offset = self._manager.arch.ret_offset
ret_expr = Register(None, None, ret_reg_offset, self._manager.arch.bits, ins_addr=self._manager.ins_addr) # ???
ret_expr = Register(None, ret_reg_offset, self._manager.arch.bits, ins_addr=self._manager.ins_addr) # ???
dest = self._get_value(self._current_op.inputs[0])
call_expr = Call(
self._manager.next_atom(),

View file

@ -1,837 +1,5 @@
# pylint:disable=missing-class-docstring
from __future__ import annotations
import logging
import pyvex
from angr.utils.constants import DEFAULT_STATEMENT
from angr.engines.vex.claripy.irop import vexop_to_simop
from angr.errors import UnsupportedIROpError
from angr.rustylib.ailment import VEXIRSBConverter
from .block import Block
from .statement import Assignment, CAS, Store, Jump, SideEffectStatement, ConditionalJump, DirtyStatement, Return
from .expression import (
Call,
Const,
Register,
Tmp,
DirtyExpression,
UnaryOp,
Convert,
BinaryOp,
Load,
ITE,
Reinterpret,
VEXCCallExpression,
)
from .converter_common import SkipConversionNotice, Converter
log = logging.getLogger(name=__name__)
class VEXExprConverter(Converter):
@staticmethod
def simop_from_vexop(vex_op):
return vexop_to_simop(vex_op)
@staticmethod
def generic_name_from_vex_op(vex_op):
return vexop_to_simop(vex_op)._generic_name
@staticmethod
def convert(expr, manager): # pylint:disable=arguments-differ
"""
:param expr:
:return:
"""
if isinstance(expr, pyvex.const.IRConst):
return VEXExprConverter.const_n(expr, manager)
func = EXPRESSION_MAPPINGS.get(type(expr))
if func is not None:
# When something goes wrong, return a DirtyExpression instead of crashing the program
try:
return func(expr, manager)
except UnsupportedIROpError:
log.warning("VEXExprConverter: Unsupported IROp %s.", expr.op)
return DirtyExpression(
manager.next_atom(),
f"unsupported_{expr.op}",
[],
bits=expr.result_size(manager.tyenv),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
log.warning("VEXExprConverter: Unsupported VEX expression of type %s.", type(expr))
try:
bits = expr.result_size(manager.tyenv)
except ValueError:
# e.g., "ValueError: Type Ity_INVALID does not have size"
bits = 0
return DirtyExpression(
manager.next_atom(),
f"unsupported_{type(expr)!s}",
[],
bits=bits,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def convert_list(exprs, manager):
converted = []
for expr in exprs:
converted.append(VEXExprConverter.convert(expr, manager))
return converted
@staticmethod
def register(offset, bits, manager):
reg_size = bits // manager.arch.byte_width
reg_name = manager.arch.translate_register_name(offset, reg_size)
return Register(
manager.next_atom(),
None,
offset,
bits,
reg_name=reg_name,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def tmp(tmp_idx, bits, manager):
return Tmp(
manager.next_atom(),
None,
tmp_idx,
bits,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def RdTmp(expr, manager):
return VEXExprConverter.tmp(expr.tmp, expr.result_size(manager.tyenv), manager)
@staticmethod
def Get(expr, manager):
return VEXExprConverter.register(expr.offset, expr.result_size(manager.tyenv), manager)
@staticmethod
def Load(expr, manager):
return Load(
manager.next_atom(),
VEXExprConverter.convert(expr.addr, manager),
expr.result_size(manager.tyenv) // 8,
expr.end,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def Unop(expr, manager):
op_name = VEXExprConverter.generic_name_from_vex_op(expr.op)
if op_name == "Reinterp":
simop = vexop_to_simop(expr.op)
return Reinterpret(
manager.next_atom(),
simop._from_size,
simop._from_type,
simop._to_size,
simop._to_type,
VEXExprConverter.convert(expr.args[0], manager),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
if op_name is None:
# is it a conversion?
simop = vexop_to_simop(expr.op)
if simop._conversion:
if simop._from_side == "HI":
# returns the high-half of the argument
inner = VEXExprConverter.convert(expr.args[0], manager)
shifted = BinaryOp(
manager.next_atom(),
"Shr",
[
inner,
Const(
manager.next_atom(),
None,
simop._to_size,
8,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
),
],
False,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
return Convert(
manager.next_atom(),
simop._from_size,
simop._to_size,
simop.is_signed,
shifted,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
return Convert(
manager.next_atom(),
simop._from_size,
simop._to_size,
simop.is_signed,
VEXExprConverter.convert(expr.args[0], manager),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
raise NotImplementedError("Unsupported operation")
if op_name == "Not" and expr.op != "Iop_Not1":
# NotN (N != 1) is equivalent to bitwise negation
op_name = "BitwiseNeg"
return UnaryOp(
manager.next_atom(),
op_name,
VEXExprConverter.convert(expr.args[0], manager),
bits=expr.result_size(manager.tyenv),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def Binop(expr, manager):
op = VEXExprConverter.simop_from_vexop(expr.op)
op_name = op._generic_name
operands = VEXExprConverter.convert_list(expr.args, manager)
if op_name == "Add" and type(operands[1]) is Const and operands[1].sign_bit == 1:
# convert it to a sub
op_name = "Sub"
op1_val, op1_bits = operands[1].value, operands[1].bits
operands[1] = Const(operands[1].idx, None, (1 << op1_bits) - op1_val, op1_bits)
signed = False
vector_count = None
vector_size = None
if op._vector_count is not None and op._vector_size is not None:
# SIMD conversions
op_name += "V" # vectorized
vector_count = op._vector_count
vector_size = op._vector_size
elif op_name in {"CmpLE", "CmpLT", "CmpGE", "CmpGT", "Div", "DivMod", "Mod", "Mul", "Mull"}:
if op.is_signed:
signed = True
if op_name == "Cmp" and op._float:
# Rename Cmp to CmpF
op_name = "CmpF"
if op_name is None and op._conversion:
# conversion
# TODO: Finish this
if op._from_type == "I" and op._to_type == "F":
# integer to floating point
rm = operands[0]
operand = operands[1]
return Convert(
manager.next_atom(),
op._from_size,
op._to_size,
op.is_signed,
operand,
from_type=Convert.TYPE_INT,
to_type=Convert.TYPE_FP,
rounding_mode=rm,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
if op._from_side == "HL":
# Concatenating the two arguments and form a new value
op_name = "Concat"
elif op._from_type == "F" and op._to_type == "F":
# floating point to floating point
rm = operands[0]
operand = operands[1]
return Convert(
manager.next_atom(),
op._from_size,
op._to_size,
op.is_signed,
operand,
from_type=Convert.TYPE_FP,
to_type=Convert.TYPE_FP,
rounding_mode=rm,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
elif op._from_type == "F" and op._to_type == "I":
# floating point to integer
# floating point to floating point
rm = operands[0]
operand = operands[1]
return Convert(
manager.next_atom(),
op._from_size,
op._to_size,
op.is_signed,
operand,
from_type=Convert.TYPE_FP,
to_type=Convert.TYPE_INT,
rounding_mode=rm,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
bits = op._output_size_bits
if op_name == "DivMod":
op1_size = op._from_size if op._from_size is not None else operands[0].bits
op2_size = op._to_size if op._to_size is not None else operands[1].bits
if op2_size < op1_size:
# e.g., DivModU64to32
operands[1] = Convert(
manager.next_atom(),
op2_size,
op1_size,
op._from_signed != "U",
operands[1],
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
chunk_bits = bits // 2
div = BinaryOp(
manager.next_atom(),
"Div",
operands,
signed,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
bits=op1_size,
)
truncated_div = Convert(
manager.next_atom(),
op1_size,
chunk_bits,
signed,
div,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
mod = BinaryOp(
manager.next_atom(),
"Mod",
operands,
signed,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
bits=op1_size,
)
truncated_mod = Convert(
manager.next_atom(),
op1_size,
chunk_bits,
signed,
mod,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
operands = [truncated_mod, truncated_div]
op_name = "Concat"
signed = False
return BinaryOp(
manager.next_atom(),
op_name,
operands,
signed,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
bits=bits,
vector_count=vector_count,
vector_size=vector_size,
)
@staticmethod
def Triop(expr, manager):
op = VEXExprConverter.simop_from_vexop(expr.op)
op_name = op._generic_name
operands = VEXExprConverter.convert_list(expr.args, manager)
bits = op._output_size_bits
if op._float:
# this is a floating-point operation where the first argument is the rounding mode. in fact, we have a
# BinaryOp here.
rm = operands[0]
return BinaryOp(
manager.next_atom(),
op_name,
operands[1:],
True, # all floating-point operations are signed
floating_point=True,
rounding_mode=rm,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
bits=bits,
)
raise TypeError(
"Please figure out what kind of operation this is (smart money says fused multiply) and convert it into "
"multiple binops"
)
@staticmethod
def Const(expr, manager):
# pyvex.IRExpr.Const
return Const(
manager.next_atom(),
None,
expr.con.value,
expr.result_size(manager.tyenv),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def const_n(expr, manager):
# pyvex.const.xxx
return Const(
manager.next_atom(),
None,
expr.value,
expr.size,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def ITE(expr, manager):
cond = VEXExprConverter.convert(expr.cond, manager)
iffalse = VEXExprConverter.convert(expr.iffalse, manager)
iftrue = VEXExprConverter.convert(expr.iftrue, manager)
return ITE(
manager.next_atom(),
cond,
iffalse,
iftrue,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def CCall(expr: pyvex.IRExpr.CCall, manager):
operands = [VEXExprConverter.convert(arg, manager) for arg in expr.args]
return VEXCCallExpression(
manager.next_atom(),
expr.cee.name,
operands,
bits=expr.result_size(manager.tyenv),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
EXPRESSION_MAPPINGS = {
pyvex.IRExpr.RdTmp: VEXExprConverter.RdTmp,
pyvex.IRExpr.Get: VEXExprConverter.Get,
pyvex.IRExpr.Unop: VEXExprConverter.Unop,
pyvex.IRExpr.Binop: VEXExprConverter.Binop,
pyvex.IRExpr.Triop: VEXExprConverter.Triop,
pyvex.IRExpr.Const: VEXExprConverter.Const,
pyvex.const.U32: VEXExprConverter.const_n,
pyvex.const.U64: VEXExprConverter.const_n,
pyvex.IRExpr.Load: VEXExprConverter.Load,
pyvex.IRExpr.ITE: VEXExprConverter.ITE,
pyvex.IRExpr.CCall: VEXExprConverter.CCall,
}
class VEXStmtConverter(Converter):
@staticmethod
def convert(idx, stmt, manager): # pylint:disable=arguments-differ
"""
:param idx:
:param stmt:
:param manager:
:return:
"""
try:
func = STATEMENT_MAPPINGS[type(stmt)]
except KeyError:
dirty = DirtyExpression(
manager.next_atom(),
str(stmt),
[],
bits=0,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
return DirtyStatement(
idx,
dirty,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
return func(idx, stmt, manager)
@staticmethod
def WrTmp(idx, stmt, manager):
var = VEXExprConverter.tmp(stmt.tmp, stmt.data.result_size(manager.tyenv), manager)
reg = VEXExprConverter.convert(stmt.data, manager)
return Assignment(
idx,
var,
reg,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def Put(idx, stmt, manager):
data = VEXExprConverter.convert(stmt.data, manager)
reg = VEXExprConverter.register(stmt.offset, data.bits, manager)
return Assignment(
idx,
reg,
data,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def Store(idx, stmt, manager):
return Store(
idx,
VEXExprConverter.convert(stmt.addr, manager),
VEXExprConverter.convert(stmt.data, manager),
stmt.data.result_size(manager.tyenv) // 8,
stmt.endness,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def Exit(idx, stmt, manager):
if stmt.jumpkind in {
"Ijk_EmWarn",
"Ijk_NoDecode",
"Ijk_MapFail",
"Ijk_NoRedir",
"Ijk_SigTRAP",
"Ijk_SigSEGV",
"Ijk_ClientReq",
"Ijk_SigFPE_IntDiv",
}:
raise SkipConversionNotice
return ConditionalJump(
idx,
VEXExprConverter.convert(stmt.guard, manager),
VEXExprConverter.convert(stmt.dst, manager),
None, # it will be filled in right afterwards
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def LoadG(idx, stmt: pyvex.IRStmt.LoadG, manager):
sizes = {
"ILGop_Ident32": (32, 32, False),
"ILGop_Ident64": (64, 64, False),
"ILGop_IdentV128": (128, 128, False),
"ILGop_8Uto32": (8, 32, False),
"ILGop_8Sto32": (8, 32, True),
"ILGop_16Uto32": (16, 32, False),
"ILGop_16Sto32": (16, 32, True),
}
dst = VEXExprConverter.tmp(stmt.dst, manager.tyenv.sizeof(stmt.dst), manager)
load_bits, convert_bits, signed = sizes[stmt.cvt]
src = Load(
manager.next_atom(),
VEXExprConverter.convert(stmt.addr, manager),
load_bits // 8,
stmt.end,
guard=VEXExprConverter.convert(stmt.guard, manager),
alt=VEXExprConverter.convert(stmt.alt, manager),
)
if convert_bits != load_bits:
src = Convert(manager.next_atom(), load_bits, convert_bits, signed, src)
return Assignment(
idx,
dst,
src,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def StoreG(idx, stmt: pyvex.IRStmt.StoreG, manager):
return Store(
idx,
VEXExprConverter.convert(stmt.addr, manager),
VEXExprConverter.convert(stmt.data, manager),
stmt.data.result_size(manager.tyenv) // 8,
stmt.endness,
guard=VEXExprConverter.convert(stmt.guard, manager),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
@staticmethod
def CAS(idx, stmt: pyvex.IRStmt.CAS, manager):
# addr
addr = VEXExprConverter.convert(stmt.addr, manager)
data_lo = VEXExprConverter.convert(stmt.dataLo, manager)
data_hi = VEXExprConverter.convert(stmt.dataHi, manager) if stmt.dataHi is not None else None
expd_lo = VEXExprConverter.convert(stmt.expdLo, manager)
expd_hi = VEXExprConverter.convert(stmt.expdHi, manager) if stmt.expdHi is not None else None
old_lo = VEXExprConverter.tmp(stmt.oldLo, manager.tyenv.sizeof(stmt.oldLo), manager)
old_hi = (
VEXExprConverter.tmp(stmt.oldHi, manager.tyenv.sizeof(stmt.oldHi), manager)
if stmt.oldHi != 0xFFFFFFFF
else None
)
return CAS(
idx, addr, data_lo, data_hi, expd_lo, expd_hi, old_lo, old_hi, stmt.endness, ins_addr=manager.ins_addr
)
@staticmethod
def Dirty(idx, stmt: pyvex.IRStmt.Dirty, manager):
# we translate it into tmp = DirtyExpression() if possible
operands = [VEXExprConverter.convert(op, manager) for op in stmt.args]
guard = VEXExprConverter.convert(stmt.guard, manager) if stmt.guard is not None else None
bits = manager.tyenv.sizeof(stmt.tmp) if stmt.tmp != 0xFFFFFFFF else 0
maddr = VEXExprConverter.convert(stmt.mAddr, manager) if stmt.mAddr is not None else None
dirty_expr = DirtyExpression(
manager.next_atom(),
stmt.cee.name,
operands,
guard=guard,
mfx=stmt.mFx,
maddr=maddr,
msize=stmt.mSize,
bits=bits,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
if stmt.tmp == 0xFFFFFFFF:
return DirtyStatement(
idx,
dirty_expr,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
tmp = VEXExprConverter.tmp(stmt.tmp, bits, manager)
return Assignment(
idx,
tmp,
dirty_expr,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=manager.vex_stmt_idx,
)
STATEMENT_MAPPINGS = {
pyvex.IRStmt.Put: VEXStmtConverter.Put,
pyvex.IRStmt.WrTmp: VEXStmtConverter.WrTmp,
pyvex.IRStmt.Store: VEXStmtConverter.Store,
pyvex.IRStmt.Exit: VEXStmtConverter.Exit,
pyvex.IRStmt.StoreG: VEXStmtConverter.StoreG,
pyvex.IRStmt.LoadG: VEXStmtConverter.LoadG,
pyvex.IRStmt.CAS: VEXStmtConverter.CAS,
pyvex.IRStmt.Dirty: VEXStmtConverter.Dirty,
}
class VEXIRSBConverter(Converter):
@staticmethod
def convert(irsb, manager): # pylint:disable=arguments-differ
"""
:param irsb:
:param manager:
:return:
"""
# convert each VEX statement into an AIL statement
statements = []
idx = 0
manager.tyenv = irsb.tyenv
manager.block_addr = irsb.addr
addr = irsb.addr
first_imark = True
conditional_jumps = []
for vex_stmt_idx, stmt in enumerate(irsb.statements):
if type(stmt) is pyvex.IRStmt.IMark:
if first_imark:
# update block address
addr = stmt.addr + stmt.delta
first_imark = False
manager.ins_addr = stmt.addr + stmt.delta
continue
if type(stmt) is pyvex.IRStmt.AbiHint:
# TODO: How can we use AbiHint?
continue
manager.vex_stmt_idx = vex_stmt_idx
try:
converted = VEXStmtConverter.convert(idx, stmt, manager)
if isinstance(converted, list):
# got multiple statements
statements.extend(converted)
idx += len(converted)
else:
# got one statement
statements.append(converted)
if type(converted) is ConditionalJump:
conditional_jumps.append(converted)
idx += 1
except SkipConversionNotice:
pass
manager.vex_stmt_idx = DEFAULT_STATEMENT
if irsb.jumpkind == "Ijk_Call" or irsb.jumpkind.startswith("Ijk_Sys"):
# FIXME: Move ret_expr and fp_ret_expr creation into angr because we cannot reliably determine which
# expressions can be returned from the call without performing further analysis
ret_reg_offset = manager.arch.ret_offset
ret_expr = Register(
manager.next_atom(),
None,
ret_reg_offset,
manager.arch.bits,
reg_name=manager.arch.translate_register_name(ret_reg_offset, size=manager.arch.bits),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=DEFAULT_STATEMENT,
)
fp_ret_reg_offset = manager.arch.fp_ret_offset
if fp_ret_reg_offset is not None and fp_ret_reg_offset != ret_reg_offset:
fp_ret_expr = Register(
manager.next_atom(),
None,
fp_ret_reg_offset,
manager.arch.bits,
reg_name=manager.arch.translate_register_name(fp_ret_reg_offset, size=manager.arch.bits),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=DEFAULT_STATEMENT,
)
else:
fp_ret_expr = None
if irsb.jumpkind == "Ijk_Call":
target = VEXExprConverter.convert(irsb.next, manager)
elif irsb.jumpkind.startswith("Ijk_Sys"):
# FIXME: This is a hack to make syscall work. We should have a better way to handle syscalls.
target = DirtyExpression(manager.next_atom(), "syscall", [], bits=manager.arch.bits)
else:
raise NotImplementedError("Unsupported jumpkind")
call_expr = Call(
manager.next_atom(),
target,
bits=ret_expr.bits,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=DEFAULT_STATEMENT,
)
statements.append(
SideEffectStatement(
manager.next_atom(),
call_expr,
ret_expr=ret_expr,
fp_ret_expr=fp_ret_expr,
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=DEFAULT_STATEMENT,
)
)
elif irsb.jumpkind == "Ijk_Boring":
if conditional_jumps:
# fill in the false target
cond_jump = conditional_jumps[-1]
cond_jump.false_target = VEXExprConverter.convert(irsb.next, manager)
else:
# jump
statements.append(
Jump(
manager.next_atom(),
VEXExprConverter.convert(irsb.next, manager),
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=DEFAULT_STATEMENT,
)
)
elif irsb.jumpkind == "Ijk_Ret":
# return
statements.append(
Return(
manager.next_atom(),
[],
ins_addr=manager.ins_addr,
vex_block_addr=manager.block_addr,
vex_stmt_idx=DEFAULT_STATEMENT,
)
)
else:
raise NotImplementedError("Unsupported jumpkind")
return Block(addr, irsb.size, statements=statements)
__all__ = ["VEXIRSBConverter"]

File diff suppressed because it is too large Load diff

View file

@ -1,34 +1,5 @@
from __future__ import annotations
import itertools
from angr.rustylib.ailment import Manager
class Manager:
_block_addr: int
def __init__(self, name: str | None = None, arch=None):
self.name = name
self.arch = arch
self.atom_ctr = itertools.count()
self._ins_addr: int | None = None
###
# vex specific
###
self.vex_stmt_idx: int | None = None
self.tyenv = None
def next_atom(self):
return next(self.atom_ctr)
def reset(self):
self.atom_ctr = itertools.count()
@property
def ins_addr(self) -> int | None:
return self._ins_addr
@ins_addr.setter
def ins_addr(self, v):
self._ins_addr = v
__all__ = ["Manager"]

File diff suppressed because it is too large Load diff

View file

@ -1,25 +1,28 @@
"""Python-side marker for the legacy ``isinstance(x, TaggedObject)`` checks.
We can get rid of this base class once IncompleteSwitchCaseHeadStatement is migrated to Rust.
"""
from __future__ import annotations
from typing import Any, TypedDict, TYPE_CHECKING
from typing import Self
import contextlib
from typing import TYPE_CHECKING, Any, TypedDict
from angr.ailment.manager import Manager
if TYPE_CHECKING:
from typing import Unpack
from angr.sim_type import SimType
from angr.sim_variable import SimVariable
from angr.rustylib.ailment import Expression, Statement # pylint:disable=import-error
class TagDict(TypedDict, total=False):
"""
Typed dict of tags for TaggedObject.
"""Schema for the ``.tags`` mapping on AIL data classes.
Only primitive-valued tags survive the Rust port. ``reg_vvars`` was
promoted to a dedicated field on ``VirtualVariable``. Variable
information (``variable``, ``variable_offset``, ``reference_*``) now
lives in a side :class:`VariableMap` (see
``angr.analyses.decompiler.variable_map``).
"""
always_propagate: bool
block_idx: int
custom_string: bool
deref_src_addr: int
extra_def: bool
extra_defs: list[int]
@ -27,43 +30,65 @@ class TagDict(TypedDict, total=False):
is_prototype_guessed: bool
keep_in_slice: bool
orig_ins_addr: int
reference_values: dict[SimType, Any]
reference_variable_offset: int
reference_variable: SimVariable
reg_name: str
type: dict[str, SimType]
uninitialized: bool
vex_block_addr: int
vex_stmt_idx: int
write_size: int
class TaggedObject:
"""
A class that takes tags.
class _TaggedObjectMeta(type):
"""``isinstance(x, TaggedObject)`` matches any Rust ``Expression`` /
``Statement`` instance."""
_MEMBERS = (Expression, Statement)
def __instancecheck__(cls, instance: Any) -> bool:
# Union of the Rust pyclasses, plus normal MRO dispatch so
# pure-Python subclasses of the compat markers still match.
return isinstance(instance, cls._MEMBERS) or type.__instancecheck__(cls, instance)
def __subclasscheck__(cls, subclass: type) -> bool:
return issubclass(subclass, cls._MEMBERS) or subclass is cls
class TaggedObject(metaclass=_TaggedObjectMeta):
"""Marker class for backward-compatible ``isinstance(x, TaggedObject)`` checks.
The real hierarchy has no shared base anymore; this class only exists so
legacy isinstance checks keep returning the same result.
Some legacy code does ``__hash__ = ailment.statement.TaggedObject.__hash__``
to opt in to the cached-hash machinery. We expose a pure-Python ``__hash__``
here so that idiom keeps working.
"""
__slots__ = (
"_hash",
"idx",
"tags",
)
if TYPE_CHECKING:
# Provided by the matching instances at runtime: the rustlib
# Expression / Statement pyclasses and the pure-Python marker
# subclasses all carry ``idx`` / ``tags``; subclasses that opt
# into the cached-hash machinery define ``_hash_core``. The
# ``__init__`` mirrors the compat markers' ``(idx, **tags)``
# constructor for pure-Python subclasses typed against this base.
idx: int
tags: Any
def __init__(self, idx: int, **kwargs: Unpack[TagDict]):
self.tags: TagDict = kwargs
self.idx = idx
self._hash = None
def __init__(self, idx: int | None = None, *args: Any, **tags: Any) -> None: ... # pylint:disable=keyword-arg-before-vararg,unused-argument
def _hash_core(self) -> int: ... # pylint:disable=no-self-use
def __hash__(self) -> int:
if self._hash is None:
self._hash = self._hash_core()
return self._hash
"""
The Rust pyclasses provide their own ``__hash__``. This one is the fallback for pure-Python subclasses that
define ``_hash_core``.
"""
cached = getattr(self, "_cached_hash", None)
if cached is not None:
return cached
h = self._hash_core() # pylint:disable=no-member,assignment-from-no-return
# Classes with ``__slots__`` that don't reserve ``_cached_hash`` fall through without caching.
with contextlib.suppress(AttributeError):
self._cached_hash = h # pylint:disable=attribute-defined-outside-init
return h
def _hash_core(self) -> int:
raise NotImplementedError
def copy(self) -> Self:
raise NotImplementedError
def deep_copy(self, manager: Manager) -> Self:
raise NotImplementedError
__all__ = ["TagDict", "TaggedObject"]

View file

@ -1,84 +1,25 @@
# pylint:disable=ungrouped-imports,wrong-import-position
from __future__ import annotations
import struct
import archinfo
from angr import ailment
try:
from claripy.ast import Bits
except ImportError:
from typing import Never as Bits
try:
import _md5 as md5lib
except ImportError:
import hashlib as md5lib
type GetBitsTypeParams = "Expression"
type GetBitsTypeParams = "ailment.expression.Expression"
def get_bits(expr: GetBitsTypeParams) -> int:
if isinstance(expr, Expression):
if isinstance(expr, ailment.expression.Expression):
return expr.bits
if isinstance(expr, Bits):
return expr.size()
raise TypeError(type(expr))
md5_unpacker = struct.Struct("4I")
def stable_hash(t: tuple) -> int:
cnt = _dump_tuple(t)
hd = md5lib.md5(cnt).digest()
return md5_unpacker.unpack(hd)[0] # 32 bits
def _dump_tuple(t: tuple) -> bytes:
cnt = b""
for item in t:
if item is not None:
type_ = type(item)
if type_ in _DUMP_BY_TYPE:
cnt += _DUMP_BY_TYPE[type_](item)
else:
# for TaggedObjects, hash(item) is stable
# other types of items may show up, such as pyvex.expr.CCall and Dirty. they will be removed some day.
cnt += struct.pack("<Q", hash(item) & 0xFFFF_FFFF_FFFF_FFFF)
cnt += b"\xf0"
return cnt
def _dump_str(t: str) -> bytes:
return t.encode("utf-8")
def _dump_int(t: int) -> bytes:
prefix = b"" if t >= 0 else b"-"
t = abs(t)
if t <= 0xFFFF:
return prefix + struct.pack("<H", t)
if t <= 0xFFFF_FFFF:
return prefix + struct.pack("<I", t)
if t <= 0xFFFF_FFFF_FFFF_FFFF:
return prefix + struct.pack("<Q", t)
cnt = b""
while t > 0:
cnt += _dump_int(t & 0xFFFF_FFFF_FFFF_FFFF)
t >>= 64
return prefix + cnt
def _dump_type(t: type) -> bytes:
return t.__name__.encode("ascii")
_DUMP_BY_TYPE = {
tuple: _dump_tuple,
str: _dump_str,
int: _dump_int,
type: _dump_type,
}
def is_none_or_likeable(arg1, arg2, is_list=False):
"""
Returns whether two things are both None or can like each other
@ -89,7 +30,7 @@ def is_none_or_likeable(arg1, arg2, is_list=False):
if is_list:
return len(arg1) == len(arg2) and all(is_none_or_likeable(a1, a2) for a1, a2 in zip(arg1, arg2))
if isinstance(arg1, Expression):
if isinstance(arg1, ailment.expression.Expression):
return arg1.likes(arg2)
return arg1 == arg2
@ -104,9 +45,36 @@ def is_none_or_matchable(arg1, arg2, is_list=False):
if is_list:
return len(arg1) == len(arg2) and all(is_none_or_matchable(a1, a2) for a1, a2 in zip(arg1, arg2))
if isinstance(arg1, Expression):
if isinstance(arg1, ailment.expression.Expression):
return arg1.matches(arg2)
return arg1 == arg2
from .expression import Expression
def is_lsb_extract(expr: ailment.expression.Expression) -> bool:
"""
Return ``True`` if ``expr`` is an ``Extract`` that takes the least-significant ``expr.bits`` bits of its base,
considering endianness.
"""
if not isinstance(expr, ailment.expression.Extract):
return False
if not isinstance(expr.offset, ailment.expression.Const):
return False
if expr.endness == archinfo.Endness.LE:
return expr.offset.value == 0
return expr.offset.value * 8 + expr.bits == expr.base.bits
def is_lsb_overwrite(expr: ailment.expression.Expression) -> bool:
"""
Return ``True`` if ``expr`` is an ``Insert`` that overwrites the least-significant ``expr.value.bits`` bits of its
base, considering endianness.
``Insert`` counterpart of ``is_lsb_extract``.
"""
if not isinstance(expr, ailment.expression.Insert):
return False
if not (isinstance(expr.offset, ailment.expression.Const) and isinstance(expr.offset.value, int)):
return False
if expr.endness == archinfo.Endness.LE:
return expr.offset.value == 0
return expr.offset.value * 8 + expr.value.bits == expr.bits

View file

@ -1,58 +1,56 @@
# " pylint:disable=wrong-import-position
from __future__ import annotations
from .analysis import Analysis, AnalysesHub, register_analysis
from .forward_analysis import ForwardAnalysis, visitors
from .propagator import PropagatorAnalysis
from .cfg import CFGFast, CFGEmulated, CFG, CFGArchOptions, CFGFastSoot
from .cdg import CDG
from .ddg import DDG
from .vfg import VFG
from .boyscout import BoyScout
from . import deobfuscator
from .analysis import AnalysesHub, Analysis, register_analysis
from .backward_slice import BackwardSlice
from .veritesting import Veritesting
from .vsa_ddg import VSA_DDG
from .bindiff import BinDiff
from .loopfinder import LoopFinder
from .congruency_check import CongruencyCheck
from .static_hooker import StaticHooker
from .reassembler import Reassembler
from .binary_optimizer import BinaryOptimizer
from .disassembly import Disassembly
from .variable_recovery import VariableRecovery, VariableRecoveryFast
from .identifier import Identifier
from .bindiff import BinDiff
from .boyscout import BoyScout
from .callee_cleanup_finder import CalleeCleanupFinder
from .reaching_definitions import ReachingDefinitionsAnalysis
from .calling_convention import CallingConventionAnalysis, FactCollector
from .code_tagging import CodeTagging
from .stack_pointer_tracker import StackPointerTracker
from .dominance_frontier import DominanceFrontier
from .data_dep import DataDependencyGraphAnalysis
from .decompiler import Decompiler
from .soot_class_hierarchy import SootClassHierarchy
from .xrefs import XRefsAnalysis
from .init_finder import InitializationFinder
from .complete_calling_conventions import CompleteCallingConventionsAnalysis
from .typehoon import Typehoon
from .proximity_graph import ProximityGraphAnalysis
from .vtable import VtableFinder
from .find_objects_static import StaticObjectFinder
from .cdg import CDG
from .cfg import CFG, CFGArchOptions, CFGEmulated, CFGFast, CFGFastSoot
from .class_identifier import ClassIdentifier
from .flirt import FlirtAnalysis
from .language_detector import LanguageDetector
from .s_propagator import SPropagatorAnalysis
from .s_reaching_definitions import SReachingDefinitionsAnalysis
from .s_liveness import SLivenessAnalysis
from .code_tagging import CodeTagging
from .codecave import CodeCaveAnalysis
from .patchfinder import PatchFinderAnalysis
from .pathfinder import Pathfinder
from .smc import SelfModifyingCodeAnalysis
from .unpacker import PackingDetector
from .complete_calling_conventions import CompleteCallingConventionsAnalysis
from .congruency_check import CongruencyCheck
from .data_dep import DataDependencyGraphAnalysis
from .ddg import DDG
from .decompiler import Decompiler
from .disassembly import Disassembly
from .dominance_frontier import DominanceFrontier
from .fcp import FastConstantPropagation
from .find_objects_static import StaticObjectFinder
from .flirt import FlirtAnalysis
from .forward_analysis import ForwardAnalysis, visitors
from .identifier import Identifier
from .init_finder import InitializationFinder
from .language_detector import LanguageDetector
from .loop_analysis import LoopAnalysis
from .loop_unroller import LoopUnroller
from . import deobfuscator
from .loopfinder import LoopFinder
from .patchfinder import PatchFinderAnalysis
from .pathfinder import Pathfinder
from .propagator import PropagatorAnalysis
from .proximity_graph import ProximityGraphAnalysis
from .reaching_definitions import ReachingDefinitionsAnalysis
from .reassembler import Reassembler
from .s_liveness import SLivenessAnalysis
from .s_propagator import SPropagator, SPropagatorAnalysis
from .s_reaching_definitions import SReachingDefinitions, SReachingDefinitionsAnalysis
from .smc import SelfModifyingCodeAnalysis
from .soot_class_hierarchy import SootClassHierarchy
from .stack_pointer_tracker import StackPointerTracker
from .static_hooker import StaticHooker
from .typehoon import Typehoon
from .unpacker import PackingDetector
from .variable_recovery import VariableRecovery, VariableRecoveryFast
from .veritesting import Veritesting
from .vfg import VFG
from .vsa_ddg import VSA_DDG
from .vtable import VtableFinder
from .xrefs import XRefsAnalysis
__all__ = (
"CDG",
@ -99,7 +97,9 @@ __all__ = (
"ReachingDefinitionsAnalysis",
"Reassembler",
"SLivenessAnalysis",
"SPropagator",
"SPropagatorAnalysis",
"SReachingDefinitions",
"SReachingDefinitionsAnalysis",
"SelfModifyingCodeAnalysis",
"SootClassHierarchy",

View file

@ -1,62 +1,59 @@
# pylint:disable=missing-class-docstring
from __future__ import annotations
import contextlib
import functools
import logging
import os
import sys
import contextlib
import time
import typing
from collections import defaultdict
from collections.abc import Callable
from inspect import Signature
from typing import TYPE_CHECKING, TypeVar, cast, Any
from types import NoneType
from itertools import chain
from traceback import format_exception
import logging
import time
import typing
from types import NoneType
from typing import TYPE_CHECKING, Any
import psutil
from rich import progress
from angr.misc.plugins import PluginVendor, VendorPreset
from angr.misc import telemetry
from angr.misc.plugins import PluginVendor, VendorPreset
from angr.misc.testing import is_testing
if TYPE_CHECKING:
from typing_extensions import ParamSpec
from angr.knowledge_base import KnowledgeBase
from angr.project import Project
from typing_extensions import ParamSpec
from .identifier import Identifier
from .callee_cleanup_finder import CalleeCleanupFinder
from .vsa_ddg import VSA_DDG
from .cdg import CDG
from .bindiff import BinDiff
from .cfg import CFGEmulated
from .cfg import CFBlanket
from .cfg import CFG
from .cfg import CFGFast
from .static_hooker import StaticHooker
from .ddg import DDG
from .congruency_check import CongruencyCheck
from .reassembler import Reassembler
from .backward_slice import BackwardSlice
from .binary_optimizer import BinaryOptimizer
from .vfg import VFG
from .loopfinder import LoopFinder
from .disassembly import Disassembly
from .veritesting import Veritesting
from .code_tagging import CodeTagging
from .bindiff import BinDiff
from .boyscout import BoyScout
from .variable_recovery import VariableRecoveryFast
from .variable_recovery import VariableRecovery
from .reaching_definitions import ReachingDefinitionsAnalysis
from .complete_calling_conventions import CompleteCallingConventionsAnalysis
from .decompiler.clinic import Clinic
from .propagator import PropagatorAnalysis
from .callee_cleanup_finder import CalleeCleanupFinder
from .calling_convention import CallingConventionAnalysis
from .cdg import CDG
from .cfg import CFG, CFBlanket, CFGEmulated, CFGFast
from .code_tagging import CodeTagging
from .complete_calling_conventions import CompleteCallingConventionsAnalysis
from .congruency_check import CongruencyCheck
from .ddg import DDG
from .decompiler.clinic import Clinic
from .decompiler.decompiler import Decompiler
from .disassembly import Disassembly
from .identifier import Identifier
from .loopfinder import LoopFinder
from .propagator import PropagatorAnalysis
from .reaching_definitions import ReachingDefinitionsAnalysis
from .reassembler import Reassembler
from .static_hooker import StaticHooker
from .variable_recovery import VariableRecovery, VariableRecoveryFast
from .veritesting import Veritesting
from .vfg import VFG
from .vsa_ddg import VSA_DDG
from .xrefs import XRefsAnalysis
AnalysisParams = ParamSpec("AnalysisParams")
@ -64,6 +61,9 @@ if TYPE_CHECKING:
l = logging.getLogger(name=__name__)
t = telemetry.get_tracer(name=__name__)
# the minimum interval (in seconds) between two consecutive sleeps in _release_gil()
GIL_RELEASE_INTERVAL = 0.05
class AnalysisLogEntry:
def __init__(self, message, exc_info=False):
@ -111,9 +111,6 @@ class AnalysisLogEntry:
return f"<AnalysisLogEntry {msg_str} with {self.exc_type.__name__}: {self.exc_value}>"
A = TypeVar("A", bound="Analysis")
class AnalysesHub(PluginVendor[Any]):
"""
This class contains functions for all the registered and runnable analyses,
@ -123,7 +120,7 @@ class AnalysesHub(PluginVendor[Any]):
super().__init__()
self.project = project
def _init_plugin(self, plugin_cls: type[A]) -> AnalysisFactory[A]:
def _init_plugin[A: Analysis](self, plugin_cls: type[A]) -> AnalysisFactory[A]:
return functools.wraps(plugin_cls)(AnalysisFactory(self.project, plugin_cls)) # type: ignore
def __getstate__(self): # type: ignore[reportIncompatibleMethodOverride]
@ -134,7 +131,7 @@ class AnalysesHub(PluginVendor[Any]):
s, self.project = sd
super().__setstate__(s)
def __getitem__(self, plugin_cls: type[A]) -> AnalysisFactory[A]:
def __getitem__[A: Analysis](self, plugin_cls: type[A]) -> AnalysisFactory[A]:
return functools.wraps(plugin_cls)(AnalysisFactory(self.project, plugin_cls)) # type: ignore
@ -182,7 +179,7 @@ class AnalysisFactory[A: Analysis]:
def __init__(self, project: Project, analysis_cls: type[A]):
self._project = project
self._analysis_cls = analysis_cls
self.__sig = Signature.from_callable(analysis_cls.__init__)
self.__sig: Signature | None = None # will be computed upon first access
def prep(
self,
@ -198,41 +195,44 @@ class AnalysisFactory[A: Analysis]:
@t.start_as_current_span(self._analysis_cls.__name__)
def wrapper(*args, **kwargs):
span = telemetry.get_current_span()
sig = cast(Signature, self.__sig)
bound = sig.bind(None, *args, **kwargs)
for name, val in chain(bound.arguments.items(), bound.arguments.get("kwargs", {}).items()):
if name in ("kwargs", "self"):
continue
if isinstance(val, (str, bytes, bool, int, float, NoneType)):
if val is None:
span.set_attribute(f"arg.{name}.is_none", True)
if span.is_recording():
if self.__sig is None:
self.__sig = Signature.from_callable(self._analysis_cls.__init__)
sig = self.__sig
bound = sig.bind(None, *args, **kwargs)
for name, val in chain(bound.arguments.items(), bound.arguments.get("kwargs", {}).items()):
if name in ("kwargs", "self"):
continue
if isinstance(val, (str, bytes, bool, int, float, NoneType)):
if val is None:
span.set_attribute(f"arg.{name}.is_none", True)
else:
span.set_attribute(f"arg.{name}", val)
elif isinstance(val, (list, tuple, set, frozenset)):
listval = list(val)
if not listval or (
isinstance(listval[0], (str, bytes, bool, int, float))
and all(type(sval) is type(listval[0]) for sval in listval)
):
span.set_attribute(f"arg.{name}", listval)
elif isinstance(val, dict):
listval_keys = list(val)
listval_values = list(val.values())
if not listval_keys or (
isinstance(listval_keys[0], (str, bytes, bool, int, float))
and all(type(sval) is type(listval_keys[0]) for sval in listval_keys)
):
span.set_attribute(f"arg.{name}.keys", listval_keys)
if not listval_values or (
isinstance(listval_values[0], (str, bytes, bool, int, float))
and all(type(sval) is type(listval_values[0]) for sval in listval_values)
):
span.set_attribute(f"arg.{name}.values", listval_values)
else:
span.set_attribute(f"arg.{name}", val)
elif isinstance(val, (list, tuple, set, frozenset)):
listval = list(val)
if not listval or (
isinstance(listval[0], (str, bytes, bool, int, float))
and all(type(sval) is type(listval[0]) for sval in listval)
):
span.set_attribute(f"arg.{name}", listval)
elif isinstance(val, dict):
listval_keys = list(val)
listval_values = list(val.values())
if not listval_keys or (
isinstance(listval_keys[0], (str, bytes, bool, int, float))
and all(type(sval) is type(listval_keys[0]) for sval in listval_keys)
):
span.set_attribute(f"arg.{name}.keys", listval_keys)
if not listval_values or (
isinstance(listval_values[0], (str, bytes, bool, int, float))
and all(type(sval) is type(listval_values[0]) for sval in listval_values)
):
span.set_attribute(f"arg.{name}.values", listval_values)
else:
span.set_attribute(f"arg.{name}.unrepresentable", True)
if self._project.filename is not None:
span.set_attribute("project.binary_name", self._project.filename)
span.set_attribute("project.arch_name", self._project.arch.name)
span.set_attribute(f"arg.{name}.unrepresentable", True)
if self._project.filename is not None:
span.set_attribute("project.binary_name", self._project.filename)
span.set_attribute("project.arch_name", self._project.arch.name)
oself = object.__new__(self._analysis_cls)
oself.named_errors = defaultdict(list)
@ -290,6 +290,7 @@ class Analysis:
named_errors: defaultdict[str, list[AnalysisLogEntry]] = defaultdict(list)
_ram_usage: float | None = None
_last_ramusage_update: float = 0.0
_last_gil_release: float = 0.0
_progress_callback: Callable | None = None
_show_progressbar = False
_progressbar = None
@ -378,22 +379,28 @@ class Analysis:
if self._progress_callback is not None:
self._progress_callback(100.0) # pylint:disable=not-callable
@staticmethod
def _release_gil(ctr, freq, sleep_time=0.001):
def _release_gil(self, ctr, freq, sleep_time=0.001):
"""
Periodically calls time.sleep() and releases the GIL so other threads (like, GUI threads) have a much better
chance to be scheduled, and other critical components (like the GUI) can be kept responsiveness.
This is, of course, a hack before we move all computational intensive tasks to pure C++ implementations.
Sleeping is throttled: regardless of how often this method is called, it sleeps at most once every
GIL_RELEASE_INTERVAL seconds. This keeps the GUI responsive while bounding the overhead on tight
analysis loops.
:param int ctr: A number provided by the caller.
:param int freq: How frequently time.sleep() should be called. time.sleep() is called when ctr % freq == 0.
:param int freq: How frequently the wall clock should be checked. The check happens when ctr % freq == 0.
:param sleep_time: Number (or fraction) of seconds to sleep.
:return: None
"""
if ctr != 0 and ctr % freq == 0:
time.sleep(sleep_time)
now = time.perf_counter()
if now - self._last_gil_release >= GIL_RELEASE_INTERVAL:
time.sleep(sleep_time)
self._last_gil_release = time.perf_counter()
@property
def ram_usage(self) -> float:

View file

@ -1,14 +1,14 @@
from __future__ import annotations
import logging
from collections import defaultdict
import networkx
import pyvex
from . import Analysis
from angr.analyses import AnalysesHub
from angr.code_location import CodeLocation
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.annocfg import AnnotatedCFG
from angr.code_location import CodeLocation
from angr.errors import AngrBackwardSlicingError
from angr.utils.constants import DEFAULT_STATEMENT
@ -323,9 +323,7 @@ class BackwardSlice(Analysis):
if simrun not in cfg:
l.error("SimRun instance %s is not in the CFG.", simrun)
stack = []
for simrun in simruns:
stack.append(simrun)
stack = simruns.copy()
self.runs_in_slice = networkx.DiGraph()
self.cfg_nodes_in_slice = networkx.DiGraph()

View file

@ -1,16 +1,17 @@
from __future__ import annotations
import logging
import re
from typing import TYPE_CHECKING
from collections import defaultdict
from typing import TYPE_CHECKING
from angr.analyses import AnalysesHub
from angr.knowledge_base import KnowledgeBase
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.analyses.cfg import CFGEmulated
from angr.analyses.ddg import DDG
from angr.codenode import HookNode
from angr.sim_variable import SimConstantVariable, SimRegisterVariable, SimMemoryVariable, SimStackVariable
from angr import SIM_PROCEDURES
from . import Analysis, CFGEmulated, DDG
from angr.knowledge_base import KnowledgeBase
from angr.procedures import SIM_PROCEDURES
from angr.sim_variable import SimConstantVariable, SimMemoryVariable, SimRegisterVariable, SimStackVariable
if TYPE_CHECKING:
from angr.knowledge_plugins import Function
@ -107,12 +108,9 @@ class BinaryOptimizer(Analysis):
BLOCKS_THRESHOLD = 500 # do not optimize a function if it has more than this number of blocks
def __init__(self, cfg, techniques):
def __init__(self, cfg, techniques: set[str]):
self.cfg = cfg
if techniques is None:
raise Exception("At least one optimization technique must be specified.")
supported_techniques = {
"constant_propagation",
"redundant_stack_variable_removal",
@ -121,7 +119,7 @@ class BinaryOptimizer(Analysis):
}
if techniques - supported_techniques:
raise Exception("At least one optimization technique specified is not supported.")
raise ValueError("At least one optimization technique specified is not supported.")
self._techniques = techniques.copy()

View file

@ -1,14 +1,16 @@
from __future__ import annotations
import logging
import math
import types
from collections import deque, defaultdict
from typing import TYPE_CHECKING
from collections import defaultdict, deque
from functools import partial
from typing import TYPE_CHECKING
import networkx
from angr.analyses import AnalysesHub, Analysis, CFGFast
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.analyses.cfg import CFGFast
from angr.errors import SimEngineError, SimMemoryError
from angr.knowledge_plugins.cfg.memory_data import MemoryDataSort
@ -745,8 +747,7 @@ class FunctionDiff:
ordered_succ.append(x)
# add the rest (sorting might be better than no order)
for s in sorted(succ - set(ordered_succ), key=lambda x: x.addr):
ordered_succ.append(s)
ordered_succ.extend(sorted(succ - set(ordered_succ), key=lambda x: x.addr))
return ordered_succ
except (SimMemoryError, SimEngineError):
return sorted(succ, key=lambda x: x.addr)

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import logging
import re
from collections import defaultdict
@ -6,7 +7,7 @@ from collections import defaultdict
from archinfo import all_arches
from archinfo.arch_arm import is_arm_arch
from angr.analyses import AnalysesHub, Analysis
from angr.analyses.analysis import AnalysesHub, Analysis
l = logging.getLogger(name=__name__)
@ -55,9 +56,7 @@ class BoyScout(Analysis):
l.debug("%s %s hits %d times", arch.name, arch.memory_endness, votes[(arch.name, arch.memory_endness)])
arch_name, endianness, hits = sorted(
[(k[0], k[1], v) for k, v in votes.items()], key=lambda x: x[2], reverse=True
)[0]
arch_name, endianness, hits = max([(k[0], k[1], v) for k, v in votes.items()], key=lambda x: x[2])
if hits < self.cookiesize * 2:
# this cannot possibly be code

View file

@ -2,8 +2,8 @@ from __future__ import annotations
import logging
from angr import SIM_PROCEDURES
from angr.analyses import AnalysesHub, Analysis
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.procedures import SIM_PROCEDURES
l = logging.getLogger(name=__name__)

View file

@ -1,4 +1,5 @@
from __future__ import annotations
from .calling_convention import CallingConventionAnalysis
from .fact_collector import FactCollector

View file

@ -1,64 +1,65 @@
# pylint:disable=no-self-use
from __future__ import annotations
from typing import TYPE_CHECKING
import logging
from collections import defaultdict
from collections.abc import Mapping
import logging
from typing import TYPE_CHECKING
import networkx
import capstone
from pyvex.stmt import Put
import networkx
from pyvex.expr import RdTmp
from pyvex.stmt import Put
from angr import ailment
from angr.code_location import ExternalCodeLocation
from angr.analyses.analysis import Analysis, register_analysis
from angr.analyses.reaching_definitions import ReachingDefinitionsAnalysis, get_all_definitions
from angr.calling_conventions import (
SimCC,
SimCCMicrosoftThiscall,
SimFunctionArgument,
SimRegArg,
SimStackArg,
SimCC,
default_cc,
SimCCMicrosoftThiscall,
)
from angr.code_location import ExternalCodeLocation
from angr.errors import SimTranslationError
from angr.knowledge_plugins.functions import Function
from angr.knowledge_plugins.key_definitions.atoms import MemoryLocation, Register, SpOffset
from angr.knowledge_plugins.key_definitions.constants import OP_AFTER, OP_BEFORE
from angr.knowledge_plugins.key_definitions.rd_model import ReachingDefinitionsModel
from angr.knowledge_plugins.key_definitions.tag import ReturnValueTag
from angr.knowledge_plugins.variables.variable_access import VariableAccessSort
from angr.knowledge_plugins.variables.variable_manager import VariableManagerInternal, VariableType
from angr.procedures import SIM_PROCEDURES
from angr.sim_type import (
PointerDisposition,
SimTypeCppFunction,
SimTypeInt,
SimTypeFunction,
SimType,
SimTypeBottom,
SimTypeChar,
SimTypeCppFunction,
SimTypeDouble,
SimTypeFloat,
SimTypeFunction,
SimTypeInt,
SimTypeInt128,
SimTypeLongLong,
SimTypePointer,
SimTypeReg,
SimTypeShort,
SimTypeChar,
SimTypeBottom,
SimTypeFloat,
SimTypeDouble,
parse_cpp_file,
SimTypeInt128,
)
from angr.sim_variable import SimStackVariable, SimRegisterVariable
from angr.knowledge_plugins.key_definitions.atoms import Register, MemoryLocation, SpOffset
from angr.knowledge_plugins.key_definitions.tag import ReturnValueTag
from angr.knowledge_plugins.key_definitions.constants import OP_BEFORE, OP_AFTER
from angr.knowledge_plugins.key_definitions.rd_model import ReachingDefinitionsModel
from angr.knowledge_plugins.variables.variable_access import VariableAccessSort
from angr.knowledge_plugins.functions import Function
from angr.sim_variable import SimRegisterVariable, SimStackVariable
from angr.utils.constants import DEFAULT_STATEMENT
from angr.utils.ssa import get_reg_offset_base_and_size, get_reg_offset_base
from angr import SIM_PROCEDURES
from angr.analyses import Analysis, register_analysis, ReachingDefinitionsAnalysis
from angr.analyses.reaching_definitions import get_all_definitions
from .utils import is_sane_register_variable
from angr.utils.ssa import get_reg_offset_base, get_reg_offset_base_and_size
from .fact_collector import KIND_REG, KIND_STACKVAL, FactCollector
from .utils import is_sane_register_variable
if TYPE_CHECKING:
from angr.knowledge_plugins.cfg import CFGModel
from angr.knowledge_plugins.key_definitions.uses import Uses
from angr.knowledge_plugins.key_definitions.definition import Definition
from angr.knowledge_plugins.key_definitions.uses import Uses
l = logging.getLogger(name=__name__)
@ -183,13 +184,24 @@ class CallingConventionAnalysis(Analysis):
assert self._function is not None
cpp_symbol_result: tuple[SimCC, SimTypeCppFunction, str | None] | None = None
demangled_name = self._function.demangled_name
if demangled_name != self._function.name:
r_demangled = self._analyze_demangled_name(demangled_name)
if r_demangled is not None:
self.cc, self.prototype, self.prototype_libname = r_demangled
self.proto_from_symbol = True
return
# Itanium names usually omit the return type, and a qualified name does
# not distinguish a namespace/static function from a non-static member.
# parse_cpp_file() consequently carries a possible-this placeholder.
# Do not let that incomplete declaration bypass callee/callsite analysis;
# refine it with machine facts below. Declarations that encode an explicit
# calling convention (such as Microsoft C++ symbols) remain authoritative.
demangled_cc, demangled_proto, demangled_libname = r_demangled
if isinstance(demangled_proto, SimTypeCppFunction) and demangled_proto.convention is None:
cpp_symbol_result = demangled_cc, demangled_proto, demangled_libname
else:
self.cc, self.prototype, self.prototype_libname = r_demangled
self.proto_from_symbol = True
return
if self._function.is_simprocedure:
hooker = self.project.hooked_by(self._function.addr)
@ -282,6 +294,9 @@ class CallingConventionAnalysis(Analysis):
r = self._analyze_function()
if r is None:
l.warning("Cannot determine calling convention for %r.", self._function)
if cpp_symbol_result is not None:
self.cc, self.prototype, self.prototype_libname = cpp_symbol_result
self.proto_from_symbol = True
else:
# adjust prototype if needed
cc, prototype = r
@ -296,9 +311,48 @@ class CallingConventionAnalysis(Analysis):
else None
)
if cpp_symbol_result is not None and prototype is not None:
prototype = self._refine_cpp_symbol_prototype(prototype, cpp_symbol_result[1])
self.cc = cc
self.prototype = prototype
@staticmethod
def _refine_cpp_symbol_prototype(
machine_proto: SimTypeFunction, symbol_proto: SimTypeCppFunction
) -> SimTypeFunction:
"""Merge encoded C++ types only where machine ABI arity disambiguates them.
The parser's first pointer is a *possible* ``this``. If machine facts recover
one fewer arguments, the qualified name was a namespace/static function and the
placeholder is removed. If arity agrees it is retained. Any other disagreement
keeps the machine-derived arguments. A non-Bottom encoded template return may
refine the return type; ordinary Itanium names keep the machine-derived return.
"""
machine_args = tuple(machine_proto.args or ())
symbol_args = tuple(symbol_proto.args or ())
if len(symbol_args) == len(machine_args):
selected = symbol_args
elif len(symbol_args) == len(machine_args) + 1 and symbol_args and isinstance(symbol_args[0], SimTypePointer):
selected = symbol_args[1:]
else:
selected = machine_args
# Opaque C++ classes cannot be laid out by a calling convention. Preserve the
# machine-derived slot for those arguments; only scalar/reference or pointer
# types from the linkage name are safe refinements.
args = tuple(
sym if isinstance(sym, (SimTypeReg, SimTypePointer)) else machine
for machine, sym in zip(machine_args, selected)
)
symbol_ret = symbol_proto.returnty
ret = (
symbol_ret
if symbol_ret is not None
and not isinstance(symbol_ret, SimTypeBottom)
and isinstance(symbol_ret, (SimTypeReg, SimTypePointer))
else machine_proto.returnty
)
return SimTypeFunction(args, ret, variadic=machine_proto.variadic)
def _analyze_callsite_only(self):
assert self.caller_func_addr is not None
assert self.callsite_block_addr is not None

View file

@ -1,20 +1,22 @@
# pylint:disable=too-many-boolean-expressions
from __future__ import annotations
from typing import TYPE_CHECKING
from collections import defaultdict
from collections.abc import Container, Iterator
from typing import TYPE_CHECKING
import pyvex
from angr.utils.bits import u2s
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.block import Block
from angr.analyses.analysis import Analysis
from angr.analyses import AnalysesHub
from angr.knowledge_plugins.functions import Function
from angr.codenode import BlockNode, HookNode, FuncNode
from angr.engines.light import SimEngineNostmtVEX, SimEngineLight
from angr.calling_conventions import SimRegArg, SimStackArg, default_cc
from angr.codenode import BlockNode, FuncNode, HookNode
from angr.engines.light import SimEngineLight, SimEngineNostmtVEX
from angr.knowledge_plugins.functions import Function
from angr.sim_type import SimTypeBottom, SimTypeFunction
from angr.utils.bits import u2s
from angr.utils.types import dereference_simtype_by_lib
from .utils import is_sane_register_variable
if TYPE_CHECKING:
@ -466,6 +468,141 @@ class FactCollector(Analysis):
state.register_written(offset, self.project.arch.registers[reg_name][1])
state.simple_regs[offset] = None
@staticmethod
def _resolve_vex_tmp(
expr: pyvex.IRExpr.IRExpr,
tmp_definitions: dict[int, pyvex.IRExpr.IRExpr],
seen_tmps: frozenset[int] = frozenset(),
) -> pyvex.IRExpr.IRExpr:
while isinstance(expr, pyvex.IRExpr.RdTmp) and expr.tmp not in seen_tmps:
definition = tmp_definitions.get(expr.tmp)
if definition is None:
break
seen_tmps |= {expr.tmp}
expr = definition
return expr
@classmethod
def _walk_vex_expr(
cls,
expr: pyvex.IRExpr.IRExpr,
tmp_definitions: dict[int, pyvex.IRExpr.IRExpr],
seen_tmps: frozenset[int] = frozenset(),
) -> Iterator[pyvex.IRExpr.IRExpr]:
if isinstance(expr, pyvex.IRExpr.RdTmp):
if expr.tmp in seen_tmps:
return
definition = tmp_definitions.get(expr.tmp)
if definition is not None:
yield from cls._walk_vex_expr(definition, tmp_definitions, seen_tmps | {expr.tmp})
return
yield expr
for child in expr.child_expressions:
yield from cls._walk_vex_expr(child, tmp_definitions, seen_tmps)
def _stack_canary_tls_location(self) -> tuple[int, int] | None:
if self.project.arch.name == "AMD64":
reg_name, offset = "fs", 0x28
elif self.project.arch.name == "X86":
reg_name, offset = "gs", 0x14
else:
return None
return self.project.arch.registers[reg_name][0], offset
@classmethod
def _is_tls_canary_load(
cls,
expr: pyvex.IRExpr.IRExpr,
tmp_definitions: dict[int, pyvex.IRExpr.IRExpr],
tls_reg_offset: int,
canary_offset: int,
) -> bool:
expr = cls._resolve_vex_tmp(expr, tmp_definitions)
if not isinstance(expr, pyvex.IRExpr.Load):
return False
addr_nodes = tuple(cls._walk_vex_expr(expr.addr, tmp_definitions))
return any(isinstance(node, pyvex.IRExpr.Get) and node.offset == tls_reg_offset for node in addr_nodes) and any(
isinstance(node, pyvex.IRExpr.Const) and node.con.value == canary_offset for node in addr_nodes
)
@classmethod
def _is_stack_load(
cls,
expr: pyvex.IRExpr.IRExpr,
tmp_definitions: dict[int, pyvex.IRExpr.IRExpr],
stack_reg_offsets: Container[int | None],
) -> bool:
expr = cls._resolve_vex_tmp(expr, tmp_definitions)
if not isinstance(expr, pyvex.IRExpr.Load):
return False
return any(
isinstance(node, pyvex.IRExpr.Get) and node.offset in stack_reg_offsets
for node in cls._walk_vex_expr(expr.addr, tmp_definitions)
)
def _has_terminal_call_successor(self, node: BlockNode) -> bool:
func_graph = self.function.transition_graph
for _, succ, data in func_graph.out_edges(node, data=True):
if data.get("type") != "transition" or data.get("outside", False) or not isinstance(succ, BlockNode):
continue
succ_block = self.project.factory.block(succ.addr, size=succ.size)
if succ_block.vex.jumpkind != "Ijk_Call":
continue
if not any(
edge_data.get("type") == "fake_return" for _, _, edge_data in func_graph.out_edges(succ, data=True)
):
return True
return False
def _is_stack_canary_retval_write(
self,
node: BlockNode,
block: Block,
expr: pyvex.IRExpr.IRExpr,
tmp_definitions: dict[int, pyvex.IRExpr.IRExpr],
) -> bool:
tls_location = self._stack_canary_tls_location()
if tls_location is None:
return False
expr = self._resolve_vex_tmp(expr, tmp_definitions)
if not isinstance(expr, pyvex.IRExpr.Binop) or expr.op not in {
"Iop_Sub32",
"Iop_Sub64",
"Iop_Xor32",
"Iop_Xor64",
}:
return False
tls_reg_offset, canary_offset = tls_location
stack_reg_offsets = {self.project.arch.sp_offset, self.project.arch.bp_offset}
op0, op1 = expr.args
if not (
(
self._is_tls_canary_load(op0, tmp_definitions, tls_reg_offset, canary_offset)
and self._is_stack_load(op1, tmp_definitions, stack_reg_offsets)
)
or (
self._is_tls_canary_load(op1, tmp_definitions, tls_reg_offset, canary_offset)
and self._is_stack_load(op0, tmp_definitions, stack_reg_offsets)
)
):
return False
if not self._has_terminal_call_successor(node):
return False
for stmt in block.vex.statements:
if not isinstance(stmt, pyvex.IRStmt.Exit):
continue
guard_nodes = tuple(self._walk_vex_expr(stmt.guard, tmp_definitions))
if any(
self._is_tls_canary_load(node, tmp_definitions, tls_reg_offset, canary_offset) for node in guard_nodes
) and any(self._is_stack_load(node, tmp_definitions, stack_reg_offsets) for node in guard_nodes):
return True
return False
def _analyze_endpoints_for_retval_size(self, end_states):
"""
Analyze all endpoints to determine the return value size.
@ -590,6 +727,7 @@ class FactCollector(Analysis):
# to account for the common case where the shorter register (e.g., al) is extended to the full register
# (e.g., rax) before returning.
block_retval_size = None
stack_canary_barrier = False
for stmt in reversed(block.vex.statements):
if isinstance(stmt, pyvex.IRStmt.Put):
assert block.vex.tyenv is not None
@ -609,6 +747,11 @@ class FactCollector(Analysis):
size = 4
if stmt.offset == retreg_offset:
if isinstance(node, BlockNode) and self._is_stack_canary_retval_write(
node, block, stmt.data, tmp_definitions
):
stack_canary_barrier = True
break
block_retval_size = max(size, 1)
if stmt.offset == overflow_retreg_offset:
overflow_retval_sizes.append(max(size, 1))
@ -616,6 +759,8 @@ class FactCollector(Analysis):
if block_retval_size is not None:
retval_sizes.append(block_retval_size)
continue
if stack_canary_barrier:
continue
for pred, _, data in func_graph.in_edges(node, data=True):
edge_type = data.get("type")
@ -845,7 +990,7 @@ class FactCollector(Analysis):
stackarg_sp_buff = cc_cls.STACKARG_SP_BUFF if cc_cls is not None else 0
for state in end_states:
for offset, size in state.stack_reads.items():
offset = u2s(offset, self.project.arch.bits)
offset = u2s(offset & ((1 << self.project.arch.bits) - 1), self.project.arch.bits)
if offset - ret_addr_offset > stackarg_sp_buff:
if offset in stack_offset_created or offset in callee_saved_reg_stack_offsets:
continue

View file

@ -1,8 +1,9 @@
from __future__ import annotations
import logging
import archinfo
from archinfo.arch_arm import is_arm_arch, ArchARMHF, ArchARMCortexM
from archinfo.arch_arm import ArchARMCortexM, ArchARMHF, is_arm_arch
from angr.calling_conventions import SimCC

View file

@ -1,11 +1,12 @@
from __future__ import annotations
import logging
import networkx
from angr.analyses import AnalysesHub
from angr.utils.graph import compute_dominance_frontier, PostDominators, TemporaryNode
from . import Analysis
import angr
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.utils.graph import PostDominators, TemporaryNode, compute_dominance_frontier
_l = logging.getLogger(name=__name__)
@ -35,11 +36,7 @@ class CDG(Analysis):
if not no_construct:
if self._cfg is None:
# This leads to import cycles otherwise
# pylint: disable=import-outside-toplevel
from angr.analyses.cfg.cfg_emulated import CFGEmulated
self._cfg = self.project.analyses[CFGEmulated].prep()()
self._cfg = self.project.analyses[angr.analyses.cfg.CFGEmulated].prep()()
# FIXME: We should not use get_any_irsb in such a real setting...
self._entry = self._cfg.model.get_any_node(self._start)

View file

@ -1,15 +1,15 @@
# analyses
from __future__ import annotations
from .cfg_fast import CFGFast
from .cfg_emulated import CFGEmulated
from .cfg import CFG
from .cfb import CFBlanket
from .cfg_fast_soot import CFGFastSoot
from .cfg import CFG
# things to make your life easier
from .cfg_arch_options import CFGArchOptions
from .cfg_base import CFGBase
from .cfg_emulated import CFGEmulated
from .cfg_fast import CFGFast
from .cfg_fast_soot import CFGFastSoot
__all__ = (
"CFG",

View file

@ -1,17 +1,17 @@
from __future__ import annotations
import logging
from typing import Any
from collections.abc import Callable
from typing import Any
import cle
from cle.backends.externs import KernelObject, ExternObject
from cle.backends.tls.elf_tls import ELFTLSObject
from cle.backends.externs import ExternObject, KernelObject
from cle.backends.ihex import Hex
from cle.backends.tls.elf_tls import ELFTLSObject
from sortedcontainers import SortedDict
from angr.analyses import AnalysesHub
from angr.knowledge_plugins.cfg.memory_data import MemoryDataSort, MemoryData
from angr.analyses.analysis import Analysis
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.knowledge_plugins.cfg.memory_data import MemoryData, MemoryDataSort
_l = logging.getLogger(name=__name__)

View file

@ -2,7 +2,8 @@ from __future__ import annotations
import sys
from angr.analyses import AnalysesHub
from angr.analyses.analysis import AnalysesHub
from .cfg_fast import CFGFast

View file

@ -1,61 +1,61 @@
# pylint:disable=line-too-long,multiple-statements
from __future__ import annotations
from typing import TYPE_CHECKING, Any, overload, Literal
from collections.abc import Callable
import logging
from collections import defaultdict
from collections.abc import Callable
from typing import TYPE_CHECKING, Any, Literal, overload
import archinfo
import networkx
from sortedcontainers import SortedDict
import pyvex
from archinfo.arch_arm import get_real_address_if_arm, is_arm_arch
from archinfo.arch_soot import SootAddressDescriptor, SootMethodDescriptor
from cle import (
ELF,
PE,
XBE,
Blob,
PEStubs,
TLSObject,
MachO,
Coff,
ExternObject,
KernelObject,
FunctionHintSource,
Hex,
Coff,
KernelObject,
MachO,
PEStubs,
SRec,
XBE,
TLSObject,
)
from cle.backends import NamedRegion
import archinfo
from archinfo.arch_soot import SootAddressDescriptor, SootMethodDescriptor
from archinfo.arch_arm import is_arm_arch, get_real_address_if_arm
from sortedcontainers import SortedDict
from angr.knowledge_plugins.functions.function_manager import FunctionManager
from angr.knowledge_plugins.cfg import IndirectJump, CFGNode, CFGENode, CFGModel # pylint:disable=unused-import
from angr.knowledge_plugins.cfg.spilling_cfg import block_key_to_addr, get_block_key, block_key_to_size
from angr.procedures.stubs.UnresolvableJumpTarget import UnresolvableJumpTarget
from angr.utils.constants import DEFAULT_STATEMENT
from angr.procedures.procedure_dict import SIM_PROCEDURES
from angr.analyses.analysis import Analysis
from angr.analyses.stack_pointer_tracker import StackPointerTracker
from angr.codenode import BlockNode, FuncNode, HookNode
from angr.engines.vex.lifter import VEX_IRSB_MAX_INST, VEX_IRSB_MAX_SIZE
from angr.errors import (
AngrCFGError,
SimTranslationError,
SimMemoryError,
SimIRSBError,
SimEngineError,
AngrUnsupportedSyscallError,
SimEngineError,
SimError,
SimIRSBError,
SimMemoryError,
SimTranslationError,
)
from angr.codenode import HookNode, BlockNode, FuncNode
from angr.engines.vex.lifter import VEX_IRSB_MAX_SIZE, VEX_IRSB_MAX_INST
from angr.analyses import Analysis
from angr.analyses.stack_pointer_tracker import StackPointerTracker
from angr.knowledge_plugins.cfg import CFGENode, CFGModel, CFGNode, IndirectJump # pylint:disable=unused-import
from angr.knowledge_plugins.cfg.spilling_cfg import block_key_to_addr, block_key_to_size, get_block_key
from angr.knowledge_plugins.functions.function_manager import FunctionManager
from angr.procedures.procedure_dict import SIM_PROCEDURES
from angr.procedures.stubs.UnresolvableJumpTarget import UnresolvableJumpTarget
from angr.utils.constants import DEFAULT_STATEMENT
from angr.utils.orderedset import OrderedSet
from .indirect_jump_resolvers.default_resolvers import default_indirect_jump_resolvers
if TYPE_CHECKING:
from angr.sim_state import SimState
from angr.knowledge_plugins.cfg.spilling_cfg import SpillingCFG
from angr.knowledge_plugins.cfg.types import K
from angr.sim_state import SimState
AddressType = int | SootAddressDescriptor
MethodType = int | SootMethodDescriptor
@ -255,6 +255,7 @@ class CFGBase(Analysis):
'"auto_load_libs" disabled, or specify "regions" to limit the scope of CFG recovery.'
)
regions_derived_from_objects = regions is None
if regions is None:
regions = self._exec_mem_regions
if not self._skip_unmapped_addrs and not regions:
@ -284,6 +285,14 @@ class CFGBase(Analysis):
for start, end in self._regions.items():
l.debug("... %#x - %#x", start, end)
if regions_derived_from_objects and not self._regions_size:
l.warning(
"CFG recovery has nothing to scan: the regions to analyze cover 0 bytes. If %s does contain code, "
'pass the address ranges that hold it in "regions", or set "force_segment" to derive regions from '
"segments instead of sections.",
self._binary,
)
def __contains__(self, cfg_node):
return cfg_node in self.graph
@ -325,6 +334,15 @@ class CFGBase(Analysis):
"""
return self.kb.functions
@property
def regions(self) -> list[tuple[int, int]]:
"""
The memory regions that this analysis covers. An empty list means it had nothing to scan.
:return: A sorted list of (start address, end address) tuples.
"""
return list(self._regions.items())
#
# Methods
#
@ -636,6 +654,24 @@ class CFGBase(Analysis):
else:
return address < self._regions[start_addr]
def _inside_regions_and_region_end(self, address: int | None) -> tuple[bool, int | None]:
"""
Check if the address is inside any existing region, and return the end of that region.
:param int address: Address to check.
:return: A tuple (True, region_end) if the address is within one of the memory regions, where
region_end is the end of that memory region; (False, None) otherwise.
"""
try:
start_addr = next(self._regions.irange(maximum=address, reverse=True))
except StopIteration:
return False, None
else:
if address < self._regions[start_addr]:
return True, self._regions[start_addr]
return False, None
def _get_min_addr(self) -> int | None:
"""
Get the minimum address out of all regions. We assume self._regions is sorted.
@ -873,6 +909,9 @@ class CFGBase(Analysis):
if not memory_regions and not has_executable:
memory_regions = [(start, start + len(backer)) for start, backer in self.project.loader.memory.backers()]
# A section or segment that maps no bytes, such as the empty .text of a data-only relocatable, is not a region.
memory_regions = [(start, end) for start, end in memory_regions if end > start]
return sorted(memory_regions, key=lambda x: x[0])
def _addr_in_exec_memory_regions(self, addr):

View file

@ -1,51 +1,55 @@
from __future__ import annotations
from typing import TYPE_CHECKING
import contextlib
import itertools
import logging
import sys
from collections import defaultdict
from functools import reduce
import contextlib
from typing import TYPE_CHECKING
import angr
import claripy
import networkx
import pyvex
from archinfo import ArchARM
from angr.analyses import ForwardAnalysis
from angr.utils.graph import GraphUtils
from angr.analyses import AnalysesHub
from angr import BP, BP_BEFORE, BP_AFTER, SIM_PROCEDURES, procedures
from angr import options as o
import angr
from angr import procedures
from angr import sim_options as o
from angr.analyses.analysis import AnalysesHub
from angr.analyses.backward_slice import BackwardSlice
from angr.analyses.cdg import CDG
from angr.analyses.ddg import DDG
from angr.analyses.forward_analysis import ForwardAnalysis
from angr.analyses.loopfinder import Loop, LoopFinder
from angr.codenode import BlockNode, FuncNode
from angr.engines.procedure import ProcedureEngine
from angr.exploration_techniques.loop_seer import LoopSeer
from angr.exploration_techniques.slicecutor import Slicecutor
from angr.exploration_techniques.explorer import Explorer
from angr.exploration_techniques.lengthlimiter import LengthLimiter
from angr.errors import (
AngrCFGError,
AngrError,
AngrExitError,
AngrSkipJobNotice,
AngrSyscallError,
SimEmptyCallStackError,
SimError,
SimValueError,
SimSolverModeError,
SimFastPathError,
SimIRSBError,
AngrExitError,
SimEmptyCallStackError,
SimSolverModeError,
SimValueError,
)
from angr.exploration_techniques.explorer import Explorer
from angr.exploration_techniques.lengthlimiter import LengthLimiter
from angr.exploration_techniques.loop_seer import LoopSeer
from angr.exploration_techniques.slicecutor import Slicecutor
from angr.knowledge_plugins.cfg import BlockID, CFGENode, IndirectJump
from angr.procedures import SIM_PROCEDURES
from angr.sim_state import SimState
from angr.state_plugins.callstack import CallStack
from angr.state_plugins.inspect import BP, BP_AFTER, BP_BEFORE
from angr.state_plugins.sim_action import SimActionData
from angr.knowledge_plugins.cfg import CFGENode, IndirectJump, BlockID
from angr.utils.constants import DEFAULT_STATEMENT
from angr.analyses.cdg import CDG
from angr.analyses.ddg import DDG
from angr.analyses.backward_slice import BackwardSlice
from angr.analyses.loopfinder import LoopFinder, Loop
from angr.utils.graph import GraphUtils
from .cfg_base import CFGBase
from .cfg_job_base import CFGJobBase
@ -1135,7 +1139,7 @@ class CFGEmulated(ForwardAnalysis, CFGBase): # pylint: disable=abstract-method
self._update_function_callsites(funcaddrs_do_not_return)
# Create all pending edges
for _, edges in self._pending_edges.items():
for edges in self._pending_edges.values():
for src_node, dst_node, data in edges:
self._graph_add_edge(src_node, dst_node, **data)

View file

@ -1,83 +1,89 @@
# pylint:disable=superfluous-parens,too-many-boolean-expressions,line-too-long
from __future__ import annotations
from typing import Any, TYPE_CHECKING
import itertools
import logging
import math
import re
import string
from collections import defaultdict, OrderedDict
import time
from collections import OrderedDict, defaultdict
from enum import Enum, unique
from typing import TYPE_CHECKING, Any
import networkx
from sortedcontainers import SortedDict
import capstone
import claripy
import cle
import networkx
import pyvex
from cle.address_translator import AT
from archinfo import Endness
from archinfo.arch_arm import get_real_address_if_arm, is_arm_arch
from archinfo.arch_soot import SootAddressDescriptor
from archinfo.arch_arm import is_arm_arch, get_real_address_if_arm
from cle.address_translator import AT
from sortedcontainers import SortedDict
from angr.analyses import AnalysesHub
from angr.misc.ux import once
from angr.knowledge_plugins.cfg.spilling_cfg import get_block_key, block_key_to_addr, block_key_to_size
from angr.knowledge_plugins.cfg import (
CFGNode,
MEMORY_DATA_SORTS,
MemoryDataSort,
MemoryData,
IndirectJump,
IndirectJumpType,
)
from angr.knowledge_plugins.xrefs import XRef, XRefType
from angr.codenode import HookNode, FuncNode
from angr.utils.ins_addr_list import InsAddrList
import angr
from angr import sim_options as o
from angr.analyses.analysis import AnalysesHub
from angr.analyses.forward_analysis import ForwardAnalysis
from angr.codenode import FuncNode, HookNode
from angr.errors import (
AngrCFGError,
AngrSkipJobNotice,
SimEngineError,
SimIRSBNoDecodeError,
SimMemoryError,
SimTranslationError,
SimValueError,
SimIRSBNoDecodeError,
)
from angr.knowledge_plugins.cfg import (
MEMORY_DATA_SORTS,
CFGNode,
IndirectJump,
IndirectJumpType,
MemoryData,
MemoryDataSort,
)
from angr.knowledge_plugins.cfg.spilling_cfg import block_key_to_addr, block_key_to_size, get_block_key
from angr.knowledge_plugins.xrefs import XRef, XRefType
from angr.misc.ux import once
from angr.rustylib import SegmentList
from angr.simos import SimWindows
from angr.utils.constants import DEFAULT_STATEMENT
from angr.utils.funcid import (
is_function_likely_security_init_cookie,
is_function_security_check_cookie,
is_function_security_check_cookie_strict,
is_function_security_init_cookie,
is_function_security_init_cookie_win8,
is_function_likely_security_init_cookie,
)
from angr.analyses import ForwardAnalysis
from angr.rustylib import SegmentList
from angr.utils.ins_addr_list import InsAddrList
from .cfg_arch_options import CFGArchOptions
from .cfg_base import CFGBase
from .indirect_jump_resolvers.jumptable import JumpTableResolver
from .meta_structs import get_data_regions_from_meta_regions, get_pointer_array_hints
from .pe_msvc_eh_structs import (
parse_funcinfo,
parse_unwind_map,
parse_try_block_map,
parse_eh4_scopetable,
FUNCINFO_SIZE,
UNWINDMAPENTRY_SIZE,
TRYBLOCKMAPENTRY_SIZE,
HANDLERTYPE_SIZE,
TRYBLOCKMAPENTRY_SIZE,
UNWINDMAPENTRY_SIZE,
parse_eh4_scopetable,
parse_funcinfo,
parse_try_block_map,
parse_unwind_map,
)
if TYPE_CHECKING:
from angr.block import Block
from angr.knowledge_plugins.cfg.spilling_cfg import SpillingCFG
from angr.engines.pcode.lifter import IRSB as PcodeIRSB
from angr.knowledge_plugins.cfg.spilling_cfg import SpillingCFG
from angr.knowledge_plugins.cfg.types import CFGNODE_K
VEX_IRSB_MAX_SIZE = 400
# the minimum interval (in seconds) between two consecutive progress notifications
PROGRESS_NOTIFY_INTERVAL = 0.05
l = logging.getLogger(name=__name__)
@ -864,6 +870,10 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
# mapping to all known thunks
self._known_thunks = {}
# when True, jump/call targets loaded from registered read-only regions (e.g. PE IAT slots) are
# constant-folded at lift time and consumed in _create_jobs without invoking indirect jump resolvers
self._fold_ro_const_loads = False
self._initial_state = None
self._next_addr: int | None = None
@ -884,6 +894,7 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
self._exception_handling_by_endaddr = SortedDict()
self._last_percentage = 0.0
self._last_progress_notify = 0.0
# record the number of resolved and unresolved indirect jumps *during* _process_unresolved_indirect_jumps()
# always clear them after returning from _process_unresolved_indirect_jumps()
self._transitory_resolved_indirect_jumps = 0
@ -990,6 +1001,12 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
if not (self._show_progressbar or self._progress_callback):
return
# Do not notify too often; this method is called hundreds of thousands of times on large binaries.
now = time.perf_counter()
if now - self._last_progress_notify < PROGRESS_NOTIFY_INTERVAL:
return
self._last_progress_notify = now
max_percentage_stage_1 = 50.0
if not skip_percentage:
percentage = min(
@ -1095,20 +1112,33 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
return None
return val
def _scan_for_printable_strings(self, start_addr):
def _scan_for_printable_strings(
self, start_addr: int, min_length_nullterminated: int = 3, min_length_non_nullterminated: int = 13
) -> int:
"""
This method finds both zero-terminated strings and non-zero terminated strings. In the case of zero-terminated
strings, the returned stirng length will include the ending null byte.
"""
addr = start_addr
sz = []
is_sz = True
# Get data until we meet a null-byte
while self._inside_regions(addr):
l.debug("Searching address %x", addr)
region_end = None
while True:
if region_end is None or addr >= region_end:
inside_region, region_end = self._inside_regions_and_region_end(addr)
else:
inside_region = addr < region_end
if not inside_region:
break
# l.debug("Searching address %x", addr)
val = self._load_a_byte_as_int(addr)
if val is None:
break
if val == 0:
if len(sz) < 4:
is_sz = False
break
if val not in self.PRINTABLES:
is_sz = False
@ -1116,29 +1146,41 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
sz.append(val)
addr += 1
if sz and is_sz:
if sz and len(sz) >= min(min_length_nullterminated, min_length_non_nullterminated):
# avoid commonly seen ambiguous cases
if is_arm_arch(self.project.arch):
# little endian
sz_bytes = bytes(sz)
if self.project.arch.memory_endness == Endness.LE and b"\x70\x47" in sz_bytes: # bx lr
return 0
return sz_bytes.find(b"\x70\x47")
if self.project.arch.memory_endness == Endness.BE and b"\x47\x70" in sz_bytes: # bx lr
return 0
l.debug("Got a string of %d chars", len(sz))
return len(sz) + 1
return sz_bytes.find(b"\x47\x70")
if is_sz and len(sz) >= min_length_nullterminated:
return len(sz) + 1
if not is_sz and len(sz) >= min_length_non_nullterminated:
return len(sz)
# no string is found
return 0
def _scan_for_printable_widestrings(self, start_addr: int, min_length: int = 3) -> int:
def _scan_for_printable_widestrings(
self, start_addr: int, min_length_nullterminated: int = 3, min_length_non_nullterminated: int = 9
) -> int:
addr = start_addr
sz = []
is_sz = True
# Get data until we meet two null bytes
while self._inside_regions(addr):
l.debug("Searching address %x", addr)
region_end = None
while True:
if region_end is None or addr >= region_end:
inside_region, region_end = self._inside_regions_and_region_end(addr)
else:
inside_region = addr < region_end
if not inside_region:
break
# l.debug("Searching address %x", addr)
val0 = self._load_a_byte_as_int(addr)
if val0 is None:
break
@ -1146,8 +1188,6 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
if val1 is None:
break
if val0 == 0 and val1 == 0:
if len(sz) < min_length * 2:
is_sz = False
break
if val0 != 0 and val1 == 0 and val0 in self.PRINTABLES:
sz += [val0, val1]
@ -1157,9 +1197,12 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
is_sz = False
break
if sz and is_sz:
if sz and len(sz) >= min(min_length_nullterminated, min_length_non_nullterminated) * 2:
l.debug("Got a wide-string of %d wide chars", len(sz))
return len(sz) + 2
if is_sz and len(sz) >= min_length_nullterminated * 2:
return len(sz) + 2
if not is_sz and len(sz) >= min_length_non_nullterminated * 2:
return len(sz)
# no wide string is found
return 0
@ -1195,6 +1238,86 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
return repeating_length
return 0
def _scan_for_fp_constants(self, start_addr: int, threshold: int = 4) -> int:
"""
Scan from a given address for a run of plausible floating-point constants.
A double-precision value qualifies when its biased exponent falls within a band covering magnitudes
between 2 ** -64 and 2 ** 64, which is where constants in compiler- and libm-generated tables (polynomial
coefficients, logarithm and trigonometry tables, etc.) almost always live. Code bytes rarely produce
multiple consecutive qualifying values.
Single-precision values are detected as well, but with a tighter magnitude band (2 ** -32 to 2 ** 32) and
twice the run-length requirement: an 8-bit exponent in a 4-byte value is a much weaker signal than an
11-bit exponent in an 8-byte value, and anything looser starts matching real code.
:param start_addr: The address to start scanning from.
:param threshold: The minimum number of consecutive qualifying double-precision values.
:return: The total size in bytes of the qualifying values, or 0 if not enough values are found.
"""
for size, exp_shift, exp_mask, exp_lo, exp_hi, min_count in (
(8, 52, 0x7FF, 959, 1087, threshold), # doubles: 1023 +/- 64
(4, 23, 0xFF, 95, 159, threshold * 2), # floats: 127 +/- 32
):
addr = start_addr
fp_count = 0
first_val = None
has_multiple_values = False
uniform_mul = ((1 << (size * 8)) - 1) // 0xFF
while self._inside_regions(addr):
val = self._fast_memory_load_pointer(addr, size=size)
if val is None:
break
if val == (val & 0xFF) * uniform_mul:
# all bytes are identical: this is filler (e.g., 0xCC padding or "????", whose bit patterns
# carry in-band exponents), not a constant
break
exponent = (val >> exp_shift) & exp_mask
if not exp_lo <= exponent <= exp_hi:
break
if first_val is None:
first_val = val
elif val != first_val:
has_multiple_values = True
fp_count += 1
addr += size
# a run of one repeated value carries no table evidence
if fp_count >= min_count and has_multiple_values:
return fp_count * size
return 0
def _scan_for_monotonic_byte_ramp(self, start_addr: int, threshold: int = 16) -> int:
"""
Scan from a given address for a run of monotonically increasing bytes, where each byte equals the previous
byte plus one, modulo 256. Character case-conversion and translation tables are laid out this way.
:param start_addr: The address to start scanning from.
:param threshold: The minimum run length.
:return: The length of the run, or 0 if the run is shorter than threshold.
"""
addr = start_addr
last_byte = None
ramp_length = 0
while self._inside_regions(addr):
val = self._load_a_byte_as_int(addr)
if val is None:
break
if last_byte is not None and val != (last_byte + 1) & 0xFF:
break
last_byte = val
ramp_length += 1
addr += 1
if ramp_length >= threshold:
return ramp_length
return 0
def _scan_for_consecutive_pointers(self, start_addr: int, threshold: int = 2) -> int:
"""
Scan from a given address and determine if there are at least `threshold` of pointers.
@ -1305,11 +1428,19 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
)
start_addr += pointer_length
elif start_addr <= 0x100000:
# for high addresses, all pointers have been found in _scan_for_consecutive_pointers() because we
# set threshold there to 1
threshold = 4
pointer_count = self._scan_for_mixed_pointers(start_addr, threshold=threshold, window=6)
else:
if start_addr <= 0x100000:
# for low addresses, in-object values are common false positives, so
# _scan_for_consecutive_pointers() ran with a high threshold and may have missed
# non-consecutive pointers; require a high pointer density here
threshold, window = 4, 6
else:
# for high addresses, all consecutive pointers have been found in
# _scan_for_consecutive_pointers() because we set threshold there to 1. what remains are
# interleaved tables (e.g., alternating value-pointer pairs), which have at most window // 2
# pointers; use a wider window with the same evidence requirement
threshold, window = 4, 8
pointer_count = self._scan_for_mixed_pointers(start_addr, threshold=threshold, window=window)
pointer_length = pointer_count * self.project.arch.bytes
if pointer_length:
@ -1321,24 +1452,54 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
start_addr += pointer_length
if not matched_something:
# find strings
# find floating-point constant tables; this must run before the string and repeating-zero scans
# because the low mantissa bytes of table entries are frequently zero or incidentally printable,
# which would misphase the table. since scanning misclassified code often dumps us in the middle
# of a table entry, probe the next 4- and 8-byte boundaries as well.
fp_addr_4 = start_addr + (-start_addr % 4)
fp_addr_8 = start_addr + (-start_addr % 8)
for fp_addr in (fp_addr_4,) if fp_addr_4 == fp_addr_8 else (fp_addr_4, fp_addr_8):
fp_length = self._scan_for_fp_constants(fp_addr)
if fp_length:
matched_something = True
if fp_addr > start_addr:
self._seg_list.occupy(start_addr, fp_addr - start_addr, "alignment")
self.model.memory_data[start_addr] = MemoryData(
start_addr, fp_addr - start_addr, MemoryDataSort.Alignment
)
self._seg_list.occupy(fp_addr, fp_length, "fp")
self.model.memory_data[fp_addr] = MemoryData(fp_addr, fp_length, MemoryDataSort.FloatingPoint)
start_addr = fp_addr + fp_length
break
if not matched_something:
# find strings; tolerate a single leading null byte, which is usually the leftover of a multi-null
# string separator (string scans only consume one null terminator of the preceding string, and a
# single remaining null byte is not caught by the repeating-zero scan below)
leading_nulls = 1 if self._load_a_byte_as_int(start_addr) == 0 else 0
str_addr = start_addr + leading_nulls
is_widestring = False
string_length = self._scan_for_printable_strings(start_addr)
string_length = self._scan_for_printable_strings(str_addr)
if string_length == 0:
is_widestring = True
string_length = self._scan_for_printable_widestrings(start_addr)
string_length = self._scan_for_printable_widestrings(str_addr)
if string_length:
matched_something = True
self._seg_list.occupy(start_addr, string_length, "string")
if leading_nulls:
self._seg_list.occupy(start_addr, leading_nulls, "alignment")
self.model.memory_data[start_addr] = MemoryData(
start_addr, leading_nulls, MemoryDataSort.Alignment
)
self._seg_list.occupy(str_addr, string_length, "string")
md = MemoryData(
start_addr,
str_addr,
string_length,
MemoryDataSort.String if not is_widestring else MemoryDataSort.UnicodeString,
)
md.fill_content(self.project.loader)
self.model.memory_data[start_addr] = md
start_addr += string_length
self.model.memory_data[str_addr] = md
start_addr = str_addr + string_length
if not matched_something and self.project.arch.name in {"X86", "AMD64"}:
cc_length = self._scan_for_repeating_bytes(start_addr, 0xCC, threshold=1)
@ -1378,6 +1539,14 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
)
start_addr += repeating_byte_length
# a long run of monotonically increasing bytes is a character or translation table, not code
ramp_length = self._scan_for_monotonic_byte_ramp(start_addr, threshold=16)
if ramp_length:
matched_something = True
self._seg_list.occupy(start_addr, ramp_length, "nodecode")
self.model.memory_data[start_addr] = MemoryData(start_addr, ramp_length, MemoryDataSort.Unknown)
start_addr += ramp_length
if not matched_something:
# umm now it's probably code
break
@ -1423,28 +1592,7 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
def _nodecode_bytes_ratio(self, cutoff_addr: int, window_size: int) -> float:
if cutoff_addr - 1 < 0:
return 0.0
idx = self._seg_list.search(cutoff_addr - 1)
if idx is None or idx >= len(self._seg_list):
return 0.0
segment = self._seg_list[idx]
if segment.sort != "nodecode":
return 0.0
total_bytes = 0
nodecode_bytes = 0
while idx >= 0:
segment = self._seg_list[idx]
if segment.sort == "nodecode":
nodecode_bytes += segment.size
total_bytes += segment.size
if total_bytes >= window_size:
break
idx -= 1
if total_bytes < window_size:
return 0.0
return nodecode_bytes / total_bytes
return self._seg_list.sort_ratio_backwards(cutoff_addr - 1, window_size, "nodecode")
def _next_code_addr_smart(self) -> int | None:
# in the smart scanning mode, if there are more than N consecutive no-decode cases, we skip an entire window of
@ -1741,6 +1889,7 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
insn = block.capstone.insns[0]
if block.bytes == b"\xff\xe0":
func.info["jmp_rax"] = True
self.kb.functions.add_key_func_addr("jmp_rax", func.addr)
elif (
insn.mnemonic == "jmp"
and insn.operands[0].type == capstone.x86.X86_OP_MEM
@ -1761,12 +1910,14 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
== b"\xff\xe0"
):
func.info["jmp_rax"] = True
self.kb.functions.add_key_func_addr("jmp_rax", func.addr)
elif (
len(func.block_addrs_set) == 2
and func.get_block(jumpout_target).bytes == b"\xff\xe0"
):
# check the second block and ensure it's jmp rax
func.info["jmp_rax"] = True
self.kb.functions.add_key_func_addr("jmp_rax", func.addr)
elif self.project.arch.name == "X86":
func_block_count = self.kb.functions.get_func_block_count(func_addr)
@ -1846,9 +1997,7 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
if not (9 <= len(func.block_addrs_set) < 12):
return
from angr.analyses.decompiler.clinic import ClinicMode # pylint:disable=import-outside-toplevel
clinic = self.project.analyses.Clinic(func, mode=ClinicMode.COLLECT_DATA_REFS)
clinic = self.project.analyses.Clinic(func, mode=angr.analyses.decompiler.ClinicMode.COLLECT_DATA_REFS)
for irsb_addr, refs in clinic.data_refs.items():
self._process_irsb_data_refs(irsb_addr, refs)
@ -2278,6 +2427,149 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
self._model.edges_to_repair = remaining_edges_to_repair
def _repair_guessed_jumptables(self):
"""
The jump targets of some jumptables are guessed purely based on heuristics. We need to make sure that the
guessed jump tables do not overlap with other jump tables.
"""
# first, collect the addresses of all jump tables
jumptable_addr_and_ijs: list[tuple[int, int, IndirectJump]] = []
for block_addr, ij in self.indirect_jumps.items():
if ij.jumptable_addr is not None:
jumptable_addr_and_ijs.append((block_addr, ij.jumptable_addr, ij))
# sort them by jump table address
jumptable_addr_and_ijs.sort(key=lambda x: x[1])
# CFG edges to remove, keyed by source block address
edges_to_remove: defaultdict[int, set[int]] = defaultdict(set)
# check if any guessed jumptables (1) overlap with others, or (2) overlap with data referenced elsewhere.
# we reduce the size of the prior jumptable if necessary
xrefs = self.kb.xrefs
for i in range(len(jumptable_addr_and_ijs) - 1):
block_addr1, addr1, ij1 = jumptable_addr_and_ijs[i]
if not (
ij1.jumptable
and len(ij1.jumptables) == 1
and ij1.jumptables[0].entries_guessed
and ij1.jumptables[0].size is not None
):
# this is not a jump table or is not guessed. skip.
continue
new_size_0: int | None = None
new_size_1: int | None = None
_, addr2, _ = jumptable_addr_and_ijs[i + 1]
# jumptable overlap with data check
stop_addr = xrefs.get_next_xref_addr_by_dst(addr1 + 1)
if stop_addr is not None and addr1 < stop_addr < addr1 + ij1.jumptables[0].size:
new_size_0 = stop_addr - addr1
# jumptables overlap check
# make sure the two jump tables do have the same start address. otherwise we cannot reduce the size of the
# first jump table.
if addr1 != addr2 and addr1 + ij1.jumptables[0].size > addr2:
new_size_1 = max(0, addr2 - addr1)
if new_size_0 is not None or new_size_1 is not None:
new_size = min(
new_size_0 if new_size_0 is not None else ij1.jumptables[0].size,
new_size_1 if new_size_1 is not None else ij1.jumptables[0].size,
)
# update the jumptable entry
ij1.jumptables[0].size = new_size
# record edges to remove
new_table_entry_count = new_size // ij1.jumptables[0].entry_size
dropped_entries = ij1.jumptables[0].entries[new_table_entry_count:]
edges_to_remove[block_addr1] |= set(dropped_entries)
new_entries = ij1.jumptables[0].entries[:new_table_entry_count]
# update the jumptable targets
# TODO: resolved_targets is a set, which means we cannot reliable drop the last N entries, so we
# TODO: cannot support updating the targets of jump tables with custom arithmetic transformations on
# TODO: jumptargets.
if ij1.resolved_targets == set(ij1.jumptables[0].entries):
ij1.resolved_targets = set(new_entries)
ij1.jumptables[0].entries = new_entries
# remove CFG edges
# we don't update function graphs because they will be re-generated in self.make_functions() later
l.debug("Removing %d edges due to jumptable repair...", sum(len(v) for v in edges_to_remove.values()))
for src_addr, dst_addrs in edges_to_remove.items():
src_node = self.model.get_any_node(src_addr)
if src_node is None:
continue
for dst_addr in dst_addrs:
dst_node = self.model.get_any_node(dst_addr)
if dst_node is None:
continue
if self.model.graph.has_edge(src_node, dst_node):
self.model.graph.remove_edge(src_node, dst_node)
def _trivial_jump_target(self, addr: int) -> int | None:
"""
If the function at ``addr`` consists of a single instruction that unconditionally and directly jumps
somewhere (i.e. it "simply jumps to another function and does nothing else"), return the jump target
address; otherwise None.
"""
try:
block = self.project.factory.block(addr)
# accessing .instructions / .vex triggers lazy lifting, which may raise -- keep it inside the try
if block.instructions != 1:
return None
vex = block.vex
except (SimMemoryError, SimTranslationError, SimEngineError):
return None
if vex.jumpkind != "Ijk_Boring":
return None
next_expr = vex.next
if isinstance(next_expr, pyvex.expr.Const):
return next_expr.con.value
return None
def _propagate_key_func_info_to_jump_thunks(self) -> None:
"""
Copy the ``info`` metadata of every key function onto any function that is a trivial jump thunk to it.
"""
functions = self.kb.functions
# Seed from the cached key-function address map (no Function deserialization).
key_func_addrs: set[int] = set()
for addrs in functions.get_key_func_type_and_addrs().values():
key_func_addrs |= addrs
if not key_func_addrs:
return
# Only single-block functions can be a trivial single-instruction jump thunk. Pull their addresses from the
# cached block-count map instead of materializing every Function.
single_block_addrs = functions.get_func_addr_with_block_count(1)
if not single_block_addrs:
return
# Fixpoint: a thunk that inherits key-function markers can itself be the target of another thunk.
changed = True
rounds = 0
while changed and rounds < 8:
changed = False
rounds += 1
for func_addr in single_block_addrs:
if func_addr in key_func_addrs:
continue
target = self._trivial_jump_target(func_addr)
if target is None or target not in key_func_addrs:
continue
src_info = functions.get_by_addr(target, meta_only=True).info
func = functions.get_by_addr(func_addr)
for key, value in src_info.items():
if key not in func.info:
func.info[key] = value
key_func_addrs.add(func_addr)
changed = True
def _post_analysis(self):
self.stage = "Analysis (Stage 2)"
@ -2286,6 +2578,7 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
self.drop_bad_functions()
self._calculate_progress_and_notify(skip_percentage=True)
self._repair_guessed_jumptables()
self._repair_edges()
self._make_completed_functions()
@ -2301,6 +2594,10 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
self._remove_redundant_overlapping_blocks(function_alignment=4, is_arm=True)
self._updated_nonreturning_functions = set()
# Propagate special metadata from key functions to trivial jump thunks that target them.
self._propagate_key_func_info_to_jump_thunks()
# Revisit all edges and rebuild all functions to correctly handle returning/non-returning functions.
self.make_functions()
self._calculate_progress_and_notify(skip_percentage=True)
@ -2946,6 +3243,34 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
return entries
def _resolve_const_folded_next(self, irsb: pyvex.IRSB | None, jumpkind: str) -> int | None:
"""
Check whether the jump/call target of this block was constant-folded from a registered read-only region
at lift time (recorded in IRSB.const_vals), e.g. an import call through the IAT (or delay-load IAT) in a PE
binary. This reproduces the decision of the timeless load-based resolvers (AMD64PeIatResolver and
MemoryLoadResolver) without re-lifting the block or dispatching the resolvers: the loaded pointer is the
same value they would read, and it is accepted when it is a valid jump target (executable or hooked), which
is exactly MemoryLoadResolver's ``_is_target_valid`` criterion (a superset of AMD64PeIatResolver's hooked
check). Everything else falls back to the regular indirect jump resolution logic.
:param irsb: The (possibly statement-less) IRSB of the block.
:param jumpkind: The jumpkind of the default exit.
:return: The resolved target, or None if unavailable.
"""
if not self._fold_ro_const_loads or irsb is None:
return None
if jumpkind not in ("Ijk_Call", "Ijk_Boring"):
return None
if not irsb.const_vals or not isinstance(irsb.next, pyvex.IRExpr.RdTmp):
return None
next_tmp = irsb.next.tmp
for cv in irsb.const_vals:
if cv.tmp == next_tmp:
if self._addr_in_exec_memory_regions(cv.value) or self.project.is_hooked(cv.value):
return cv.value
return None
return None
def _create_jobs(
self,
target: Any,
@ -3031,14 +3356,24 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
jumpkind in ("Ijk_Boring", "Ijk_Call", "Ijk_InvalICache") or jumpkind.startswith("Ijk_Sys")
):
# This is an indirect jump. Try to resolve it.
# FIXME: in some cases, a statementless irsb will be missing its instr addresses
# and this next part will fail. Use the real IRSB instead
irsb = self._lift(cfg_node.addr, size=cfg_node.size).vex
assert irsb is not None
cfg_node.instruction_addrs = InsAddrList.from_addr_list(irsb.instruction_addresses)
resolved, resolved_targets, ij = self._indirect_jump_encountered(
addr, cfg_node, irsb, current_function_addr, stmt_idx
)
# fast path: the target may have been constant-folded from a read-only region at lift time
# (e.g. an AMD64 PE IAT slot); consuming it here avoids re-lifting the block and running the
# indirect jump resolvers
folded_target = self._resolve_const_folded_next(irsb, jumpkind)
if folded_target is not None:
# the statement-less irsb already carries the instruction addresses that the resolver path
# would recompute from a re-lift
cfg_node.instruction_addrs = InsAddrList.from_addr_list(irsb.instruction_addresses)
resolved, resolved_targets, ij = True, {folded_target}, None
else:
# FIXME: in some cases, a statementless irsb will be missing its instr addresses
# and this next part will fail. Use the real IRSB instead
irsb = self._lift(cfg_node.addr, size=cfg_node.size).vex
assert irsb is not None
cfg_node.instruction_addrs = InsAddrList.from_addr_list(irsb.instruction_addresses)
resolved, resolved_targets, ij = self._indirect_jump_encountered(
addr, cfg_node, irsb, current_function_addr, stmt_idx
)
if resolved:
for resolved_target in resolved_targets:
if jumpkind == "Ijk_Call":
@ -3409,7 +3744,10 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
target_func_addr = node.function_address
# case 2: if the source instruction is the first instruction of the current function, has only one branch
# to the target address, and is a jump (Ijk_Boring, not a call), then the target address is likely the
# start of another function
# start of another function. A compiler may also begin a function with an
# unconditional jump to an internal loop guard (loop rotation). When the loader
# supplies a non-empty function symbol, its extent is stronger evidence than this
# tail-jump heuristic: keep a target inside that extent in the current function.
if (
target_func_addr is None
and len(src_node.instruction_addrs) == 1
@ -3418,7 +3756,17 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
and all_successors is not None
and len(all_successors) == 1
):
target_func_addr = target_addr
current_symbol = self.project.loader.find_symbol(current_function_addr)
current_symbol_size = getattr(current_symbol, "size", 0) or 0
target_is_inside_current_symbol = (
current_symbol is not None
and current_symbol.is_function
and current_symbol.rebased_addr == current_function_addr
and current_symbol_size > 0
and current_function_addr <= target_addr < current_function_addr + current_symbol_size
)
if not target_is_inside_current_symbol:
target_func_addr = target_addr
# last resort: the block probably belongs to the current function
if target_func_addr is None:
target_func_addr = current_function_addr
@ -5006,6 +5354,30 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
self._ro_region_cdata_cache.append(content_buf)
pyvex.pvc.register_readonly_region(section.vaddr, section.memsize, content_buf)
elif self.project.arch.name in {"AMD64", "X86"} and isinstance(self.project.simos, SimWindows):
# register sections that hold jump/call targets so that rip-relative import calls and jumps
# (call/jmp qword ptr [rip+disp]) can be constant-folded at lift time and resolved without a re-lift
# and resolver dispatch:
# - non-writable sections (e.g. .rdata, the bound IAT), and
# - the delay-load import table (.didat): although writable, its slots are static during CFG recovery
# and point to the delay-load thunks, which MemoryLoadResolver already resolves by reading them.
# The folded value is only accepted when it is a valid jump target, so registering these regions cannot
# introduce edges the timeless load resolvers would not also produce.
self._ro_region_cdata_cache = []
for section in self.project.loader.main_object.sections:
register = (section.is_readable and not section.is_writable and section.memsize >= 8) or (
section.name == ".didat" and section.is_readable and section.memsize >= 8
)
if register:
try:
content = self.project.loader.memory.load(section.vaddr, section.memsize)
except KeyError:
continue
content_buf = pyvex.ffi.from_buffer(content)
self._ro_region_cdata_cache.append(content_buf)
pyvex.pvc.register_readonly_region(section.vaddr, section.memsize, content_buf)
self._fold_ro_const_loads = bool(self._ro_region_cdata_cache)
def _lifter_deregister_readonly_regions(self):
pyvex.pvc.deregister_all_readonly_regions()
self._ro_region_cdata_cache = None
@ -5269,6 +5641,7 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
collect_data_refs=True,
strict_block_end=True,
load_from_ro_regions=True,
const_prop=self._fold_ro_const_loads,
initial_regs=initial_regs,
)
irsb = lifted_block.vex_nostmt # may raise SimTranslationError
@ -5313,6 +5686,7 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
collect_data_refs=True,
strict_block_end=True,
load_from_ro_regions=True,
const_prop=self._fold_ro_const_loads,
initial_regs=initial_regs,
)
irsb = lifted_block.vex_nostmt
@ -5491,6 +5865,16 @@ class CFGFast(ForwardAnalysis[CFGNode, CFGNode, CFGJob, int, object], CFGBase):
)
irsb_string = irsb_string[: irsb.size]
if (
cfg_job.job_type == CFGJobType.COMPLETE_SCANNING
and addr == current_function_addr
and irsb.jumpkind == "Ijk_NoDecode"
):
# linear sweep decided that this block is undecodable. because drop_bad_functions() will remove this
# function anyway, we bail out early and mark the whole block as nodecode for performance.
self._seg_list.occupy(real_addr, max(irsb.size, 1), "nodecode")
return None, None, None, None
# Occupy the block in segment list
if irsb is not None and irsb.size > 0:
self._seg_list.occupy(real_addr, irsb.size, "code")

View file

@ -1,29 +1,29 @@
from __future__ import annotations
import logging
from collections import defaultdict
from sortedcontainers import SortedDict
from copy import copy
from archinfo.arch_soot import SootMethodDescriptor, SootAddressDescriptor
from archinfo.arch_soot import SootAddressDescriptor, SootMethodDescriptor
from sortedcontainers import SortedDict
from angr.utils.constants import DEFAULT_STATEMENT
from angr.errors import AngrCFGError, SimMemoryError, SimEngineError
from angr.analyses.analysis import register_analysis
from angr.codenode import HookNode, SootBlockNode
from angr.errors import AngrCFGError, SimEngineError, SimMemoryError
from angr.knowledge_plugins.cfg import CFGNode
from angr.analyses import register_analysis
from .cfg_fast import CFGFast, CFGJob, PendingJobs, FunctionTransitionEdge
from angr.utils.constants import DEFAULT_STATEMENT
from .cfg_fast import CFGFast, CFGJob, FunctionTransitionEdge, PendingJobs
l = logging.getLogger(name=__name__)
try:
from pysoot.sootir.soot_value import SootLocal
from pysoot.sootir.soot_statement import IfStmt, InvokeStmt, GotoStmt, AssignStmt
from pysoot.sootir.soot_expr import (
SootStaticInvokeExpr,
SootInvokeExpr,
SootStaticInvokeExpr,
)
from pysoot.sootir.soot_statement import AssignStmt, GotoStmt, IfStmt, InvokeStmt
from pysoot.sootir.soot_value import SootLocal
PYSOOT_INSTALLED = True
except ImportError:
@ -341,7 +341,6 @@ class CFGFastSoot(CFGFast):
addr = cfg_node.addr
stmts_count = cfg_node.size
else:
addr = addr
stmts_count = size
if addr is None:

View file

@ -1,10 +1,11 @@
from __future__ import annotations
import logging
from typing import TYPE_CHECKING
from archinfo.arch_soot import SootAddressDescriptor
from angr.errors import SimValueError, SimSolverModeError
from angr.errors import SimSolverModeError, SimValueError
from angr.state_plugins.callstack import CallStack
if TYPE_CHECKING:

View file

@ -1,17 +1,17 @@
from __future__ import annotations
from .mips_elf_fast import MipsElfFastResolver
from .mips_elf_got import MipsElfGotResolver
from .x86_elf_pic_plt import X86ElfPicPltResolver
from .jumptable import JumpTableResolver
from .x86_pe_iat import X86PeIatResolver
from .aarch64_macho_got import AArch64MachOGotResolver
from .amd64_elf_got import AMD64ElfGotResolver
from .amd64_pe_iat import AMD64PeIatResolver
from .arm_elf_fast import ArmElfFastResolver
from .const_resolver import ConstantResolver
from .amd64_pe_iat import AMD64PeIatResolver
from .jumptable import JumpTableResolver
from .memload_resolver import MemoryLoadResolver
from .mips_elf_fast import MipsElfFastResolver
from .mips_elf_got import MipsElfGotResolver
from .syscall_resolver import SyscallResolver
from .aarch64_macho_got import AArch64MachOGotResolver
from .x86_elf_pic_plt import X86ElfPicPltResolver
from .x86_pe_iat import X86PeIatResolver
__all__ = (
"AArch64MachOGotResolver",

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import logging
import archinfo

View file

@ -1,8 +1,8 @@
from __future__ import annotations
import logging
from capstone.x86_const import X86_REG_RIP
from pyvex.stmt import IMark
from .resolver import IndirectJumpResolver

View file

@ -1,9 +1,11 @@
from __future__ import annotations
import logging
from capstone.x86_const import X86_OP_MEM, X86_REG_RIP
from angr.simos import SimWindows
from .resolver import IndirectJumpResolver
l = logging.getLogger(name=__name__)
@ -23,7 +25,7 @@ class AMD64PeIatResolver(IndirectJumpResolver):
if jumpkind not in {"Ijk_Call", "Ijk_Boring"}:
return False
insns = self.project.factory.block(addr).capstone.insns
insns = self.project.factory.block(addr, size=block.size).capstone.insns
if not insns:
return False
if not insns[-1].insn.operands:
@ -34,7 +36,7 @@ class AMD64PeIatResolver(IndirectJumpResolver):
return bool(opnd.type == X86_OP_MEM and opnd.mem.disp and opnd.mem.base == X86_REG_RIP and opnd.mem.index == 0)
def resolve(self, cfg, addr, func_addr, block, jumpkind, func_graph_complete: bool = True, **kwargs): # pylint:disable=unused-argument
call_insn = self.project.factory.block(addr).capstone.insns[-1].insn
call_insn = self.project.factory.block(addr, size=block.size).capstone.insns[-1].insn
addr = (call_insn.disp + call_insn.address + call_insn.size) & 0xFFFF_FFFF_FFFF_FFFF
target = cfg._fast_memory_load_pointer(addr)
if target is None:

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import logging
import archinfo
@ -6,6 +7,7 @@ import pyvex
from angr.blade import Blade
from angr.utils.constants import DEFAULT_STATEMENT
from .resolver import IndirectJumpResolver
_l = logging.getLogger(name=__name__)

View file

@ -1,18 +1,20 @@
from __future__ import annotations
from typing import TYPE_CHECKING
import logging
from typing import TYPE_CHECKING
import claripy
import pyvex
from angr.analyses.propagator import vex_vars
from angr.blade import Blade
from angr.code_location import CodeLocation
from angr.knowledge_plugins.propagations import PropagationModel
from angr.utils.constants import DEFAULT_STATEMENT
from angr.code_location import CodeLocation
from angr.blade import Blade
from angr.analyses.propagator import vex_vars
from angr.utils.vex import get_tmp_def_stmt
from .resolver import IndirectJumpResolver
from .propagator_utils import PropagatorLoadCallback
from .resolver import IndirectJumpResolver
if TYPE_CHECKING:
from angr import Block

View file

@ -1,17 +1,19 @@
from __future__ import annotations
from typing import TYPE_CHECKING, Any
import logging
from typing import TYPE_CHECKING, Any
import claripy
from angr.code_location import CodeLocation
from angr.project import Project
from angr.analyses.propagator.vex_vars import VEXReg
from angr.code_location import CodeLocation
from .propagator_utils import PropagatorLoadCallback
if TYPE_CHECKING:
from angr import SimState
from angr.knowledge_plugins import Function
from angr.project import Project
l = logging.getLogger(name=__name__)

View file

@ -1,18 +1,22 @@
from __future__ import annotations
import cle
from angr.analyses.cfg.indirect_jump_resolvers import MemoryLoadResolver
from . import MipsElfFastResolver
from . import X86ElfPicPltResolver
from . import JumpTableResolver
from . import X86PeIatResolver
from . import AMD64ElfGotResolver
from . import ConstantResolver
from . import ArmElfFastResolver
from . import AMD64PeIatResolver
from . import MipsElfGotResolver
from . import SyscallResolver
from . import AArch64MachOGotResolver
from . import (
AArch64MachOGotResolver,
AMD64ElfGotResolver,
AMD64PeIatResolver,
ArmElfFastResolver,
ConstantResolver,
JumpTableResolver,
MipsElfFastResolver,
MipsElfGotResolver,
SyscallResolver,
X86ElfPicPltResolver,
X86PeIatResolver,
)
DEFAULT_RESOLVERS = {
"X86": {

View file

@ -1,36 +1,37 @@
# pylint:disable=wrong-import-position,wrong-import-order
from __future__ import annotations
import enum
from typing import TYPE_CHECKING, Literal, cast
from collections.abc import Sequence
from collections import defaultdict, OrderedDict
import logging
import functools
import contextlib
import pyvex
import contextlib
import enum
import functools
import logging
from collections import OrderedDict, defaultdict
from collections.abc import Sequence
from typing import TYPE_CHECKING, Literal, cast
import claripy
import pyvex
from archinfo.arch_arm import is_arm_arch
from claripy.annotation import UninitializedAnnotation
from angr import sim_options as o
from angr import BP, BP_BEFORE, BP_AFTER
from angr.misc.ux import once
from angr.concretization_strategies import SimConcretizationStrategyAny
from angr.knowledge_plugins.cfg import IndirectJump, IndirectJumpType
from angr.engines.vex.claripy import ccall
from angr.engines.light import SimEngineNostmtVEX, SpOffset, RegisterOffset
from angr.errors import AngrError, SimError
from angr.blade import Blade
from angr.annocfg import AnnotatedCFG
from angr.exploration_techniques.slicecutor import Slicecutor
from angr.exploration_techniques.local_loop_seer import LocalLoopSeer
from angr.exploration_techniques.explorer import Explorer
from angr.utils.constants import DEFAULT_STATEMENT
from angr.analyses.propagator.top_checker_mixin import ClaripyDataVEXEngineMixin
from angr.annocfg import AnnotatedCFG
from angr.blade import Blade
from angr.concretization_strategies import SimConcretizationStrategyAny
from angr.engines.light import RegisterOffset, SimEngineNostmtVEX, SpOffset
from angr.engines.vex.claripy import ccall
from angr.engines.vex.claripy.datalayer import value
from .resolver import IndirectJumpResolver
from angr.errors import AngrError, SimError
from angr.exploration_techniques.explorer import Explorer
from angr.exploration_techniques.local_loop_seer import LocalLoopSeer
from angr.exploration_techniques.slicecutor import Slicecutor
from angr.knowledge_plugins.cfg import IndirectJump, IndirectJumpType
from angr.misc.ux import once
from angr.state_plugins.inspect import BP, BP_AFTER, BP_BEFORE
from angr.utils.constants import DEFAULT_STATEMENT
from .constant_value_manager import ConstantValueManager
from .resolver import IndirectJumpResolver
try:
from angr.engines import pcode
@ -1075,6 +1076,7 @@ class JumpTableResolver(IndirectJumpResolver):
# Get the jumping targets
for r in simgr.found:
jt2, jt2_addr, jt2_entrysize, jt2_size = None, None, None, None
entries_guessed = False
if load_stmt is not None:
ret = self._try_resolve_targets_load(
r,
@ -1101,6 +1103,7 @@ class JumpTableResolver(IndirectJumpResolver):
jt2_addr,
jt2_entrysize,
jt2_size,
entries_guessed,
) = ret
if sort == "jumptable":
ij_type = IndirectJumpType.Jumptable_AddressLoadedFromMemory
@ -1135,10 +1138,11 @@ class JumpTableResolver(IndirectJumpResolver):
all_targets = [t_ for t_ in all_targets if t_ % alignment == 0]
l.info(
"Jump table at %#x has %d targets: %s",
"Jump table at %#x has %d targets: %s (guessed: %s)",
addr,
len(all_targets),
", ".join([hex(a) for a in all_targets]),
entries_guessed,
)
# write to the IndirectJump object in CFG
@ -1153,7 +1157,14 @@ class JumpTableResolver(IndirectJumpResolver):
ij.jumptable = True
else:
ij.jumptable = False
ij.add_jumptable(jumptable_addr, jumptable_size, entry_size, jump_table, is_primary=True)
ij.add_jumptable(
jumptable_addr,
jumptable_size,
entry_size,
jump_table,
is_primary=True,
entries_guessed=entries_guessed,
)
ij.resolved_targets = set(jump_table)
ij.type = ij_type
else:
@ -1807,11 +1818,16 @@ class JumpTableResolver(IndirectJumpResolver):
if table_base_addr is not None:
addr = table_base_addr
# Stop at the immediate next referenced data address
stop_addr = cfg.kb.xrefs.get_next_xref_addr_by_dst(table_base_addr + 1)
# FIXME: May want to support NULL targets for handlers that are not filled in / placeholders
# FIXME: Try negative offsets too? (this would be unusual)
l.debug("Inspecting table at %#x for plausible targets...", addr)
for i in range(self._max_targets):
target = cfg._fast_memory_load_pointer(addr, size=load_size)
if stop_addr is not None and addr >= stop_addr:
l.debug("Reached the next referenced data address %#x. Stop scanning.", stop_addr)
break
if target is None or not self._is_jumptarget_legal(target):
break
l.debug("- %#x[%d] -> %#x", table_base_addr, i, target)
@ -1840,6 +1856,7 @@ class JumpTableResolver(IndirectJumpResolver):
None,
None,
None,
True,
)
# We resolved too many targets for this indirect jump. Something might have gone wrong.
@ -2045,6 +2062,7 @@ class JumpTableResolver(IndirectJumpResolver):
jt_2nd_baseaddr,
jt_2nd_entrysize,
jt_2nd_size,
False,
)
def _try_resolve_targets_ite(self, r, addr, cfg, annotatedcfg, ite_stmt: pyvex.IRStmt.WrTmp): # pylint:disable=unused-argument
@ -2107,16 +2125,27 @@ class JumpTableResolver(IndirectJumpResolver):
:return: None
"""
def _scratch_gated(hook, block_addr, stmt_idx):
def _action(_s):
if _s.scratch.bbl_addr == block_addr and _s.scratch.stmt_idx == stmt_idx:
hook(_s)
return _action
def _statement_gated(hook, block_addr, stmt_idx):
def _action(_s):
if _s.scratch.bbl_addr == block_addr and _s.inspect.attrs.statement == stmt_idx:
hook(_s)
return _action
for sort, block_addr, stmt_idx in stmts_to_instrument:
l.debug("Add a %s hook to overwrite memory/register values at %#x:%d.", sort, block_addr, stmt_idx)
if sort == "mem_write":
bp = BP(
when=BP_BEFORE,
enabled=True,
action=StoreHook.hook,
condition=lambda _s, a=block_addr, idx=stmt_idx: (
_s.scratch.bbl_addr == a and _s.scratch.stmt_idx == idx
),
action=_scratch_gated(StoreHook.hook, block_addr, stmt_idx),
)
state.inspect.add_breakpoint("mem_write", bp)
elif sort == "mem_read":
@ -2124,29 +2153,20 @@ class JumpTableResolver(IndirectJumpResolver):
bp0 = BP(
when=BP_BEFORE,
enabled=True,
action=hook.hook_before,
condition=lambda _s, a=block_addr, idx=stmt_idx: (
_s.scratch.bbl_addr == a and _s.scratch.stmt_idx == idx
),
action=_scratch_gated(hook.hook_before, block_addr, stmt_idx),
)
state.inspect.add_breakpoint("mem_read", bp0)
bp1 = BP(
when=BP_AFTER,
enabled=True,
action=hook.hook_after,
condition=lambda _s, a=block_addr, idx=stmt_idx: (
_s.scratch.bbl_addr == a and _s.scratch.stmt_idx == idx
),
action=_scratch_gated(hook.hook_after, block_addr, stmt_idx),
)
state.inspect.add_breakpoint("mem_read", bp1)
elif sort == "reg_write":
bp = BP(
when=BP_BEFORE,
enabled=True,
action=PutHook.hook,
condition=lambda _s, a=block_addr, idx=stmt_idx: (
_s.scratch.bbl_addr == a and _s.scratch.stmt_idx == idx
),
action=_scratch_gated(PutHook.hook, block_addr, stmt_idx),
)
state.inspect.add_breakpoint("reg_write", bp)
else:
@ -2154,9 +2174,6 @@ class JumpTableResolver(IndirectJumpResolver):
reg_val = 0x13370000
def bp_condition(block_addr, stmt_idx, _s):
return _s.scratch.bbl_addr == block_addr and _s.inspect.attrs.statement == stmt_idx
for block_addr, stmt_idx, reg_offset, reg_bits in regs_to_initialize:
l.debug(
"Add a hook to initialize register %s at %x:%d.",
@ -2167,8 +2184,11 @@ class JumpTableResolver(IndirectJumpResolver):
bp = BP(
when=BP_BEFORE,
enabled=True,
action=RegisterInitializerHook(reg_offset, reg_bits, reg_val).hook,
condition=functools.partial(bp_condition, block_addr, stmt_idx),
action=_statement_gated(
RegisterInitializerHook(reg_offset, reg_bits, reg_val).hook,
block_addr,
stmt_idx,
),
)
state.inspect.add_breakpoint("statement", bp)
reg_val += 16

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import logging
import pyvex

View file

@ -1,15 +1,16 @@
# pylint:disable=too-many-boolean-expressions,global-statement,too-many-positional-arguments
from __future__ import annotations
from typing import TYPE_CHECKING
import logging
from enum import Enum
from typing import TYPE_CHECKING
import archinfo
import pyvex
from angr.blade import Blade
from angr.utils.constants import DEFAULT_STATEMENT
from .resolver import IndirectJumpResolver
if TYPE_CHECKING:

View file

@ -1,19 +1,19 @@
# pylint:disable=too-many-positional-arguments
from __future__ import annotations
import logging
import cle
from capstone.mips_const import (
MIPS_OP_IMM,
MIPS_OP_REG,
MIPS_REG_RA,
MIPS_REG_T7,
MIPS_REG_T8,
MIPS_REG_T9,
MIPS_REG_RA,
MIPS_REG_ZERO,
MIPS_OP_REG,
MIPS_OP_IMM,
)
import cle
from .resolver import IndirectJumpResolver
l = logging.getLogger(name=__name__)

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import claripy

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import typing
from angr.errors import SimMemoryError

View file

@ -1,24 +1,25 @@
from __future__ import annotations
import contextlib
from typing import TYPE_CHECKING
import logging
from typing import TYPE_CHECKING
from angr import sim_options as o
from angr import BP, BP_AFTER
from angr.errors import (
AngrUnsupportedSyscallError,
SimOperationError,
SimError,
SimOperationError,
)
from angr.state_plugins.inspect import BP, BP_AFTER
from .resolver import IndirectJumpResolver
from .constant_value_manager import ConstantValueManager
from .resolver import IndirectJumpResolver
if TYPE_CHECKING:
from angr import Block
from angr.engines import SimSuccessors
from angr.sim_state import SimState
from angr.sim_procedure import SimProcedure
from angr.sim_state import SimState
_l = logging.getLogger(name=__name__)

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import logging
import archinfo

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import logging
from capstone.x86_const import X86_OP_MEM

View file

@ -14,10 +14,6 @@ from __future__ import annotations
import logging
import struct
from typing import TYPE_CHECKING
if TYPE_CHECKING:
pass
log = logging.getLogger(__name__)

View file

@ -1,7 +1,7 @@
from __future__ import annotations
from .graph import slice_callgraph, slice_cfg_graph, slice_function_graph
from .cfg_slice_to_sink import CFGSliceToSink
from .graph import slice_callgraph, slice_cfg_graph, slice_function_graph
__all__ = (
"CFGSliceToSink",

View file

@ -1,6 +1,7 @@
from __future__ import annotations
from typing import Any
from functools import reduce
from typing import Any
from .transitions import merge_transitions

View file

@ -1,9 +1,10 @@
from __future__ import annotations
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.analyses.cfg import CFGFast
from angr.analyses.vtable import VtableFinder
from angr.sim_type import SimCppClass, SimTypeCppFunction
from angr.analyses import AnalysesHub
from angr.utils.cpp import is_cpp_funcname_ctor
from . import Analysis, CFGFast, VtableFinder
class ClassIdentifier(Analysis):

View file

@ -1,9 +1,10 @@
from __future__ import annotations
import pyvex
from angr.utils import looks_like_sql
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.knowledge_plugins.xrefs import XRef
from . import Analysis, AnalysesHub
from angr.utils import looks_like_sql
class CodeTags:

View file

@ -1,10 +1,11 @@
from __future__ import annotations
import logging
from dataclasses import dataclass
from enum import Enum, auto
from typing import TYPE_CHECKING
from dataclasses import dataclass
from angr.analyses import Analysis, AnalysesHub
from angr.analyses.analysis import AnalysesHub, Analysis
if TYPE_CHECKING:
from angr.knowledge_plugins import Function

View file

@ -1,36 +1,44 @@
# pylint:disable=import-outside-toplevel
from __future__ import annotations
from typing import TYPE_CHECKING
from collections.abc import Callable, Iterable
import logging
import queue
import threading
import time
import logging
from collections import defaultdict
from collections.abc import Callable, Iterable
from enum import Enum
import networkx
from typing import TYPE_CHECKING
import claripy
import networkx
from angr.utils.graph import GraphUtils
from angr.simos import SimWindows
from angr.utils.mp import mp_context, Initializer
from angr.analyses.analysis import Analysis, register_analysis
from angr.analyses.calling_convention import CallingConventionAnalysis
from angr.analyses.cfg import CFGFast
from angr.analyses.variable_recovery import VariableRecoveryFast
from angr.errors import AngrRuntimeError
from angr.knowledge_plugins.cfg import CFGModel
from angr.knowledge_plugins.functions.function import PrototypeSource
from . import Analysis, register_analysis, VariableRecoveryFast, CallingConventionAnalysis, CFGFast
from angr.simos import SimWindows
from angr.utils.graph import GraphUtils
from angr.utils.mp import Initializer, mp_context
if TYPE_CHECKING:
from angr.calling_conventions import SimCC
from angr.sim_type import SimTypeFunction
from angr.knowledge_plugins.variables.variable_manager import VariableManagerInternal
from angr.knowledge_plugins.functions.function_manager import Function
from angr.knowledge_plugins.variables.variable_manager import VariableManagerInternal
from angr.sim_type import SimTypeFunction
_l = logging.getLogger(name=__name__)
_mp_context = mp_context()
# How long (in seconds) the result collector keeps polling the results queue after it observes that every worker
# process has exited.
DEAD_WORKER_GRACE_PERIOD = 5.0
class CallingConventionAnalysisMode(Enum):
"""
@ -292,6 +300,9 @@ class CompleteCallingConventionsAnalysis(Analysis):
self._update_progress(0)
idx = 0
assert self._results_lock is not None
# the timestamp when we first noticed that all workers had exited.
# None while at least one worker is still alive.
all_workers_dead_since: float | None = None
while idx < total_funcs:
try:
with self._results_lock:
@ -299,9 +310,26 @@ class CompleteCallingConventionsAnalysis(Analysis):
True, timeout=0.01
)
except queue.Empty:
# No result is available right now.
# If all workers have exited then no result will ever become available, and looping here would
# hang forever (angr #6529). Detect that and fail loudly.
if any(proc.is_alive() for proc in procs):
all_workers_dead_since = None
elif all_workers_dead_since is None:
all_workers_dead_since = time.time()
elif time.time() - all_workers_dead_since >= DEAD_WORKER_GRACE_PERIOD:
exitcodes = ", ".join(f"{proc.name}: {proc.exitcode}" for proc in procs)
raise AngrRuntimeError(
f"All {len(procs)} CompleteCallingConventions worker processes exited before the "
f"analysis finished; only {idx} of {total_funcs} functions were analyzed. Worker exit "
f"codes: {exitcodes}."
) from None
time.sleep(0.1)
continue
# we made progress, so any previously observed all-dead state is no longer interesting
all_workers_dead_since = None
func = self.kb.functions.get_by_addr(func_addr)
if cc is not None or proto is not None:
func.calling_convention = cc
@ -373,8 +401,13 @@ class CompleteCallingConventionsAnalysis(Analysis):
except Exception: # pylint:disable=broad-except
_l.error("Worker %d: Exception occurred during _analyze_core().", worker_id, exc_info=True)
cc, proto, proto_libname, proto_source, varman = None, None, None, None, None
with self._results_lock:
self._results.put((func_addr, cc, proto, proto_libname, proto_source, varman))
try:
with self._results_lock:
self._results.put((func_addr, cc, proto, proto_libname, proto_source, varman))
except Exception: # pylint:disable=broad-except
_l.error(
"Worker %d: Failed to report the result for function %#x.", worker_id, func_addr, exc_info=True
)
def _analyze_core(
self, func_addr: int

View file

@ -1,11 +1,11 @@
from __future__ import annotations
import logging
import claripy
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.errors import AngrIncongruencyError
from angr.analyses import AnalysesHub
from . import Analysis
l = logging.getLogger(name=__name__)

View file

@ -1,7 +1,7 @@
from __future__ import annotations
from .data_dependency_analysis import DataDependencyGraphAnalysis
from .dep_nodes import DepNodeTypes, BaseDepNode, VarDepNode, MemDepNode, ConstantDepNode, TmpDepNode, RegDepNode
from .dep_nodes import BaseDepNode, ConstantDepNode, DepNodeTypes, MemDepNode, RegDepNode, TmpDepNode, VarDepNode
__all__ = (
"BaseDepNode",

View file

@ -6,22 +6,23 @@ import logging
import math
from typing import TYPE_CHECKING
from networkx import DiGraph
import claripy
from claripy.ast import BV
from .dep_nodes import DepNodeTypes, ConstantDepNode, MemDepNode, VarDepNode, RegDepNode, TmpDepNode
from .sim_act_location import SimActLocation, DEFAULT_LOCATION, ParsedInstruction
from angr.analyses import Analysis
from angr.analyses import AnalysesHub
from angr.errors import AngrDDGError, AngrAnalysisError, SimValueError
from angr.state_plugins import SimActionData
from networkx import DiGraph
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.errors import AngrAnalysisError, AngrDDGError, SimValueError
from angr.state_plugins.sim_action import SimActionData
from angr.storage import DefaultMemory
from .dep_nodes import ConstantDepNode, DepNodeTypes, MemDepNode, RegDepNode, TmpDepNode, VarDepNode
from .sim_act_location import DEFAULT_LOCATION, ParsedInstruction, SimActLocation
if TYPE_CHECKING:
from .dep_nodes import BaseDepNode
from angr import SimState
from .dep_nodes import BaseDepNode
logger = logging.getLogger(name=__name__)

View file

@ -1,9 +1,11 @@
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from claripy.ast.bv import BV
from angr.state_plugins import SimActionData
from angr.state_plugins.sim_action import SimActionData
class DepNodeTypes:

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import logging
from angr.sim_procedure import SimProcedure

View file

@ -1,4 +1,5 @@
from __future__ import annotations
import logging
from collections import defaultdict
@ -6,15 +7,15 @@ import claripy
import networkx
import pyvex
from angr.analyses import Analysis, AnalysesHub
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.code_location import CodeLocation
from angr.errors import SimSolverModeError, SimUnsatError, AngrDDGError
from angr.errors import AngrDDGError, SimSolverModeError, SimUnsatError
from angr.sim_variable import (
SimRegisterVariable,
SimMemoryVariable,
SimTemporaryVariable,
SimConstantVariable,
SimMemoryVariable,
SimRegisterVariable,
SimStackVariable,
SimTemporaryVariable,
)
l = logging.getLogger(name=__name__)
@ -755,7 +756,7 @@ class DDG(Analysis):
matched = True
except (SimUnsatError, SimSolverModeError, ZeroDivisionError):
# ignore
matched = matched
pass
if not matched:
break
@ -1016,7 +1017,7 @@ class DDG(Analysis):
self._stmt_graph_annotate_edges(self._register_edges[reg_offset], subtype="mem_addr")
reg_variable = SimRegisterVariable(reg_offset, self._get_register_size(reg_offset))
prev_defs = self._def_lookup(reg_variable)
for loc, _ in prev_defs.items():
for loc in prev_defs:
v = ProgramVariable(reg_variable, loc, arch=self.project.arch)
self._data_graph_add_edge(v, prog_var, type="mem_addr")
@ -1038,7 +1039,7 @@ class DDG(Analysis):
self._stmt_graph_annotate_edges(self._register_edges[reg_offset], subtype="mem_data")
reg_variable = SimRegisterVariable(reg_offset, self._get_register_size(reg_offset))
prev_defs = self._def_lookup(reg_variable)
for loc, _ in prev_defs.items():
for loc in prev_defs:
v = ProgramVariable(reg_variable, loc, arch=self.project.arch)
self._data_graph_add_edge(v, prog_var, type="mem_data")
@ -1155,9 +1156,11 @@ class DDG(Analysis):
if not action.reg_deps and not action.tmp_deps:
# moving a constant into the register
# try to parse out the constant from statement
const_variable = SimConstantVariable(size=1)
if statement is not None and isinstance(statement.data, pyvex.IRExpr.Const):
const_variable = SimConstantVariable(value=statement.data.con.value, size=statement.data.con.size)
else:
# use a default value of 0 if we cannot find the constant
const_variable = SimConstantVariable(1, value=0)
const_pv = ProgramVariable(const_variable, location, arch=self.project.arch)
self._data_graph_add_edge(const_pv, pv)
@ -1228,7 +1231,7 @@ class DDG(Analysis):
if not action.tmp_deps and not self._variables_per_statement and not ast:
# read in a constant
# try to parse out the constant from statement
const_variable = SimConstantVariable(size=1)
const_variable = SimConstantVariable(size=1, value=0) # default value if we can't find the constant
if statement is not None:
if isinstance(statement, pyvex.IRStmt.Dirty):
l.warning("Dirty statements are not supported in DDG for now.")
@ -1496,7 +1499,7 @@ class DDG(Analysis):
# Group all dependencies first
block_addr_to_func = {}
for _, func in self.kb.functions.items():
for func in self.kb.functions.values():
for block in func.blocks:
block_addr_to_func[block.addr] = func

View file

@ -1,19 +1,21 @@
from __future__ import annotations
from .region_identifier import RegionIdentifier
from .structured_codegen import CStructuredCodeGenerator, ImportSourceCode
from .clinic import Clinic
from .region_simplifiers import RegionSimplifier
from .decompiler import Decompiler
from .decompilation_options import options, options_by_category
from . import optimization_passes, structuring
from .ail_simplifier import AILSimplifier
from .block_simplifier import BlockSimplifier
from .callsite_maker import CallSiteMaker
from .ail_simplifier import AILSimplifier
from .ssailification import Ssailification
from .clinic import Clinic, ClinicMode
from .decompilation_cache import DecompilationCache
from .decompilation_options import options, options_by_category
from .decompiler import Decompiler
from .dephication import GraphDephication, SeqNodeDephication
from .presets import DECOMPILATION_PRESETS
from . import structuring
from . import optimization_passes
from .region_identifier import RegionIdentifier
from .region_overlay import RegionOverlay
from .region_simplifiers import RegionSimplifier
from .ssailification import Ssailification
from .structured_codegen import BaseStructuredCodeGenerator, CStructuredCodeGenerator, ImportSourceCode
from .variable_map import VariableMap
StructuredCodeGenerator = CStructuredCodeGenerator
@ -21,18 +23,23 @@ StructuredCodeGenerator = CStructuredCodeGenerator
__all__ = (
"DECOMPILATION_PRESETS",
"AILSimplifier",
"BaseStructuredCodeGenerator",
"BlockSimplifier",
"CStructuredCodeGenerator",
"CallSiteMaker",
"Clinic",
"ClinicMode",
"DecompilationCache",
"Decompiler",
"GraphDephication",
"ImportSourceCode",
"RegionIdentifier",
"RegionOverlay",
"RegionSimplifier",
"SeqNodeDephication",
"Ssailification",
"StructuredCodeGenerator",
"VariableMap",
"optimization_passes",
"options",
"options_by_category",

View file

@ -1,69 +1,73 @@
# pylint:disable=too-many-boolean-expressions,consider-using-enumerate
from __future__ import annotations
from typing import Any, TYPE_CHECKING
from collections.abc import Container
from collections.abc import Iterable
from collections import defaultdict
from enum import Enum
import logging
import os
from collections import defaultdict
from collections.abc import Iterable
from enum import Enum
from typing import TYPE_CHECKING, Any
import networkx
from angr.ailment import AILBlockRewriter, AILBlockViewer, Address
from angr.ailment import Address, AILBlockRewriter, AILBlockViewer
from angr.ailment.block import Block
from angr.ailment.expression import (
BinaryOp,
Call,
Const,
Convert,
DirtyExpression,
Expression,
FunctionLikeMacro,
Insert,
Load,
Register,
StackBaseOffset,
Tmp,
UnaryOp,
VEXCCallExpression,
VirtualVariable,
)
from angr.ailment.statement import (
Statement,
Assignment,
Store,
SideEffectStatement,
ConditionalJump,
DirtyStatement,
WeakAssignment,
NoOp,
Return,
SideEffectStatement,
Statement,
Store,
WeakAssignment,
)
from angr.ailment.expression import (
Call,
Insert,
Register,
Convert,
Load,
StackBaseOffset,
Expression,
DirtyExpression,
VEXCCallExpression,
Tmp,
Const,
BinaryOp,
VirtualVariable,
UnaryOp,
FunctionLikeMacro,
)
from angr.analyses.s_propagator import SPropagatorAnalysis
from angr.analyses.s_reaching_definitions import SRDAModel, SReachingDefinitionsAnalysis
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.analyses.s_propagator import SPropagator
from angr.analyses.s_reaching_definitions import SRDAModel, SReachingDefinitions
from angr.code_location import AILCodeLocation
from angr.errors import AngrRuntimeError
from angr.knowledge_plugins.functions.function import Function
from angr.utils.ail import is_phi_assignment, HasExprWalker, is_expr_used_as_reg_base_value
from angr.knowledge_plugins.key_definitions import atoms
from angr.knowledge_plugins.key_definitions.constants import OP_BEFORE
from angr.knowledge_plugins.key_definitions.definition import Definition
from angr.knowledge_plugins.propagations.states import Equivalence
from angr.sim_variable import SimMemoryVariable, SimStackVariable, SimVariable
from angr.utils.ail import HasExprWalker, is_expr_used_as_reg_base_value, is_phi_assignment
from angr.utils.ssa import (
has_call_in_between_stmts,
has_store_stmt_in_between_stmts,
has_load_expr_in_between_stmts,
has_store_stmt_in_between_stmts,
is_vvar_eliminatable,
)
from angr.code_location import AILCodeLocation
from angr.sim_variable import SimStackVariable, SimMemoryVariable, SimVariable
from angr.knowledge_plugins.propagations.states import Equivalence
from angr.knowledge_plugins.key_definitions import atoms
from angr.knowledge_plugins.key_definitions.definition import Definition
from angr.knowledge_plugins.key_definitions.constants import OP_BEFORE
from angr.errors import AngrRuntimeError
from angr.analyses import Analysis, AnalysesHub
from angr.utils.timing import timethis
from .ailgraph_walker import AILGraphWalker
from .expression_narrower import ExprNarrowingInfo, EffectiveSizeExtractor, ExpressionNarrower
from .block_simplifier import BlockSimplifier
from .block_walkers import HasCallExprWalker, HasCallNotification
from .ccall_rewriters import CCALL_REWRITERS
from .dirty_rewriters import DIRTY_REWRITERS
from .counters.expression_counters import SingleExpressionCounter
from .dirty_rewriters import DIRTY_REWRITERS
from .expression_narrower import EffectiveSizeExtractor, ExpressionNarrower, ExprNarrowingInfo
from .stackarg_offset_manager import StackArgOffsetManager
if TYPE_CHECKING:
from angr.ailment.manager import Manager
@ -71,11 +75,61 @@ if TYPE_CHECKING:
_l = logging.getLogger(__name__)
# When enabled (env var VERIFY_INCREMENTAL_RD), every incremental reaching-definitions update is checked against a
# full rebuild. Used to validate the incremental update; off by default because the full rebuild defeats its purpose.
_VERIFY_INCREMENTAL_RD = os.environ.get("VERIFY_INCREMENTAL_RD", "").lower() not in {"", "0", "no", "false"}
class HasCallNotification(Exception):
def _strongly_connected_components(succs: dict[int, set[int]]):
"""
Notifies the existence of a call statement.
Iterative Tarjan SCC over a plain adjacency map. Yields sets of node IDs, like
networkx.strongly_connected_components().
"""
index_of: dict[int, int] = {}
lowlink: dict[int, int] = {}
on_stack: set[int] = set()
stack: list[int] = []
counter = 0
for root, root_succs in succs.items():
if root in index_of:
continue
work = [(root, iter(root_succs))]
index_of[root] = lowlink[root] = counter
counter += 1
stack.append(root)
on_stack.add(root)
while work:
node, it = work[-1]
advanced = False
for succ in it:
if succ not in index_of:
index_of[succ] = lowlink[succ] = counter
counter += 1
stack.append(succ)
on_stack.add(succ)
work.append((succ, iter(succs[succ])))
advanced = True
break
if succ in on_stack and index_of[succ] < lowlink[node]:
lowlink[node] = index_of[succ]
if advanced:
continue
work.pop()
if work:
parent = work[-1][0]
lowlink[parent] = min(lowlink[parent], lowlink[node])
if lowlink[node] == index_of[node]:
scc = set()
while True:
member = stack.pop()
on_stack.discard(member)
scc.add(member)
if member == node:
break
yield scc
class HasVVarNotification(Exception):
@ -84,6 +138,9 @@ class HasVVarNotification(Exception):
"""
_HAS_CALL_EXPRS_WALKER = HasCallExprWalker()
class HasRefVVarNotification(Exception):
"""
Notifies the existence of a reference to a VirtualVariable.
@ -149,8 +206,8 @@ class PartialConstantExprRewriter(AILBlockRewriter):
if new_mask == mask:
return expr
if new_mask == 0:
return Const(expr_idx, None, 0, expr.bits, **expr.tags)
new_mask_expr = Const(mask_expr.idx, mask_expr.variable, new_mask, mask_expr.bits, **mask_expr.tags)
return Const(expr_idx, 0, expr.bits, **expr.tags)
new_mask_expr = Const(mask_expr.idx, new_mask, mask_expr.bits, **mask_expr.tags)
return BinaryOp(expr_idx, expr.op, [vvar, new_mask_expr], bits=expr.bits, **expr.tags)
return super()._handle_BinaryOp(expr_idx, expr, stmt_idx, stmt, block)
@ -166,7 +223,7 @@ class AILSimplifier(Analysis):
func_graph: networkx.DiGraph[Block],
ail_manager: Manager,
remove_dead_memdefs=False,
stack_arg_offsets: set[tuple[int, int]] | None = None,
stackarg_offset_manager: StackArgOffsetManager | None = None,
unify_variables=False,
gp: int | None = None,
narrow_expressions=False,
@ -184,10 +241,10 @@ class AILSimplifier(Analysis):
self.func = func
self.func_graph = func_graph
self._reaching_definitions: SRDAModel | None = None
self._propagator: SPropagatorAnalysis | None = None
self._propagator: SPropagator | None = None
self._remove_dead_memdefs = remove_dead_memdefs
self._stack_arg_offsets = stack_arg_offsets
self._stackarg_offset_manager = stackarg_offset_manager
self._unify_vars = unify_variables
self._ail_manager = ail_manager
self._gp = gp
@ -203,12 +260,20 @@ class AILSimplifier(Analysis):
self._arg_vvars = arg_vvars
self._avoid_vvar_ids = avoid_vvar_ids if avoid_vvar_ids is not None else set()
self._propagator_dead_vvar_ids: set[int] = set()
# per-block cache of dirty/ccall-defined vvar IDs, keyed by block key, validated by block identity
self._dirty_vvar_scan_cache: dict[tuple[int, int | None], tuple[Block, set[int]]] = {}
# only set to True when any simplification pass has modified the graph or updated any blocks.
# skips _remove_dead_assignments if this flag is False.
self._should_eliminate_dead_assignments: bool = True
self._calls_to_remove: set[AILCodeLocation] = set()
self._assignments_to_remove: set[AILCodeLocation] = set()
self.blocks: dict[Block, Block] = {} # Mapping nodes to simplified blocks
self.simplified: bool = False
# (addr, idx) of every block modified during simplification. Lets callers re-simplify only the blocks that
# actually changed instead of the whole graph.
self.simplified_blocks: set[tuple[int, int | None]] = set()
self._simplify()
def _simplify(self):
@ -311,48 +376,61 @@ class AILSimplifier(Analysis):
_l.debug("... dead assignments removed")
self.simplified = True
# Dead-assignment removal leaves NoOp placeholders in the graph (so reaching definitions could be updated
# incrementally instead of rebuilt between iterations). All simplification steps are done now, so compact them
# away. The reaching-definitions result is not needed past this point, so just invalidate it.
if self._compact_noop_statements():
self._clear_cache()
def _rebuild_func_graph(self):
def _handler(node):
return self.blocks.get(node, None)
new_block = self.blocks.get(node, None)
if new_block is not None:
# every block modification funnels through here, so this captures all dirty blocks
self.simplified_blocks.add((new_block.addr, new_block.idx))
return new_block
AILGraphWalker(self.func_graph, _handler, replace_nodes=True).walk()
self.blocks = {}
self._should_eliminate_dead_assignments = True
def _compute_reaching_definitions(self) -> SRDAModel:
# Computing reaching definitions or return the cached one
if self._reaching_definitions is not None:
return self._reaching_definitions
func_args = {vvar for vvar, _ in self._arg_vvars.values()} if self._arg_vvars else set()
rd = (
self.project.analyses[SReachingDefinitionsAnalysis]
.prep()(
subject=self.func,
func_graph=self.func_graph,
func_args=func_args,
use_callee_saved_regs_at_return=self._use_callee_saved_regs_at_return,
# track_tmps=True,
)
.model
)
rd = SReachingDefinitions(
self.project,
subject=self.func,
func_graph=self.func_graph,
func_args=func_args,
use_callee_saved_regs_at_return=self._use_callee_saved_regs_at_return,
# track_tmps=True,
).model
self._reaching_definitions = rd
return rd
@timethis
def _compute_propagation(self) -> SPropagatorAnalysis:
def _compute_propagation(self) -> SPropagator:
# Propagate expressions or return the existing result
if self._propagator is not None:
return self._propagator
func_args = {vvar for vvar, _ in self._arg_vvars.values()} if self._arg_vvars else set()
prop = self.project.analyses[SPropagatorAnalysis].prep(fail_fast=self._fail_fast)(
prop = SPropagator(
self.project,
subject=self.func,
func_graph=self.func_graph,
func_args=func_args,
# gp=self._gp,
only_consts=self._only_consts,
stack_arg_offsets={x for _, x in self._stack_arg_offsets} if self._stack_arg_offsets is not None else None,
stack_arg_offsets=self._stackarg_offset_manager.get_stackarg_offsets()
if self._stackarg_offset_manager is not None
else None,
ail_manager=self._ail_manager,
)
self._propagator = prop
self._propagator_dead_vvar_ids = prop.dead_vvar_ids
self._should_eliminate_dead_assignments = True
return prop
@timethis
@ -434,6 +512,10 @@ class AILSimplifier(Analysis):
# compute effective sizes for each vvar
effective_sizes = self._compute_effective_sizes(rd, sorted_defs, addr_and_idx_to_block)
# per-statement EffectiveSizeExtractor cache; the blocks do not change while we collect narrowing candidates,
# so one walk per statement serves the queries of all definitions
extractor_cache: dict[AILCodeLocation, EffectiveSizeExtractor] = {}
narrowing_candidates: dict[int, tuple[Definition, ExprNarrowingInfo]] = {}
for def_ in sorted_defs:
if isinstance(def_.atom, atoms.VirtualVariable) and (def_.atom.was_reg or def_.atom.was_parameter):
@ -449,7 +531,7 @@ class AILSimplifier(Analysis):
if skip_def:
continue
narrow = self._narrowing_needed(def_, rd, addr_and_idx_to_block, effective_sizes)
narrow = self._narrowing_needed(def_, rd, addr_and_idx_to_block, effective_sizes, extractor_cache)
if narrow.narrowable:
# we cannot narrow it immediately because any definition that is used by phi variables must be
# narrowed together with all other definitions that can reach the phi variables.
@ -502,8 +584,8 @@ class AILSimplifier(Analysis):
vvar, simvar = self._arg_vvars[func_arg_idx]
if vvar.varid == new_vvar.varid:
simvar_new = simvar.copy()
simvar_new._hash = None
simvar_new.size = new_vvar.size
simvar_new.clear_hash()
self._arg_vvars[func_arg_idx] = new_vvar, simvar_new
return narrowed
@ -669,6 +751,7 @@ class AILSimplifier(Analysis):
rd: SRDAModel,
addr_and_idx_to_block: dict[Address, Block],
effective_sizes: dict[int, int],
extractor_cache: dict[AILCodeLocation, EffectiveSizeExtractor] | None = None,
) -> ExprNarrowingInfo:
def_size = def_.size
# find its uses
@ -714,7 +797,7 @@ class AILSimplifier(Analysis):
if is_expr_used_as_reg_base_value(stmt, expr, rd):
continue
expr_size, use_type = self._extract_expression_effective_size(stmt, expr)
expr_size, use_type = self._extract_expression_effective_size(stmt, expr, loc, extractor_cache)
if expr_size is None:
if use_type == "insert-base":
# don't care
@ -820,25 +903,37 @@ class AILSimplifier(Analysis):
result.append((atom, loc, expr))
return result, phi_vars
def _extract_expression_effective_size(self, statement, expr) -> tuple[int | None, str | None]:
def _extract_expression_effective_size(
self,
statement,
expr,
loc: AILCodeLocation | None = None,
extractor_cache: dict[AILCodeLocation, EffectiveSizeExtractor] | None = None,
) -> tuple[int | None, str | None]:
"""
Determine the effective size of an expression when it's used.
"""
walker = EffectiveSizeExtractor(expr)
walker.walk_statement(statement)
if not isinstance(expr, VirtualVariable):
return None, None
effective_bit_ranges = set()
for expr_, (lo_bits, hi_bits) in walker.expr_to_effective_bits.items():
if expr.likes(expr_):
effective_bit_ranges.add((lo_bits, hi_bits))
walker = None
if extractor_cache is not None and loc is not None:
walker = extractor_cache.get(loc)
if walker is None:
walker = EffectiveSizeExtractor()
walker.walk_statement(statement)
if extractor_cache is not None and loc is not None:
extractor_cache[loc] = walker
if effective_bit_ranges:
highest_bit = max(hi_bits for _, hi_bits in effective_bit_ranges)
effective_bits_by_occurrence = walker.vvar_effective_bits.get(expr.varid)
if effective_bits_by_occurrence:
highest_bit = max(hi_bits for _, hi_bits in effective_bits_by_occurrence.values())
return highest_bit // self.project.arch.byte_width, "expr"
if walker.expr_used_as_call_arg_effective_bits is not None:
return walker.expr_used_as_call_arg_effective_bits[1] // self.project.arch.byte_width, "call-arg"
if walker.expr_used_as_insert_base:
call_arg_bits = walker.vvar_call_arg_effective_bits.get(expr.varid)
if call_arg_bits is not None:
return call_arg_bits[1] // self.project.arch.byte_width, "call-arg"
if expr.varid in walker.vvars_used_as_insert_base:
return None, "insert-base"
return None, None
@ -871,11 +966,9 @@ class AILSimplifier(Analysis):
self, replacements: dict[tuple[int, int | None], dict[AILCodeLocation, dict[Expression, Expression]]]
) -> bool:
blocks_by_addr_and_idx = {(node.addr, node.idx): node for node in self.func_graph.nodes()}
if self._stack_arg_offsets:
insn_addrs_using_stack_args = {ins_addr for ins_addr, _ in self._stack_arg_offsets}
else:
insn_addrs_using_stack_args = None
insn_addrs_using_stack_args = (
self._stackarg_offset_manager.get_stackarg_insaddrs() if self._stackarg_offset_manager is not None else None
)
replaced = False
for (block_addr, block_idx), reps in replacements.items():
@ -897,11 +990,12 @@ class AILSimplifier(Analysis):
}
reps = filtered_reps
r, new_block = BlockSimplifier._replace_and_build(
r, new_block = BlockSimplifier.replace_and_build(
block, reps, self._ail_manager, gp=self._gp, replace_loads=replace_loads
)
replaced |= r
self.blocks[block] = new_block
if r:
self.blocks[block] = new_block
if replaced:
# blocks have been rebuilt - expression propagation results are no longer reliable
@ -1026,7 +1120,7 @@ class AILSimplifier(Analysis):
continue
if use_loc not in replacements[key]:
replacements[key][use_loc] = {}
replacements[key][use_loc][expr] = Const(self._ail_manager.next_atom(), None, value, bits, **expr.tags)
replacements[key][use_loc][expr] = Const(self._ail_manager.next_atom(), value, bits, **expr.tags)
return self._replace_exprs_in_blocks(replacements) if replacements else False
@ -1063,6 +1157,9 @@ class AILSimplifier(Analysis):
# out-of-date
updated_locs: set[AILCodeLocation] = set()
# built on-demand
stack_defs_by_offset: dict[int, list[Definition[atoms.VirtualVariable, AILCodeLocation]]] | None = None
for _, atom in sorted_loc_and_atoms:
eqs = equivalences[atom]
filtered_eqs: list[tuple[Equivalence, VirtualVariable, bool]] = []
@ -1133,14 +1230,9 @@ class AILSimplifier(Analysis):
rd = self._compute_reaching_definitions()
the_def = None
if to_replace_is_def:
# find defs
defs: Container[Definition[atoms.VirtualVariable, AILCodeLocation]] = []
for def_ in rd.all_definitions:
if def_.atom.varid == to_replace.varid:
defs.append(def_)
if len(defs) != 1:
the_def = rd.get_definition_by_varid(to_replace.varid)
if the_def is None:
continue
the_def = defs[0]
else:
# find uses
defs = rd.get_uses_by_location(eq.codeloc)
@ -1166,38 +1258,42 @@ class AILSimplifier(Analysis):
# (a) the on-stack or in-register copy of it has never been modified in this function
# (b) the function argument register has never been updated.
# TODO: we may loosen requirement (b) once we have real register versioning in AIL.
defs = [def_ for def_ in rd.all_definitions if def_.codeloc == eq.codeloc]
defs = rd.get_defs_by_location(eq.codeloc)
all_uses_with_def = None
replace_with = None
remove_initial_assignment = None
def_eq_rel = DefEqRelation.DEF_IS_FUNCARG
if defs and len(defs) == 1:
arg_copy_def = defs[0]
if len(defs) == 1:
arg_copy_def = next(iter(defs))
if (isinstance(arg_copy_def.atom, atoms.VirtualVariable) and arg_copy_def.atom.was_stack) or (
isinstance(arg_copy_def.atom, atoms.VirtualVariable) and arg_copy_def.atom.was_reg
):
# found the copied definition (either a stack variable or a register variable)
# Make sure there is no other write to this stack location if the copy is a stack variable
if (
isinstance(arg_copy_def.atom, atoms.VirtualVariable)
and arg_copy_def.atom.was_stack
and any(
(def_ != arg_copy_def and def_.atom.stack_offset == arg_copy_def.atom.stack_offset)
for def_ in rd.all_definitions
if isinstance(def_.atom, atoms.VirtualVariable) and def_.atom.was_stack
)
):
continue
if isinstance(arg_copy_def.atom, atoms.VirtualVariable) and arg_copy_def.atom.was_stack:
if stack_defs_by_offset is None:
stack_defs_by_offset = defaultdict(list)
for def_ in rd.all_definitions:
if def_.atom.was_stack:
stack_defs_by_offset[def_.atom.stack_offset].append(def_)
if any(
def_ != arg_copy_def
for def_ in stack_defs_by_offset.get(arg_copy_def.atom.stack_offset, ())
):
continue
# Make sure the register is never updated across this function
if any(
(def_ != the_def and def_.atom == the_def.atom)
for def_ in rd.all_definitions
if isinstance(def_.atom, atoms.VirtualVariable)
and def_.atom.was_reg
and rd.get_vvar_uses(def_.atom)
# Make sure the register is never updated across this function. Only the definition of
# the_def's own vvar id can have an equal atom (atom equality is (varid, size)), and SSA gives
# that id exactly one definition, so this is a single lookup rather than a full scan.
other_def = rd.get_definition_by_varid(the_def.atom.varid)
if (
other_def is not None
and other_def != the_def
and other_def.atom == the_def.atom
and other_def.atom.was_reg
and rd.get_vvar_uses(other_def.atom)
):
continue
@ -1233,13 +1329,7 @@ class AILSimplifier(Analysis):
def_eq_rel = DefEqRelation.DEF_EQ_SAME_BLOCK
else:
# the definition is in the predecessor block of the eq
eq_block = next(
iter(
bb
for bb in self.func_graph
if bb.addr == eq.codeloc.block_addr and bb.idx == eq.codeloc.block_idx
)
)
eq_block = addr_and_idx_to_block[(eq.codeloc.block_addr, eq.codeloc.block_idx)]
eq_block_preds = set(self.func_graph.predecessors(eq_block))
if not any(
pred.addr == the_def.codeloc.block_addr and pred.idx == the_def.codeloc.block_idx
@ -1279,7 +1369,6 @@ class AILSimplifier(Analysis):
new_idx,
Const(
self._ail_manager.next_atom(),
None,
eq.atom0.addr,
self.project.arch.bits,
),
@ -1623,6 +1712,16 @@ class AILSimplifier(Analysis):
assert the_def.codeloc.block_addr is not None
assert the_def.codeloc.stmt_idx is not None
# Do not fold a call whose defining statement carries extra_defs (side-effect writes through pointer
# arguments, e.g. a call that fills a stack buffer). Folding moves the call to its single return-value
# use site, which would move the side-effect write as well and leave other uses of the written-through
# vvars reading an undefined value.
def_block = addr_and_idx_to_block.get((the_def.codeloc.block_addr, the_def.codeloc.block_idx))
if def_block is not None:
def_block = self.blocks.get(def_block, def_block)
if def_block.statements[the_def.codeloc.stmt_idx].tags.get("extra_defs"):
continue
all_uses = rd.get_vvar_uses_with_expr(the_def.atom)
if eq.is_weakassignment:
# eliminate the "use" at the weak assignment site
@ -1693,13 +1792,6 @@ class AILSimplifier(Analysis):
if isinstance(eq.atom0, VirtualVariable):
src = used_expr
dst: Expression = call.copy()
if isinstance(dst, SideEffectStatement):
dst_bits = dst.ret_expr.bits if dst.ret_expr is not None else dst.bits
# extract the Call expression from the SideEffectStatement
dst = dst.expr
dst.bits = dst_bits
if src.bits != dst.bits and not eq.is_weakassignment:
dst = Convert(
self._ail_manager.next_atom(),
@ -1804,16 +1896,69 @@ class AILSimplifier(Analysis):
@timethis
def _iteratively_remove_dead_assignments(self) -> bool:
if (
not self._should_eliminate_dead_assignments
and not self.blocks
and not self._calls_to_remove
and not self._assignments_to_remove
):
# nothing that _remove_dead_assignments() reads has changed since it last reported nothing to remove
return False
anything_removed = False
while True:
r = self._remove_dead_assignments()
r, changed_block_keys = self._remove_dead_assignments()
if not r:
return anything_removed
break
anything_removed = True
self._rebuild_func_graph()
self._clear_cache()
# Instead of discarding the reaching-definitions cache and recomputing it from scratch on every iteration,
# incrementally update it: removed statements were replaced in place by NoOp placeholders, so statement
# indices are stable and we only need to drop the removed vvar definitions and their now-eliminated uses.
if self._reaching_definitions is not None and changed_block_keys:
edited_blocks = [
block for block in self.func_graph.nodes() if (block.addr, block.idx) in changed_block_keys
]
self._reaching_definitions.update_after_block_edits(edited_blocks)
if _VERIFY_INCREMENTAL_RD:
self._verify_incremental_reaching_definitions()
# propagation results are no longer reliable after removing statements
self._propagator = None
@timethis
def _remove_dead_assignments(self) -> bool:
self._should_eliminate_dead_assignments = False
# NoOp placeholders are left in the graph and the reaching-definitions cache is kept valid: subsequent
# simplification steps reuse it instead of rebuilding from scratch. The placeholders are compacted away once,
# at the end of _simplify().
return anything_removed
def _compact_noop_statements(self) -> bool:
found = False
for block in list(self.func_graph.nodes()):
if any(isinstance(stmt, NoOp) for stmt in block.statements):
new_block = block.copy()
new_block.statements = [stmt for stmt in block.statements if not isinstance(stmt, NoOp)]
self.blocks[block] = new_block
found = True
if found:
self._rebuild_func_graph()
return found
def _verify_incremental_reaching_definitions(self) -> None:
# Debug-only (env var VERIFY_INCREMENTAL_RD): assert the incrementally-updated model is identical to a full
# rebuild on the current (NoOp-containing) graph.
assert self._reaching_definitions is not None
func_args = {vvar for vvar, _ in self._arg_vvars.values()} if self._arg_vvars else set()
reference = SReachingDefinitions(
self.project,
subject=self.func,
func_graph=self.func_graph,
func_args=func_args,
use_callee_saved_regs_at_return=self._use_callee_saved_regs_at_return,
).model
if self._reaching_definitions.canonical_form() != reference.canonical_form():
raise AssertionError("Incremental SRDA update diverged from a full rebuild")
def _remove_dead_assignments(self) -> tuple[bool, set[tuple[int, int | None]]]:
# keeping tracking of statements to remove and statements (as well as dead vvars) to keep allows us to handle
# cases where a statement defines more than one atom, e.g., a call statement that defines both the return
# value and the floating-point return value.
@ -1826,12 +1971,7 @@ class AILSimplifier(Analysis):
}
# Find all statements that should be removed
mask = (1 << self.project.arch.bits) - 1
rd = self._compute_reaching_definitions()
stackarg_offsets = (
{(tpl[1] & mask) for tpl in self._stack_arg_offsets} if self._stack_arg_offsets is not None else None
)
retpoints: set[Address] = {
(node.addr, node.idx)
for node in self.func_graph
@ -1852,7 +1992,7 @@ class AILSimplifier(Analysis):
if uses is None:
vvar = rd.varid_to_vvar[vvar_id]
def_codeloc = rd.all_vvar_definitions[vvar_id]
def_codeloc = codeloc
if def_codeloc.is_extern:
def_stmt = None
else:
@ -1872,10 +2012,10 @@ class AILSimplifier(Analysis):
# note that this is a hack! we should rely on more reliable stack variable
# eliminatability detection.
pass
elif stackarg_offsets is not None:
# we always remove definitions for stack arguments
assert vvar.stack_offset is not None
if (vvar.stack_offset & mask) not in stackarg_offsets:
elif self._stackarg_offset_manager is not None:
if not self._stackarg_offset_manager.is_stackarg_vvar(vvar.varid):
# this stack variable is not a stack argument for any of the call sites that consume
# this offset. it is not eliminatable.
continue
else:
continue
@ -1941,6 +2081,7 @@ class AILSimplifier(Analysis):
stmts_to_remove_per_block[codeloc.block_addr, codeloc.block_idx].add(codeloc.stmt_idx)
simplified = False
changed_block_keys: set[tuple[int, int | None]] = set()
# Remove the statements
for old_block in self.func_graph.nodes():
@ -1953,7 +2094,7 @@ class AILSimplifier(Analysis):
if (block.addr, block.idx) not in stmts_to_remove_per_block:
continue
new_statements = []
new_statements: list[Statement] = []
stmts_to_remove = stmts_to_remove_per_block[(block.addr, block.idx)]
stmts_to_keep = stmts_to_keep_per_block[(block.addr, block.idx)]
@ -1965,12 +2106,22 @@ class AILSimplifier(Analysis):
# this statement declares more than one variable. we should handle it surgically
# case 1: stmt.ret_expr and stmt.fp_ret_expr are both set, but one of them is not used
if isinstance(stmt.ret_expr, VirtualVariable) and stmt.ret_expr.varid in dead_vvar_ids:
stmt = stmt.copy()
stmt.ret_expr = None
stmt = SideEffectStatement(
self._ail_manager.next_atom(),
stmt.expr,
ret_expr=None,
fp_ret_expr=stmt.fp_ret_expr,
**stmt.tags,
)
simplified = True
if isinstance(stmt.fp_ret_expr, VirtualVariable) and stmt.fp_ret_expr.varid in dead_vvar_ids:
stmt = stmt.copy()
stmt.fp_ret_expr = None
stmt = SideEffectStatement(
self._ail_manager.next_atom(),
stmt.expr,
ret_expr=stmt.ret_expr,
fp_ret_expr=None,
**stmt.tags,
)
simplified = True
if idx in stmts_to_remove and idx not in stmts_to_keep and not isinstance(stmt, DirtyStatement):
@ -1990,12 +2141,14 @@ class AILSimplifier(Analysis):
codeloc = AILCodeLocation(block.addr, block.idx, idx, stmt.tags.get("ins_addr"))
if codeloc in self._assignments_to_remove:
# it should be removed
new_statements.append(NoOp(stmt.idx, ins_addr=stmt.tags.get("ins_addr", -1)))
simplified = True
continue
if self._statement_has_call_exprs(stmt):
if codeloc in self._calls_to_remove:
# it has a call and must be removed
new_statements.append(NoOp(stmt.idx, ins_addr=stmt.tags.get("ins_addr", -1)))
simplified = True
continue
if isinstance(stmt, Assignment) and isinstance(stmt.dst, VirtualVariable):
@ -2014,12 +2167,14 @@ class AILSimplifier(Analysis):
pass
else:
# no calls. remove it
new_statements.append(NoOp(stmt.idx, ins_addr=stmt.tags.get("ins_addr", -1)))
simplified = True
continue
elif isinstance(stmt, SideEffectStatement):
codeloc = AILCodeLocation(block.addr, block.idx, idx, stmt.tags.get("ins_addr"))
if codeloc in self._calls_to_remove:
# this call can be removed
new_statements.append(NoOp(stmt.idx, ins_addr=stmt.tags.get("ins_addr", -1)))
simplified = True
continue
@ -2027,9 +2182,9 @@ class AILSimplifier(Analysis):
isinstance(stmt.ret_expr, VirtualVariable) and stmt.ret_expr.was_combo_reg
):
# both the return expr and the fp_ret_expr are not used
stmt = stmt.copy()
stmt.ret_expr = None
stmt.fp_ret_expr = None
stmt = SideEffectStatement(
self._ail_manager.next_atom(), stmt.expr, ret_expr=None, fp_ret_expr=None, **stmt.tags
)
simplified = True
else:
# Should not happen!
@ -2040,13 +2195,14 @@ class AILSimplifier(Analysis):
new_block = block.copy()
new_block.statements = new_statements
self.blocks[old_block] = new_block
changed_block_keys.add((new_block.addr, new_block.idx))
# we can only use calls_to_remove and assignments_to_remove once; if any statements in blocks are removed, then
# the statement IDs in calls_to_remove and assignments_to_remove no longer match!
self._calls_to_remove.clear()
self._assignments_to_remove.clear()
return simplified
return simplified, changed_block_keys
@staticmethod
def _get_vvar_used_by(
@ -2078,17 +2234,32 @@ class AILSimplifier(Analysis):
def _find_cyclic_dependent_phis_and_dirty_vvars(self, rd: SRDAModel, dead_vvar_ids: set[int]) -> set[int]:
blocks_dict: dict[tuple[int, int | None], Block] = {(bb.addr, bb.idx): bb for bb in self.func_graph}
# find dirty vvars and vexccall vvars
dirty_vvar_ids = set()
# cache dirty or ccall vvar IDs per block to avoid re-scanning
# TODO: Move this cache to ailment.Block once per-block defs/uses cache lands on master.
cache = self._dirty_vvar_scan_cache
for bb in self.func_graph:
for stmt in bb.statements:
if (
isinstance(stmt, Assignment)
and isinstance(stmt.dst, VirtualVariable)
and stmt.dst.was_reg
and isinstance(stmt.src, (DirtyExpression, VEXCCallExpression))
):
dirty_vvar_ids.add(stmt.dst.varid)
key = bb.addr, bb.idx
entry = cache.get(key)
if entry is not None and entry[0] is bb:
block_dirty_ids = entry[1]
else:
block_dirty_ids = set()
for stmt in bb.statements:
# reg/tmp = ccall(...)
# we see tmps when it's used in a cycle;
# see binary ddc2b4cbf6ac841524375cdf82b93b9948f8ea09bbf6e8bf3410e6bc410a9d95 function 0x18001722c
# block 0x18001724c
if (
isinstance(stmt, Assignment)
and isinstance(stmt.dst, VirtualVariable)
and (stmt.dst.was_reg or stmt.dst.was_tmp)
and isinstance(stmt.src, (DirtyExpression, VEXCCallExpression))
):
block_dirty_ids.add(stmt.dst.varid)
cache[key] = bb, block_dirty_ids
if block_dirty_ids:
dirty_vvar_ids |= block_dirty_ids
phi_and_dirty_vvar_ids = (rd.phi_vvar_ids | dirty_vvar_ids).difference(dead_vvar_ids)
@ -2106,18 +2277,25 @@ class AILSimplifier(Analysis):
vvar_used_by[used_by_varid].add(var_id) # probably unnecessary
vvar_used_by[var_id] |= self._get_vvar_used_by(var_id, rd, blocks_dict).difference(dead_vvar_ids)
g = networkx.DiGraph()
# build a plain adjacency map instead of a throwaway networkx DiGraph for better performance. the performance
# improvement is observable on notepad.exe:NPInit
# TODO: Investigate if switching to rustworkx eliminates the need for this optimization.
dummy_vvar_id = -1
succs_map: dict[int, set[int]] = {}
for var_id, used_by_initial in vvar_used_by.items():
for u in used_by_initial:
if u is None:
# we can't have None in networkx.DiGraph
g.add_edge(var_id, dummy_vvar_id)
else:
g.add_edge(var_id, u)
if not used_by_initial:
continue
targets = {dummy_vvar_id if u is None else u for u in used_by_initial}
if var_id in succs_map:
succs_map[var_id] |= targets
else:
succs_map[var_id] = set(targets)
for target in targets:
if target not in succs_map:
succs_map[target] = set()
cyclic_dependent_phi_varids = set()
for scc in networkx.strongly_connected_components(g):
for scc in _strongly_connected_components(succs_map):
if len(scc) == 1:
continue
@ -2128,11 +2306,9 @@ class AILSimplifier(Analysis):
if varid in vvar_used_by and None in vvar_used_by[varid]:
bail = True
break
if bail is False:
succs = list(g.successors(varid))
if any(succ_varid not in scc for succ_varid in succs):
bail = True
break
if any(succ_varid not in scc for succ_varid in succs_map[varid]):
bail = True
break
if bail:
continue
@ -2162,7 +2338,14 @@ class AILSimplifier(Analysis):
def _handle_VEXCCallExpression(
expr_idx: int, expr: VEXCCallExpression, stmt_idx: int, stmt: Statement | None, block: Block | None
) -> Expression:
r_expr = AILBlockRewriter._handle_VEXCCallExpression(walker, expr_idx, expr, stmt_idx, stmt, block)
r_expr = AILBlockRewriter._handle_VEXCCallExpression( # pylint:disable=protected-access
walker,
expr_idx,
expr,
stmt_idx,
stmt,
block,
)
rewriter = rewriter_cls(r_expr, self.project, self._ail_manager, rename_ccalls=self._should_rename_ccalls)
if rewriter.result is not None:
_any_update.v = True
@ -2205,10 +2388,10 @@ class AILSimplifier(Analysis):
stmt_idx: int, stmt: DirtyStatement, block: Block | None
) -> Statement:
# we do not want to trigger _handle_DirtyExpression, which is why we do not call the superclass method
rewriter = rewriter_cls(stmt, self.project.arch)
rewriter = rewriter_cls(stmt, self.project.arch, self._ail_manager)
if rewriter.result is not None:
_any_update.v = True
if walker._update_block and block is not None:
if walker._update_block and block is not None: # pylint:disable=protected-access
block.statements[stmt_idx] = rewriter.result # type: ignore
assert isinstance(rewriter.result, Statement)
return rewriter.result
@ -2217,9 +2400,16 @@ class AILSimplifier(Analysis):
def _handle_DirtyExpression(
expr_idx: int, expr: DirtyExpression, stmt_idx: int, stmt: Statement | None, block: Block | None
):
r_expr = AILBlockRewriter._handle_DirtyExpression(walker, expr_idx, expr, stmt_idx, stmt, block)
r_expr = AILBlockRewriter._handle_DirtyExpression( # pylint:disable=protected-access
walker,
expr_idx,
expr,
stmt_idx,
stmt,
block,
)
assert isinstance(r_expr, DirtyExpression)
rewriter = rewriter_cls(r_expr, self.project.arch)
rewriter = rewriter_cls(r_expr, self.project.arch, self._ail_manager)
if rewriter.result is not None:
_any_update.v = True
assert isinstance(rewriter.result, Expression)
@ -2247,34 +2437,18 @@ class AILSimplifier(Analysis):
@staticmethod
def _statement_has_call_exprs(stmt: Statement) -> bool:
def _handle_callexpr(expr_idx, expr, stmt_idx, stmt, block): # pylint:disable=unused-argument
raise HasCallNotification
def _handle_macroexpr(expr_idx, expr, stmt_idx, stmt, block):
raise HasCallNotification
walker = AILBlockViewer()
walker.expr_handlers[Call] = _handle_callexpr
walker.expr_handlers[FunctionLikeMacro] = _handle_macroexpr
try:
walker.walk_statement(stmt)
_HAS_CALL_EXPRS_WALKER.walk_statement(stmt)
except HasCallNotification:
return True
return False
@staticmethod
def _expression_has_call_exprs(expr: Expression) -> bool:
def _handle_callexpr(expr_idx, expr, stmt_idx, stmt, block): # pylint:disable=unused-argument
raise HasCallNotification
walker = AILBlockViewer()
walker.expr_handlers[Call] = _handle_callexpr
try:
walker.walk_expression(expr)
_HAS_CALL_EXPRS_WALKER.walk_expression(expr)
except HasCallNotification:
return True
return False
@staticmethod

View file

@ -1,11 +1,14 @@
from __future__ import annotations
from typing import TYPE_CHECKING
from collections.abc import Sequence
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from typing import Any
from collections.abc import Callable
from typing import Any
import networkx
from angr import ailment

View file

@ -1,34 +1,33 @@
from __future__ import annotations
from collections import defaultdict
from typing import Any
from angr.ailment import Block
from angr.ailment.block_walker import AILBlockViewer, _dispatch_key
from angr.ailment.expression import (
ITE,
BinaryOp,
Call,
Const,
Convert,
Expression,
Load,
StackBaseOffset,
Tmp,
UnaryOp,
VirtualVariable,
)
from angr.ailment.statement import (
Assignment,
ConditionalJump,
Jump,
Return,
SideEffectStatement,
Statement,
ConditionalJump,
Assignment,
Store,
Return,
Jump,
)
from angr.ailment.expression import (
Call,
Load,
VirtualVariable,
Expression,
BinaryOp,
UnaryOp,
Convert,
ITE,
Tmp,
Const,
StackBaseOffset,
)
from angr.ailment.block_walker import AILBlockViewer
from angr.knowledge_plugins.key_definitions.atoms import MemoryLocation, Register, SpOffset, ConstantSrc
from angr.knowledge_plugins.key_definitions.atoms import ConstantSrc, MemoryLocation, Register, SpOffset
class BlockIOFinder(AILBlockViewer):
@ -185,11 +184,7 @@ class BlockIOFinder(AILBlockViewer):
block: Block | None,
is_memory=False,
) -> Any:
try:
handler = self.expr_handlers[type(expr)]
except KeyError:
handler = None
handler = self.expr_handlers.get(_dispatch_key(expr))
if handler:
return handler(expr_idx, expr, stmt_idx, stmt, block, is_memory=is_memory)
return None
@ -211,7 +206,7 @@ class BlockIOFinder(AILBlockViewer):
return load_loc
return None
def _handle_CallExpr(
def _handle_Call(
self, expr_idx: int, expr: Call, stmt_idx: int, stmt: Statement, block: Block | None, is_memory=False
):
args = set()

View file

@ -1,8 +1,11 @@
from __future__ import annotations
import logging
import networkx as nx
from angr.ailment.block import Block
from angr.ailment.statement import Statement, ConditionalJump
from angr.ailment.statement import ConditionalJump, Statement
from .utils import find_block_by_addr_and_idx
@ -37,6 +40,10 @@ def is_similar(
if type(ail_obj1) is not type(ail_obj2):
return False
k1 = getattr(ail_obj1, "kind", None)
k2 = getattr(ail_obj2, "kind", None)
if k1 is not None and k2 is not None and k1 != k2:
return False
if ail_obj1 is ail_obj2:
return True

View file

@ -1,83 +1,159 @@
# pylint:disable=too-many-boolean-expressions
from __future__ import annotations
import logging
from typing import TYPE_CHECKING
from collections.abc import Iterable, Mapping
import logging
from collections.abc import Iterable, Mapping
from typing import TYPE_CHECKING
from angr.ailment.expression import Call, Const, Convert, Expression, Load, Register, Tmp, VirtualVariable
from angr.ailment.manager import Manager
from angr.ailment.statement import Statement, Assignment, SideEffectStatement, Store, Jump
from angr.ailment.expression import Call, Tmp, Load, Const, Register, Convert, Expression, VirtualVariable
from angr.ailment import AILBlockViewer
from angr.ailment.statement import Assignment, Jump, SideEffectStatement, Statement, Store
from angr.analyses.s_propagator import SPropagator
from angr.analyses.s_reaching_definitions import SRDAModel, SReachingDefinitions
from angr.code_location import AILCodeLocation
from angr.knowledge_plugins.key_definitions import atoms
from angr.analyses.s_propagator import SPropagatorAnalysis
from angr.analyses.s_reaching_definitions import SReachingDefinitionsAnalysis, SRDAModel
from angr.analyses import Analysis, register_analysis
from angr.utils.ssa import has_reference_to_vvar
from .block_walkers import HasCallExprWalker, HasCallNotification
from .peephole_optimizations import (
EXPR_OPTS,
MULTI_STMT_OPTS,
STMT_OPTS,
EXPR_OPTS,
PeepholeOptimizationStmtBase,
PeepholeOptimizationExprBase,
PeepholeOptimizationMultiStmtBase,
PeepholeOptimizationStmtBase,
)
from .utils import (
_PeepholeExprsWalker,
build_stmt_opts_by_kind,
peephole_optimize_exprs,
peephole_optimize_multistmts,
peephole_optimize_stmts,
)
from .utils import peephole_optimize_exprs, peephole_optimize_stmts, peephole_optimize_multistmts
if TYPE_CHECKING:
from angr.ailment.block import Block
from angr.project import Project
_l = logging.getLogger(name=__name__)
class HasCallExprWalker(AILBlockViewer):
_HAS_CALL_EXPR_WALKER = HasCallExprWalker()
class PeepholeOptimizationBundle:
"""
Test if an expression contains a call expression inside.
PeepholeOptimizationBundle describes a set of initialized peephole optimizer instances and the dispatch structures
derived from them. This bundle of peephole optimizations is reusable across `BlockSimplifier` invocations (so we
avoid rebuilding the same optimizer instances).
"""
def __init__(self):
super().__init__()
self.has_call_expr = False
__slots__ = (
"_params",
"expr_opts",
"expr_walker",
"multistmt_opts",
"stmt_opts",
"stmt_opts_by_kind",
)
def _handle_SideEffectStatement(self, stmt_idx: int, stmt: SideEffectStatement, block: Block | None): # pylint:disable=unused-argument
self.has_call_expr = True
def _handle_CallExpr( # pylint:disable=unused-argument
self, expr_idx: int, expr: Call, stmt_idx: int, stmt: Statement | None, block: Block | None
def __init__(
self,
project,
kb,
ail_manager: Manager,
func_addr: int | None = None,
preserve_vvar_ids: set[int] | None = None,
type_hints: list[tuple[atoms.VirtualVariable | atoms.MemoryLocation, str]] | None = None,
peephole_optimizations: Iterable[
type[PeepholeOptimizationStmtBase | PeepholeOptimizationExprBase | PeepholeOptimizationMultiStmtBase]
]
| None = None,
):
self.has_call_expr = True
if peephole_optimizations is None:
expr_classes: Iterable = EXPR_OPTS
stmt_classes: Iterable = STMT_OPTS
multistmt_classes: Iterable = MULTI_STMT_OPTS
else:
peephole_optimizations = tuple(peephole_optimizations)
expr_classes = [cls for cls in peephole_optimizations if issubclass(cls, PeepholeOptimizationExprBase)]
stmt_classes = [cls for cls in peephole_optimizations if issubclass(cls, PeepholeOptimizationStmtBase)]
multistmt_classes = [
cls for cls in peephole_optimizations if issubclass(cls, PeepholeOptimizationMultiStmtBase)
]
args = (project, kb, ail_manager, func_addr, preserve_vvar_ids, type_hints)
self.expr_opts = [cls(*args) for cls in expr_classes]
self.stmt_opts = [cls(*args) for cls in stmt_classes]
self.multistmt_opts = [cls(*args) for cls in multistmt_classes]
self.stmt_opts_by_kind = build_stmt_opts_by_kind(self.stmt_opts)
self.expr_walker = _PeepholeExprsWalker(expr_opts=self.expr_opts)
self._params = (project, ail_manager, func_addr, preserve_vvar_ids, type_hints, peephole_optimizations)
def matches(
self,
project,
ail_manager: Manager,
func_addr: int | None,
preserve_vvar_ids: set[int] | None,
type_hints: list | None,
peephole_optimizations,
) -> bool:
p_project, p_manager, p_func_addr, p_preserve, p_hints, p_opts = self._params
return (
p_project is project
and p_manager is ail_manager
and p_func_addr == func_addr
and p_preserve is preserve_vvar_ids
and p_hints is type_hints
and (
p_opts is peephole_optimizations
or (
p_opts is not None
and peephole_optimizations is not None
and p_opts == tuple(peephole_optimizations)
)
)
)
class BlockSimplifier(Analysis):
class BlockSimplifier:
"""
Simplify an AIL block.
Deliberately not an :class:`Analysis`: it is instantiated once per block, hundreds of times per decompilation,
so it skips the analysis-factory ceremony. Instantiate it directly with the project as the first argument;
exceptions always propagate.
"""
def __init__(
self,
project: Project,
block: Block | None,
ail_manager: Manager,
func_addr: int | None = None,
stack_pointer_tracker=None,
peephole_optimizations: None
| (
Iterable[
type[PeepholeOptimizationStmtBase]
| type[PeepholeOptimizationExprBase]
| type[PeepholeOptimizationMultiStmtBase]
]
) = None,
peephole_optimizations: Iterable[
type[PeepholeOptimizationStmtBase | PeepholeOptimizationExprBase | PeepholeOptimizationMultiStmtBase]
]
| None = None,
preserve_vvar_ids: set[int] | None = None,
type_hints: list[tuple[atoms.VirtualVariable | atoms.MemoryLocation, str]] | None = None,
cached_reaching_definitions=None,
cached_propagator=None,
peephole_bundle: PeepholeOptimizationBundle | None = None,
):
"""
:param block: The AIL block to simplify. Setting it to None to skip calling self._analyze(), which is useful
in test cases.
:param peephole_bundle: A pre-built PeepholeOptimizationBundle to reuse. Its construction parameters must
match this BlockSimplifier's; callers that simplify many blocks should build one bundle and
pass it to every BlockSimplifier they create.
"""
self.project = project
self.kb = project.kb
self.block = block
self.func_addr = func_addr
@ -86,38 +162,26 @@ class BlockSimplifier(Analysis):
self._type_hints = type_hints
self._ail_manager = ail_manager
if peephole_optimizations is None:
self._expr_peephole_opts = [
cls(self.project, self.kb, ail_manager, self.func_addr, self._preserve_vvar_ids, self._type_hints)
for cls in EXPR_OPTS
]
self._stmt_peephole_opts = [
cls(self.project, self.kb, ail_manager, self.func_addr, self._preserve_vvar_ids, self._type_hints)
for cls in STMT_OPTS
]
self._multistmt_peephole_opts = [
cls(self.project, self.kb, ail_manager, self.func_addr, self._preserve_vvar_ids, self._type_hints)
for cls in MULTI_STMT_OPTS
]
else:
self._expr_peephole_opts = [
cls(self.project, self.kb, ail_manager, self.func_addr, self._preserve_vvar_ids, self._type_hints)
for cls in peephole_optimizations
if issubclass(cls, PeepholeOptimizationExprBase)
]
self._stmt_peephole_opts = [
cls(self.project, self.kb, ail_manager, self.func_addr, self._preserve_vvar_ids, self._type_hints)
for cls in peephole_optimizations
if issubclass(cls, PeepholeOptimizationStmtBase)
]
self._multistmt_peephole_opts = [
cls(self.project, self.kb, ail_manager, self.func_addr, self._preserve_vvar_ids, self._type_hints)
for cls in peephole_optimizations
if issubclass(cls, PeepholeOptimizationMultiStmtBase)
]
if peephole_bundle is None:
peephole_bundle = PeepholeOptimizationBundle(
self.project,
self.kb,
ail_manager,
func_addr=self.func_addr,
preserve_vvar_ids=self._preserve_vvar_ids,
type_hints=self._type_hints,
peephole_optimizations=peephole_optimizations,
)
self._expr_peephole_opts = peephole_bundle.expr_opts
self._stmt_peephole_opts = peephole_bundle.stmt_opts
self._multistmt_peephole_opts = peephole_bundle.multistmt_opts
self._stmt_peephole_opts_by_kind = peephole_bundle.stmt_opts_by_kind
self.result_block = None
# cached peephole expression walker
self._expr_peephole_walker = peephole_bundle.expr_walker
# cached Propagator and ReachingDefinitions results. Clear them if the block is updated
self._propagator = cached_propagator
self._reaching_definitions = cached_reaching_definitions
@ -127,24 +191,33 @@ class BlockSimplifier(Analysis):
def _analyze(self):
block = self.block
assert block is not None
ctr = 0
max_ctr = 30
new_block = self._eliminate_self_assignments(block)
new_block, changed = self._eliminate_self_assignments(block)
# True once dead-assignment elimination is known to have nothing to do on the block the loop below starts
# from -- either because it just ran over it without a change, or because its gate is off for that block.
dead_assignments_clean = True
if self._count_nonconstant_statements(new_block) >= 2 and self._has_propagatable_assignments(new_block):
new_block = self._eliminate_dead_assignments(new_block)
if new_block != block:
new_block, dead_changed = self._eliminate_dead_assignments(new_block)
changed |= dead_changed
dead_assignments_clean = not dead_changed
if changed:
self._clear_cache()
block = new_block
while True:
ctr += 1
# block.pp()
new_block = self._simplify_block_once(block)
# print()
# new_block.pp()
if new_block == block:
# the entry peephole pass is only useful on the first iteration: every later iteration receives the
# output of the previous iteration's exit peephole pass, so running peephole again on entry is redundant.
new_block, changed = self._simplify_block_once(
block, entry_peephole=ctr == 1, dead_assignments_clean=dead_assignments_clean
)
if not changed:
break
assert new_block is not None
self._clear_cache()
block = new_block
if ctr >= max_ctr:
@ -157,9 +230,10 @@ class BlockSimplifier(Analysis):
self.result_block = block
def _compute_propagation(self, block) -> SPropagatorAnalysis:
def _compute_propagation(self, block) -> SPropagator:
if self._propagator is None:
self._propagator = self.project.analyses[SPropagatorAnalysis].prep(fail_fast=self._fail_fast)(
self._propagator = SPropagator(
self.project,
subject=block,
func_addr=self.func_addr,
stack_pointer_tracker=self._stack_pointer_tracker,
@ -169,15 +243,12 @@ class BlockSimplifier(Analysis):
def _compute_reaching_definitions(self, block) -> SRDAModel:
if self._reaching_definitions is None:
self._reaching_definitions = (
self.project.analyses[SReachingDefinitionsAnalysis]
.prep(fail_fast=self._fail_fast)(
subject=block,
track_tmps=True,
func_addr=self.func_addr,
)
.model
)
self._reaching_definitions = SReachingDefinitions(
self.project,
subject=block,
track_tmps=True,
func_addr=self.func_addr,
).model
return self._reaching_definitions
def _clear_cache(self):
@ -192,35 +263,56 @@ class BlockSimplifier(Analysis):
def _count_nonconstant_statements(block) -> int:
return sum(1 for stmt in block.statements if not (isinstance(stmt, Jump) and isinstance(stmt.target, Const)))
def _simplify_block_once(self, block):
block = self._peephole_optimize(block)
def _simplify_block_once(
self, block, entry_peephole: bool = True, dead_assignments_clean: bool = False
) -> tuple[Block, bool]:
"""
Run one round of simplification. Returns the new block and if any step reported a change.
:param dead_assignments_clean: True if dead-assignment elimination is known to have nothing to do on
``block`` as passed in. Only meaningful together with ``entry_peephole``.
"""
changed = False
# True once we know ``block`` is untouched and already at the fixpoint of every pass that has run over it:
# re-running those passes on it cannot report a change.
clean = False
if entry_peephole:
block, peephole_changed, exprs_updated = self._peephole_optimize(block)
changed |= peephole_changed
clean = dead_assignments_clean and not peephole_changed and not exprs_updated
nonconstant_stmts = self._count_nonconstant_statements(block)
has_propagatable_assignments = self._has_propagatable_assignments(block)
# propagator
# only call propagation if something is potentially propagatable
if nonconstant_stmts >= 2 and has_propagatable_assignments:
propagator = self._compute_propagation(block)
new_block = block
if propagator.model is not None:
replacements = propagator.model.replacements
if replacements:
_, new_block = self._replace_and_build(
replaced, new_block = self.replace_and_build(
block, replacements, self._ail_manager, replace_registers=True
)
new_block = self._eliminate_self_assignments(new_block)
changed |= replaced
new_block, self_assign_changed = self._eliminate_self_assignments(new_block)
changed |= self_assign_changed
self._clear_cache()
else:
# Skipped calling Propagator
new_block = block
if nonconstant_stmts >= 2 and has_propagatable_assignments:
new_block = self._eliminate_dead_assignments(new_block)
if clean and new_block is block:
return block, False
return self._peephole_optimize(new_block)
if nonconstant_stmts >= 2 and has_propagatable_assignments:
new_block, dead_changed = self._eliminate_dead_assignments(new_block)
changed |= dead_changed
new_block, peephole_changed, _ = self._peephole_optimize(new_block)
return new_block, changed | peephole_changed
@staticmethod
def _replace_and_build(
def replace_and_build(
block: Block,
replacements: Mapping[AILCodeLocation, Mapping[Expression, Expression]],
ail_manager: Manager,
@ -310,11 +402,11 @@ class BlockSimplifier(Analysis):
return True, new_block
@staticmethod
def _eliminate_self_assignments(block):
def _eliminate_self_assignments(block) -> tuple[Block, bool]:
new_statements = []
for stmt in block.statements:
if type(stmt) is Assignment:
if isinstance(stmt, Assignment):
if stmt.dst.likes(stmt.src):
continue
if (
@ -330,28 +422,35 @@ class BlockSimplifier(Analysis):
continue
new_statements.append(stmt)
return block.copy(statements=new_statements)
if len(new_statements) == len(block.statements):
# nothing was eliminated; keep the original block
return block, False
return block.copy(statements=new_statements), True
def _eliminate_dead_assignments(self, block):
def _eliminate_dead_assignments(self, block) -> tuple[Block, bool]:
def _statement_has_calls(stmt: Statement) -> bool:
"""
Check if a statement has any Call expressions.
"""
walker = HasCallExprWalker()
walker.walk_statement(stmt)
return walker.has_call_expr
try:
_HAS_CALL_EXPR_WALKER.walk_statement(stmt)
except HasCallNotification:
return True
return False
def _expression_has_calls(expr: Expression) -> bool:
"""
Check if an expression has any Call expressions.
"""
walker = HasCallExprWalker()
walker.walk_expression(expr)
return walker.has_call_expr
try:
_HAS_CALL_EXPR_WALKER.walk_expression(expr)
except HasCallNotification:
return True
return False
new_statements = []
if not block.statements:
return block
return block, False
rd = self._compute_reaching_definitions(block)
block_loc = (block.addr, block.idx)
@ -379,45 +478,59 @@ class BlockSimplifier(Analysis):
used_tmps.add(tmp.tmp_idx)
# Remove dead assignments
changed = False
for idx, stmt in enumerate(block.statements):
if type(stmt) is Assignment:
if isinstance(stmt, Assignment):
# tmps can't execute new code
if (type(stmt.dst) is Tmp and stmt.dst.tmp_idx not in used_tmps) or idx in dead_defs_stmt_idx:
if (isinstance(stmt.dst, Tmp) and stmt.dst.tmp_idx not in used_tmps) or idx in dead_defs_stmt_idx:
# is it assigning to an unused tmp or a dead virgin?
# does .src involve any Call expressions? if so, we cannot remove it
if not _expression_has_calls(stmt.src):
changed = True
continue
if type(stmt.dst) is Tmp and isinstance(stmt.src, Call):
if isinstance(stmt.dst, Tmp) and isinstance(stmt.src, Call):
# eliminate the assignment and replace it with the call
stmt = SideEffectStatement(stmt.idx, stmt.src, **stmt.tags)
stmt = SideEffectStatement(self._ail_manager.next_atom(), stmt.src, **stmt.tags)
changed = True
if isinstance(stmt, Assignment) and stmt.src == stmt.dst:
changed = True
continue
new_statements.append(stmt)
return block.copy(statements=new_statements)
if not changed:
# nothing was eliminated; keep the original block
return block, False
return block.copy(statements=new_statements), True
#
# Peephole optimization
#
def _peephole_optimize(self, block):
# expressions are updated in place
peephole_optimize_exprs(block, self._expr_peephole_opts)
def _peephole_optimize(self, block) -> tuple[Block, bool, bool]:
"""
Run all three peephole optimization levels on the block.
:return: (block, changed, exprs_updated), where ``changed`` is True if any optimization applied and
``exprs_updated`` is True if the expression walker rewrote any expression.
"""
exprs_updated = peephole_optimize_exprs(block, self._expr_peephole_opts, walker=self._expr_peephole_walker)
# run statement-level optimizations
statements, stmts_updated = peephole_optimize_stmts(block, self._stmt_peephole_opts)
statements, stmts_updated = peephole_optimize_stmts(
block,
self._stmt_peephole_opts,
stmt_opts_by_kind=self._stmt_peephole_opts_by_kind,
fixpoint_exprs=self._expr_peephole_walker.fixpoint_stmts,
)
new_block = block.copy(statements=statements) if stmts_updated else block
statements, multi_stmts_updated = peephole_optimize_multistmts(new_block, self._multistmt_peephole_opts)
if not multi_stmts_updated:
return new_block
return new_block.copy(statements=statements)
register_analysis(BlockSimplifier, "AILBlockSimplifier")
if multi_stmts_updated:
new_block = new_block.copy(statements=statements)
return new_block, stmts_updated or multi_stmts_updated, exprs_updated

View file

@ -0,0 +1,24 @@
from __future__ import annotations
from angr.ailment import AILBlockViewer
class HasCallNotification(Exception):
"""
Abort the walk on the first Call / SideEffectStatement encountered.
"""
class HasCallExprWalker(AILBlockViewer):
"""
Singleton walker that raises ``HasCallNotification`` on the first Call / SideEffectStatement it visits.
"""
def _handle_SideEffectStatement(self, stmt_idx, stmt, block): # pylint:disable=unused-argument
raise HasCallNotification
def _handle_Call(self, expr_idx, expr, stmt_idx, stmt, block): # pylint:disable=unused-argument
raise HasCallNotification
def _handle_FunctionLikeMacro(self, expr_idx, expr, stmt_idx, stmt, block): # pylint:disable=unused-argument
raise HasCallNotification

View file

@ -1,49 +1,64 @@
from __future__ import annotations
from typing import cast, Any, TYPE_CHECKING
import copy
import logging
from typing import TYPE_CHECKING, Any, cast
import archinfo
from angr.ailment import Stmt, Expr, Const
from angr.ailment.manager import Manager
from angr.ailment import Const, Expr, Stmt
from angr.ailment.manager import Manager
from angr.analyses.s_reaching_definitions import SRDAView
from angr.calling_conventions import (
SimCC,
SimComboArg,
SimFunctionArgument,
SimReferenceArgument,
SimRegArg,
SimStackArg,
SimStructArg,
)
from angr.knowledge_plugins.key_definitions.constants import OP_BEFORE
from angr.procedures.stubs.format_parser import FormatParser, FormatSpecifier
from angr.sim_type import (
SimType,
SimTypeBottom,
SimTypePointer,
SimTypeChar,
SimTypeFloat,
SimTypeFunction,
SimTypePointer,
)
from angr.calling_conventions import (
SimReferenceArgument,
SimRegArg,
SimStackArg,
SimCC,
SimStructArg,
SimComboArg,
SimFunctionArgument,
)
from angr.knowledge_plugins.key_definitions.constants import OP_BEFORE
from angr.analyses import Analysis, register_analysis
from angr.analyses.s_reaching_definitions import SRDAView
from angr.utils.types import dereference_simtype_by_lib
from .stackarg_offset_manager import StackArgOffsetManager
from .variable_map import variable_map_of
if TYPE_CHECKING:
from angr.analyses.s_reaching_definitions import SRDAModel
from angr.knowledge_plugins.functions import Function
from angr.knowledge_plugins.key_definitions.definition import Definition
from angr.project import Project
l = logging.getLogger(name=__name__)
class CallSiteMaker(Analysis):
class CallSiteMaker:
"""
Add calling convention, declaration, and args to a call site.
"""
def __init__(self, block, *, ail_manager: Manager, reaching_definitions=None, stack_pointer_tracker=None):
def __init__(
self,
project: Project,
block,
*,
ail_manager: Manager,
reaching_definitions: SRDAModel | None = None,
stack_pointer_tracker=None,
):
self.project = project
self.kb = project.kb
self.block = block
self._reaching_definitions = reaching_definitions
@ -51,7 +66,8 @@ class CallSiteMaker(Analysis):
self._ail_manager: Manager = ail_manager
self.result_block = None
self.stack_arg_offsets: set[tuple[int, int]] | None = None # call ins addr, stack_offset
# block addr, call ins addr, stack offset, arg size (in bytes)
self.stackarg_offset_manager: StackArgOffsetManager = StackArgOffsetManager(self.project.arch.bits)
self.removed_vvar_ids: set[int] = set()
self._analyze()
@ -62,9 +78,9 @@ class CallSiteMaker(Analysis):
last_stmt = self.block.statements[-1]
if type(last_stmt) is Stmt.SideEffectStatement:
if isinstance(last_stmt, Stmt.SideEffectStatement):
call_expr = last_stmt.expr
elif isinstance(last_stmt, Stmt.Assignment) and type(last_stmt.src) is Expr.Call:
elif isinstance(last_stmt, Stmt.Assignment) and isinstance(last_stmt.src, Expr.Call):
call_expr = last_stmt.src
elif (
isinstance(last_stmt, Stmt.Assignment)
@ -206,7 +222,6 @@ class CallSiteMaker(Analysis):
vvar_use,
Expr.Const(
self._ail_manager.next_atom(),
None,
(offset - vvar_def_reg_offset) * 8,
8,
),
@ -227,7 +242,6 @@ class CallSiteMaker(Analysis):
else:
reg = Expr.Register(
self._atom_idx(),
None,
offset,
size * 8,
reg_name=arg_loc.reg_name,
@ -321,15 +335,18 @@ class CallSiteMaker(Analysis):
"Failed to calculate the stack pointer offset at pc %#x. You may find redundant Store statements.",
call_expr.tags["ins_addr"],
)
self.stack_arg_offsets = None
else:
if sp_offset >= (1 << (self.project.arch.bits - 1)):
# make it a signed integer
sp_offset -= 1 << self.project.arch.bits
self.stack_arg_offsets = {
(call_expr.tags["ins_addr"], sp_offset + arg.stack_offset - stackarg_sp_diff)
for arg in stack_arg_locs
}
for arg in stack_arg_locs:
self.stackarg_offset_manager.add_call_stack_arg_offset(
self.block.addr,
self.block.idx,
call_expr.tags["ins_addr"],
sp_offset + arg.stack_offset - stackarg_sp_diff,
arg.size,
)
if isinstance(last_stmt, Stmt.SideEffectStatement):
ret_expr = last_stmt.ret_expr
@ -374,12 +391,13 @@ class CallSiteMaker(Analysis):
new_call = Expr.Call(
call_expr.idx,
call_expr.target,
calling_convention=cc,
prototype=prototype,
args=args,
arg_vvars=arg_vvars,
**tags,
)
vm = variable_map_of(self._ail_manager)
vm.set_calling_convention(new_call, cc)
vm.set_prototype(new_call, prototype)
if isinstance(last_stmt, Stmt.Assignment):
if not new_call.bits:
new_call.bits = last_stmt.src.bits
@ -416,11 +434,14 @@ class CallSiteMaker(Analysis):
return None
stmt = self.block.statements[def_.codeloc.stmt_idx]
if type(stmt) is Stmt.Assignment:
if isinstance(stmt, Stmt.Assignment):
return stmt.dst
if type(stmt) is Stmt.Store:
if isinstance(stmt, Stmt.Store):
return stmt.addr
l.warning("TODO: Unsupported statement type %s for definitions.", type(stmt))
l.warning(
"TODO: Unsupported statement type %s for definitions.",
getattr(stmt, "kind_name", None) or type(stmt).__name__,
)
return None
def _resolve_register_argument(self, arg_loc) -> tuple[Expr.Expression | None, Expr.VirtualVariable] | None:
@ -428,7 +449,7 @@ class CallSiteMaker(Analysis):
if self._reaching_definitions is not None:
# Find its definition
view = SRDAView(self._reaching_definitions.model)
view = SRDAView(self._reaching_definitions)
vvar = view.get_reg_vvar_by_stmt(
offset,
arg_loc.size,
@ -466,7 +487,7 @@ class CallSiteMaker(Analysis):
if self._reaching_definitions is not None:
# find its definition
view = SRDAView(self._reaching_definitions.model)
view = SRDAView(self._reaching_definitions)
vvar = view.get_stack_vvar_by_stmt(
sp_offset, size, self.block.addr, self.block.idx, len(self.block.statements) - 1, OP_BEFORE
)
@ -519,7 +540,7 @@ class CallSiteMaker(Analysis):
:return:
"""
if type(stmt.target) is Expr.Const:
if isinstance(stmt.target, Expr.Const):
return stmt.target.value
return None
@ -639,6 +660,3 @@ class CallSiteMaker(Analysis):
def _atom_idx(self) -> int:
return self._ail_manager.next_atom()
register_analysis(CallSiteMaker, "AILCallSiteMaker")

View file

@ -1,7 +1,8 @@
from __future__ import annotations
from .amd64_ccalls import AMD64CCallRewriter
from .x86_ccalls import X86CCallRewriter
from .arm_ccalls import ARMCCallRewriter
from .x86_ccalls import X86CCallRewriter
CCALL_REWRITERS = {
"X86": X86CCallRewriter,

View file

@ -1,8 +1,10 @@
from __future__ import annotations
from angr.ailment import Expr
from angr.ailment import Expr
from angr.analyses.decompiler.variable_map import variable_map_of
from angr.calling_conventions import SimCCUsercall
from angr.engines.vex.claripy.ccall import data
from .rewriter_base import CCallRewriterBase
AMD64_CondTypes = data["AMD64"]["CondTypes"]
@ -207,7 +209,7 @@ class AMD64CCallRewriter(CCallRewriterBase):
r = Expr.BinaryOp(
ccall.idx,
expr_op,
(dep_1, Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits)),
(dep_1, Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits)),
False,
**ccall.tags,
)
@ -230,7 +232,7 @@ class AMD64CCallRewriter(CCallRewriterBase):
)
expr_op = "CmpEQ" if cond_v == AMD64_CondTypes["CondZ"] else "CmpNE"
zero = Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits)
r = Expr.BinaryOp(ccall.idx, expr_op, (dep_1, zero), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
if op_v == AMD64_OpTypes["G_CC_OP_COPY"]:
@ -238,12 +240,13 @@ class AMD64CCallRewriter(CCallRewriterBase):
bitmask = AMD64_CondBitMasks["G_CC_MASK_Z"]
assert isinstance(bitmask, int)
flag = Expr.Const(self.ail_manager.next_atom(), None, bitmask, dep_1.bits)
flag = Expr.Const(self.ail_manager.next_atom(), bitmask, dep_1.bits)
masked_dep = Expr.BinaryOp(
self.ail_manager.next_atom(), "And", [dep_1, flag], False, **ccall.tags
)
zero = Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits)
expr_op = "CmpEQ" if cond_v == AMD64_CondTypes["CondZ"] else "CmpNE"
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits)
# dep_1 holds the old flags: ZF is *set* iff the masked bit is non-zero
expr_op = "CmpNE" if cond_v == AMD64_CondTypes["CondZ"] else "CmpEQ"
r = Expr.BinaryOp(ccall.idx, expr_op, (masked_dep, zero), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
@ -264,7 +267,7 @@ class AMD64CCallRewriter(CCallRewriterBase):
)
expr_op = "CmpEQ" if cond_v == AMD64_CondTypes["CondZ"] else "CmpNE"
zero = Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits)
r = Expr.BinaryOp(ccall.idx, expr_op, (dep_1, zero), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
elif cond_v == AMD64_CondTypes["CondL"]:
@ -313,10 +316,59 @@ class AMD64CCallRewriter(CCallRewriterBase):
AMD64_OpTypes["G_CC_OP_LOGICL"],
ccall.tags,
)
zero = Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits)
r = Expr.BinaryOp(ccall.idx, "CmpLT", (dep_1, zero), True, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
elif cond_v == AMD64_CondTypes["CondNL"]:
if op_v in {
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
AMD64_OpTypes["G_CC_OP_SUBQ"],
}:
# CondNL (jge) is SF == OF, i.e. dep_1 >=s dep_2
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
ccall.tags,
)
dep_2 = self._fix_size(
dep_2,
op_v,
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
ccall.tags,
)
r = Expr.BinaryOp(ccall.idx, "CmpGE", (dep_1, dep_2), True, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
if op_v in {
AMD64_OpTypes["G_CC_OP_LOGICB"],
AMD64_OpTypes["G_CC_OP_LOGICW"],
AMD64_OpTypes["G_CC_OP_LOGICL"],
AMD64_OpTypes["G_CC_OP_LOGICQ"],
}:
# and/or/xor clear OF, so CondNL = SF == 0, i.e. the result dep_1 >=s 0
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_LOGICB"],
AMD64_OpTypes["G_CC_OP_LOGICW"],
AMD64_OpTypes["G_CC_OP_LOGICL"],
ccall.tags,
)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits)
r = Expr.BinaryOp(ccall.idx, "CmpGE", (dep_1, zero), True, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
elif cond_v == AMD64_CondTypes["CondNBE"]:
if op_v in {
AMD64_OpTypes["G_CC_OP_SUBB"],
@ -345,8 +397,10 @@ class AMD64CCallRewriter(CCallRewriterBase):
r = Expr.BinaryOp(ccall.idx, "CmpGT", (dep_1, dep_2), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
elif cond_v == AMD64_CondTypes["CondB"]:
if op_v in {
elif cond_v in {AMD64_CondTypes["CondB"], AMD64_CondTypes["CondBE"]}:
# CondB tests CF; CondBE tests CF | ZF
is_be = cond_v == AMD64_CondTypes["CondBE"]
if not is_be and op_v in {
AMD64_OpTypes["G_CC_OP_ADDB"],
AMD64_OpTypes["G_CC_OP_ADDW"],
AMD64_OpTypes["G_CC_OP_ADDL"],
@ -371,21 +425,25 @@ class AMD64CCallRewriter(CCallRewriterBase):
ccall.tags,
)
return Expr.Call(
cfadd_call = Expr.Call(
ccall.idx,
"__CFADD__",
calling_convention=SimCCUsercall(self.project.arch, [], None),
args=[dep_1, dep_2],
bits=ccall.bits,
**ccall.tags,
)
variable_map_of(self.ail_manager).set_calling_convention(
cfadd_call, SimCCUsercall(self.project.arch, [], None)
)
return cfadd_call
if op_v in {
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
AMD64_OpTypes["G_CC_OP_SUBQ"],
}:
# dep_1 <u dep_2
# CF is dep_1 <u dep_2 and ZF is dep_1 == dep_2, so
# dep_1 <u dep_2 for CondB, dep_1 <=u dep_2 for CondBE
dep_1 = self._fix_size(
dep_1,
@ -406,12 +464,110 @@ class AMD64CCallRewriter(CCallRewriterBase):
r = Expr.BinaryOp(
ccall.idx,
"CmpLT",
"CmpLE" if is_be else "CmpLT",
(dep_1, dep_2),
False,
**ccall.tags,
)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
if op_v in {
AMD64_OpTypes["G_CC_OP_LOGICB"],
AMD64_OpTypes["G_CC_OP_LOGICW"],
AMD64_OpTypes["G_CC_OP_LOGICL"],
AMD64_OpTypes["G_CC_OP_LOGICQ"],
}:
# and/or/xor always clear CF, so CondB is never true and CondBE degenerates to ZF
if not is_be:
return Expr.Const(self.ail_manager.next_atom(), 0, ccall.bits, **ccall.tags)
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_LOGICB"],
AMD64_OpTypes["G_CC_OP_LOGICW"],
AMD64_OpTypes["G_CC_OP_LOGICL"],
ccall.tags,
)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits)
r = Expr.BinaryOp(ccall.idx, "CmpEQ", (dep_1, zero), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
elif cond_v == AMD64_CondTypes["CondNB"]:
if op_v in {
AMD64_OpTypes["G_CC_OP_ADDB"],
AMD64_OpTypes["G_CC_OP_ADDW"],
AMD64_OpTypes["G_CC_OP_ADDL"],
AMD64_OpTypes["G_CC_OP_ADDQ"],
}:
# CondNB is !CF, i.e. the negation of the __CFADD__ carry test that CondB
# emits. An inline (a + b) >=u a comparison would be wrong here: C integer
# promotion keeps sub-int additions from wrapping, making it a tautology at
# 8/16-bit widths.
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_ADDB"],
AMD64_OpTypes["G_CC_OP_ADDW"],
AMD64_OpTypes["G_CC_OP_ADDL"],
ccall.tags,
)
dep_2 = self._fix_size(
dep_2,
op_v,
AMD64_OpTypes["G_CC_OP_ADDB"],
AMD64_OpTypes["G_CC_OP_ADDW"],
AMD64_OpTypes["G_CC_OP_ADDL"],
ccall.tags,
)
cfadd_call = Expr.Call(
self.ail_manager.next_atom(),
"__CFADD__",
args=[dep_1, dep_2],
bits=ccall.bits,
**ccall.tags,
)
variable_map_of(self.ail_manager).set_calling_convention(
cfadd_call, SimCCUsercall(self.project.arch, [], None)
)
zero = Expr.Const(self.ail_manager.next_atom(), 0, cfadd_call.bits)
r = Expr.BinaryOp(ccall.idx, "CmpEQ", (cfadd_call, zero), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
if op_v in {
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
AMD64_OpTypes["G_CC_OP_SUBQ"],
}:
# dep_1 >=u dep_2
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
ccall.tags,
)
dep_2 = self._fix_size(
dep_2,
op_v,
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
ccall.tags,
)
r = Expr.BinaryOp(ccall.idx, "CmpGE", (dep_1, dep_2), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
if op_v in {
AMD64_OpTypes["G_CC_OP_LOGICB"],
AMD64_OpTypes["G_CC_OP_LOGICW"],
AMD64_OpTypes["G_CC_OP_LOGICL"],
AMD64_OpTypes["G_CC_OP_LOGICQ"],
}:
# and/or/xor always clear CF, so CondNB is always true
return Expr.Const(self.ail_manager.next_atom(), 1, ccall.bits, **ccall.tags)
elif (
cond_v == AMD64_CondTypes["CondS"]
and op_v
@ -491,6 +647,184 @@ class AMD64CCallRewriter(CCallRewriterBase):
)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
elif cond_v in {AMD64_CondTypes["CondO"], AMD64_CondTypes["CondNO"]}:
# overflow flag (jo / jno)
is_no = cond_v == AMD64_CondTypes["CondNO"]
if op_v in {
AMD64_OpTypes["G_CC_OP_LOGICB"],
AMD64_OpTypes["G_CC_OP_LOGICW"],
AMD64_OpTypes["G_CC_OP_LOGICL"],
AMD64_OpTypes["G_CC_OP_LOGICQ"],
}:
# and/or/xor always clear OF: CondO -> 0, CondNO -> 1
return Expr.Const(self.ail_manager.next_atom(), 1 if is_no else 0, ccall.bits, **ccall.tags)
if op_v in {
AMD64_OpTypes["G_CC_OP_ADDB"],
AMD64_OpTypes["G_CC_OP_ADDW"],
AMD64_OpTypes["G_CC_OP_ADDL"],
AMD64_OpTypes["G_CC_OP_ADDQ"],
}:
# signed overflow of dep_1 + dep_2
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_ADDB"],
AMD64_OpTypes["G_CC_OP_ADDW"],
AMD64_OpTypes["G_CC_OP_ADDL"],
ccall.tags,
)
dep_2 = self._fix_size(
dep_2,
op_v,
AMD64_OpTypes["G_CC_OP_ADDB"],
AMD64_OpTypes["G_CC_OP_ADDW"],
AMD64_OpTypes["G_CC_OP_ADDL"],
ccall.tags,
)
return self._overflow_helper(ccall, "__OFADD__", dep_1, dep_2, is_no)
if op_v in {
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
AMD64_OpTypes["G_CC_OP_SUBQ"],
}:
# signed overflow of dep_1 - dep_2
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
ccall.tags,
)
dep_2 = self._fix_size(
dep_2,
op_v,
AMD64_OpTypes["G_CC_OP_SUBB"],
AMD64_OpTypes["G_CC_OP_SUBW"],
AMD64_OpTypes["G_CC_OP_SUBL"],
ccall.tags,
)
return self._overflow_helper(ccall, "__OFSUB__", dep_1, dep_2, is_no)
if op_v in {
AMD64_OpTypes["G_CC_OP_UMULB"],
AMD64_OpTypes["G_CC_OP_UMULW"],
AMD64_OpTypes["G_CC_OP_UMULL"],
AMD64_OpTypes["G_CC_OP_UMULQ"],
}:
# unsigned multiply overflow: high half of the full product is nonzero
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_UMULB"],
AMD64_OpTypes["G_CC_OP_UMULW"],
AMD64_OpTypes["G_CC_OP_UMULL"],
ccall.tags,
)
dep_2 = self._fix_size(
dep_2,
op_v,
AMD64_OpTypes["G_CC_OP_UMULB"],
AMD64_OpTypes["G_CC_OP_UMULW"],
AMD64_OpTypes["G_CC_OP_UMULL"],
ccall.tags,
)
return self._overflow_helper(ccall, "__OFUMUL__", dep_1, dep_2, is_no)
if op_v in {
AMD64_OpTypes["G_CC_OP_SMULB"],
AMD64_OpTypes["G_CC_OP_SMULW"],
AMD64_OpTypes["G_CC_OP_SMULL"],
AMD64_OpTypes["G_CC_OP_SMULQ"],
}:
# signed multiply overflow
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_SMULB"],
AMD64_OpTypes["G_CC_OP_SMULW"],
AMD64_OpTypes["G_CC_OP_SMULL"],
ccall.tags,
)
dep_2 = self._fix_size(
dep_2,
op_v,
AMD64_OpTypes["G_CC_OP_SMULB"],
AMD64_OpTypes["G_CC_OP_SMULW"],
AMD64_OpTypes["G_CC_OP_SMULL"],
ccall.tags,
)
return self._overflow_helper(ccall, "__OFSMUL__", dep_1, dep_2, is_no)
if op_v in {
AMD64_OpTypes["G_CC_OP_INCB"],
AMD64_OpTypes["G_CC_OP_INCW"],
AMD64_OpTypes["G_CC_OP_INCL"],
AMD64_OpTypes["G_CC_OP_INCQ"],
}:
# inc overflows only when the result is the signed minimum
nbits = self._op_nbits(
op_v,
AMD64_OpTypes["G_CC_OP_INCB"],
AMD64_OpTypes["G_CC_OP_INCW"],
AMD64_OpTypes["G_CC_OP_INCL"],
)
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_INCB"],
AMD64_OpTypes["G_CC_OP_INCW"],
AMD64_OpTypes["G_CC_OP_INCL"],
ccall.tags,
)
signmin = Expr.Const(self.ail_manager.next_atom(), 1 << (nbits - 1), dep_1.bits)
expr_op = "CmpNE" if is_no else "CmpEQ"
r = Expr.BinaryOp(ccall.idx, expr_op, (dep_1, signmin), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
if op_v in {
AMD64_OpTypes["G_CC_OP_DECB"],
AMD64_OpTypes["G_CC_OP_DECW"],
AMD64_OpTypes["G_CC_OP_DECL"],
AMD64_OpTypes["G_CC_OP_DECQ"],
}:
# dec overflows only when the result is the signed maximum
nbits = self._op_nbits(
op_v,
AMD64_OpTypes["G_CC_OP_DECB"],
AMD64_OpTypes["G_CC_OP_DECW"],
AMD64_OpTypes["G_CC_OP_DECL"],
)
dep_1 = self._fix_size(
dep_1,
op_v,
AMD64_OpTypes["G_CC_OP_DECB"],
AMD64_OpTypes["G_CC_OP_DECW"],
AMD64_OpTypes["G_CC_OP_DECL"],
ccall.tags,
)
signmax = Expr.Const(self.ail_manager.next_atom(), (1 << (nbits - 1)) - 1, dep_1.bits)
expr_op = "CmpNE" if is_no else "CmpEQ"
r = Expr.BinaryOp(ccall.idx, expr_op, (dep_1, signmax), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
if op_v == AMD64_OpTypes["G_CC_OP_COPY"]:
# dep_1 holds the packed flags; test the stored OF bit
bitmask = AMD64_CondBitMasks["G_CC_MASK_O"]
assert isinstance(bitmask, int)
flag = Expr.Const(self.ail_manager.next_atom(), bitmask, dep_1.bits)
masked_dep = Expr.BinaryOp(
self.ail_manager.next_atom(), "And", [dep_1, flag], False, **ccall.tags
)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits)
expr_op = "CmpEQ" if is_no else "CmpNE"
r = Expr.BinaryOp(ccall.idx, expr_op, (masked_dep, zero), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
elif ccall.callee == "amd64g_calculate_rflags_c":
# calculate the carry flag
op = ccall.operands[0]
@ -535,8 +869,8 @@ class AMD64CCallRewriter(CCallRewriterBase):
],
False,
),
Expr.Const(self.ail_manager.next_atom(), None, 0, ccall.bits),
Expr.Const(self.ail_manager.next_atom(), None, 1, ccall.bits),
Expr.Const(self.ail_manager.next_atom(), 0, ccall.bits),
Expr.Const(self.ail_manager.next_atom(), 1, ccall.bits),
**ccall.tags,
)
@ -595,10 +929,10 @@ class AMD64CCallRewriter(CCallRewriterBase):
Expr.BinaryOp(
self.ail_manager.next_atom(),
"And",
[ndep, Expr.Const(self.ail_manager.next_atom(), None, bitmask, 64)],
[ndep, Expr.Const(self.ail_manager.next_atom(), bitmask, 64)],
False,
),
Expr.Const(self.ail_manager.next_atom(), None, bitmask_1, 64),
Expr.Const(self.ail_manager.next_atom(), bitmask_1, 64),
],
False,
**ccall.tags,
@ -606,6 +940,33 @@ class AMD64CCallRewriter(CCallRewriterBase):
return None
@staticmethod
def _op_nbits(op_v: int, type_8bit, type_16bit, type_32bit) -> int:
if op_v == type_8bit:
return 8
if op_v == type_16bit:
return 16
if op_v == type_32bit:
return 32
return 64
def _overflow_helper(self, ccall, name: str, dep_1, dep_2, is_no: bool):
# Emit a named overflow-helper call (mirrors the __CFADD__ arm). The helper
# returns a 0/1 flag; for the negated condition (CondNO) compare it to 0.
call = Expr.Call(
ccall.idx,
name,
args=[dep_1, dep_2],
bits=ccall.bits,
**ccall.tags,
)
variable_map_of(self.ail_manager).set_calling_convention(call, SimCCUsercall(self.project.arch, [], None))
if not is_no:
return call
zero = Expr.Const(self.ail_manager.next_atom(), 0, ccall.bits)
r = Expr.BinaryOp(self.ail_manager.next_atom(), "CmpEQ", (call, zero), False, **ccall.tags)
return Expr.Convert(self.ail_manager.next_atom(), r.bits, ccall.bits, False, r, **ccall.tags)
def _fix_size(self, expr, op_v: int, type_8bit, type_16bit, type_32bit, tags):
if op_v == type_8bit:
bits = 8
@ -617,6 +978,6 @@ class AMD64CCallRewriter(CCallRewriterBase):
bits = 64
if bits < 64:
if isinstance(expr, Expr.Const):
return Expr.Const(expr.idx, None, expr.value_int & ((1 << bits) - 1), bits, **tags)
return Expr.Const(expr.idx, expr.value_int & ((1 << bits) - 1), bits, **tags)
return Expr.Convert(self.ail_manager.next_atom(), 64, bits, False, expr, **tags)
return expr

View file

@ -2,6 +2,12 @@ from __future__ import annotations
from angr.ailment import Expr
from angr.engines.vex.claripy.ccall import (
ARMG_CC_OP_ADD,
ARMG_CC_OP_LOGIC,
ARMG_CC_OP_MUL,
ARMG_CC_OP_NUMBER,
ARMG_CC_OP_SBB,
ARMG_CC_OP_SUB,
ARMCondAL,
ARMCondEQ,
ARMCondGE,
@ -16,12 +22,6 @@ from angr.engines.vex.claripy.ccall import (
ARMCondNE,
ARMCondNV,
ARMCondPL,
ARMG_CC_OP_ADD,
ARMG_CC_OP_LOGIC,
ARMG_CC_OP_MUL,
ARMG_CC_OP_NUMBER,
ARMG_CC_OP_SBB,
ARMG_CC_OP_SUB,
)
from .rewriter_base import CCallRewriterBase
@ -99,9 +99,9 @@ class ARMCCallRewriter(CCallRewriterBase):
# AL (always) / NV (never) — independent of operation
if cond_v == ARMCondAL:
return Expr.Const(ccall.idx, None, 1, ccall.bits, **ccall.tags)
return Expr.Const(ccall.idx, 1, ccall.bits, **ccall.tags)
if cond_v == ARMCondNV:
return Expr.Const(ccall.idx, None, 0, ccall.bits, **ccall.tags)
return Expr.Const(ccall.idx, 0, ccall.bits, **ccall.tags)
if op_v == ARMG_CC_OP_SUB:
return self._rewrite_sub(ccall, cond_v, inv, dep_1, dep_2)
@ -158,7 +158,7 @@ class ARMCCallRewriter(CCallRewriterBase):
op_v = cc_op.value_int
dep_1 = ccall.operands[1]
dep_2 = ccall.operands[2]
zero = Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits, **ccall.tags)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits, **ccall.tags)
if op_v == ARMG_CC_OP_SUB:
# N = sign(dep_1 - dep_2) → (dep_1 - dep_2) <s 0 → dep_1 <s dep_2
@ -184,7 +184,7 @@ class ARMCCallRewriter(CCallRewriterBase):
op_v = cc_op.value_int
dep_1 = ccall.operands[1]
dep_2 = ccall.operands[2]
zero = Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits, **ccall.tags)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits, **ccall.tags)
if op_v == ARMG_CC_OP_SUB:
# Z = (dep_1 == dep_2)
@ -285,7 +285,7 @@ class ARMCCallRewriter(CCallRewriterBase):
ADD: flags from ``dep_1 + dep_2``.
"""
add_expr = Expr.BinaryOp(self.ail_manager.next_atom(), "Add", (dep_1, dep_2), signed=False, **ccall.tags)
zero = Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits, **ccall.tags)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits, **ccall.tags)
# EQ/NE — Z flag: (dep_1 + dep_2) == 0
if cond_v in {ARMCondEQ, ARMCondNE}:
@ -314,7 +314,7 @@ class ARMCCallRewriter(CCallRewriterBase):
LOGIC: flags from AND/OR/XOR result (dep_1 = result).
MUL: flags from multiply result (dep_1 = result).
"""
zero = Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits, **ccall.tags)
zero = Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits, **ccall.tags)
# EQ/NE — Z flag: dep_1 == 0
if cond_v in {ARMCondEQ, ARMCondNE}:

View file

@ -1,7 +1,8 @@
from __future__ import annotations
from typing import TYPE_CHECKING
import angr.ailment as ailment
from angr import ailment
if TYPE_CHECKING:
from angr.ailment.manager import Manager
@ -24,6 +25,9 @@ class CCallRewriterBase:
self.project = project
self.ail_manager = ail_manager
self.result: ailment.Expr.Expression | None = self._rewrite(ccall)
assert self.result is None or self.result.bits == ccall.bits, (
f"Rewritten ccall expression has {self.result.bits} bits, expecting {ccall.bits} bits"
)
if rename_ccalls and self.result is None and ccall.callee != "_ccall":
renamed = ccall.copy()
renamed.callee = "_ccall"

View file

@ -1,10 +1,11 @@
from __future__ import annotations
from angr.ailment import Expr
from angr.ailment.expression import Call, Convert, VirtualVariable
from angr.analyses.decompiler.variable_map import variable_map_of
from angr.engines.vex.claripy.ccall import data
from angr.procedures.definitions import SIM_LIBRARIES
from .rewriter_base import CCallRewriterBase
X86_CondTypes = data["X86"]["CondTypes"]
@ -90,7 +91,6 @@ class X86CCallRewriter(CCallRewriterBase):
)
max_signed = Expr.Const(
self.ail_manager.next_atom(),
None,
(1 << (dep_1.bits - 1)),
bits=dep_1.bits * 2,
**ccall.tags,
@ -113,7 +113,6 @@ class X86CCallRewriter(CCallRewriterBase):
)
max_signed = Expr.Const(
self.ail_manager.next_atom(),
None,
(1 << (dep_1.bits - 1)),
bits=dep_1.bits,
**ccall.tags,
@ -129,7 +128,6 @@ class X86CCallRewriter(CCallRewriterBase):
# dep_1 is the result
overflowed = Expr.Const(
self.ail_manager.next_atom(),
None,
1 << (dep_1.bits - 1),
dep_1.bits,
**ccall.tags,
@ -147,8 +145,8 @@ class X86CCallRewriter(CCallRewriterBase):
return Expr.ITE(
ccall.idx,
ret_cond,
Expr.Const(self.ail_manager.next_atom(), None, 0, ccall.bits, **ccall.tags),
Expr.Const(self.ail_manager.next_atom(), None, 1, ccall.bits, **ccall.tags),
Expr.Const(self.ail_manager.next_atom(), 0, ccall.bits, **ccall.tags),
Expr.Const(self.ail_manager.next_atom(), 1, ccall.bits, **ccall.tags),
**ccall.tags,
)
elif cond_v == X86_CondTypes["CondZ"]:
@ -168,7 +166,6 @@ class X86CCallRewriter(CCallRewriterBase):
)
zero = Expr.Const(
self.ail_manager.next_atom(),
None,
0,
dep_1.bits,
**ccall.tags,
@ -210,7 +207,7 @@ class X86CCallRewriter(CCallRewriterBase):
cmp = Expr.BinaryOp(
ccall.idx,
"CmpEQ",
(dep_1, Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits, **ccall.tags)),
(dep_1, Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits, **ccall.tags)),
True,
bits=1,
**ccall.tags,
@ -246,7 +243,7 @@ class X86CCallRewriter(CCallRewriterBase):
cmp = Expr.BinaryOp(
ccall.idx,
"CmpLT",
(dep_1, Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits, **ccall.tags)),
(dep_1, Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits, **ccall.tags)),
True,
**ccall.tags,
)
@ -275,7 +272,6 @@ class X86CCallRewriter(CCallRewriterBase):
)
zero = Expr.Const(
self.ail_manager.next_atom(),
None,
0,
dep_1.bits,
**ccall.tags,
@ -298,7 +294,21 @@ class X86CCallRewriter(CCallRewriterBase):
}:
# dep_1 <= dep_2 if CondBE
# dep_1 < dep_2 if CondB
return Expr.BinaryOp(
dep_1 = self._fix_size(
dep_1,
op_v,
X86_OpTypes["G_CC_OP_SUBB"],
X86_OpTypes["G_CC_OP_SUBW"],
ccall.tags,
)
dep_2 = self._fix_size(
dep_2,
op_v,
X86_OpTypes["G_CC_OP_SUBB"],
X86_OpTypes["G_CC_OP_SUBW"],
ccall.tags,
)
cmp = Expr.BinaryOp(
ccall.idx,
"CmpLE" if cond_v == X86_CondTypes["CondBE"] else "CmpLT",
(dep_1, dep_2),
@ -306,6 +316,9 @@ class X86CCallRewriter(CCallRewriterBase):
bits=1,
**ccall.tags,
)
return Expr.Convert(
self.ail_manager.next_atom(), cmp.bits, ccall.bits, False, cmp, **ccall.tags
)
if op_v in {
X86_OpTypes["G_CC_OP_LOGICB"],
X86_OpTypes["G_CC_OP_LOGICW"],
@ -316,7 +329,7 @@ class X86CCallRewriter(CCallRewriterBase):
cmp = Expr.BinaryOp(
ccall.idx,
"CmpLE" if cond_v == X86_CondTypes["CondBE"] else "CmpLT",
(dep_1, Expr.Const(self.ail_manager.next_atom(), None, 0, dep_1.bits, **ccall.tags)),
(dep_1, Expr.Const(self.ail_manager.next_atom(), 0, dep_1.bits, **ccall.tags)),
False,
bits=1,
**ccall.tags,
@ -348,10 +361,10 @@ class X86CCallRewriter(CCallRewriterBase):
ccall.idx,
X86_Win32_TIB_Funcs[virtual_addr.value_int],
args=[],
prototype=prototype,
bits=returnty_bits,
**ccall.tags,
)
variable_map_of(self.ail_manager).set_prototype(call_expr, prototype)
call_expr.tags["is_prototype_guessed"] = False
ref_expr = Expr.UnaryOp(self.ail_manager.next_atom(), "Reference", call_expr, **ccall.tags)
if returnty_bits == ccall.bits:
@ -370,6 +383,6 @@ class X86CCallRewriter(CCallRewriterBase):
bits = 32
if bits < 32:
if isinstance(expr, Expr.Const):
return Expr.Const(expr.idx, None, expr.value & ((1 << bits) - 1), bits, **tags)
return Expr.Const(expr.idx, expr.value & ((1 << bits) - 1), bits, **tags)
return Expr.Convert(self.ail_manager.next_atom(), 32, bits, False, expr, **tags)
return expr

File diff suppressed because it is too large Load diff

View file

@ -1,38 +1,39 @@
from __future__ import annotations
from collections import defaultdict, OrderedDict
from typing import Any, TYPE_CHECKING
from collections.abc import Callable
from collections.abc import Generator
import operator
import logging
import operator
from collections import OrderedDict, defaultdict
from collections.abc import Callable, Generator
from typing import TYPE_CHECKING, Any
import claripy
import networkx
from angr import ailment
from angr.ailment.expression import StringLiteral
from angr.utils.graph import GraphUtils
from angr.utils.graph import dominates, inverted_idoms
from angr.utils.ail import is_head_controlled_loop_block
from angr.block import Block, BlockNode
from angr.errors import AngrRuntimeError
from angr.utils.ail import is_head_controlled_loop_block
from angr.utils.graph import GraphUtils, dominates, inverted_idoms
from .peephole_optimizations import InvertNegatedLogicalConjunctionsAndDisjunctions, RemoveRedundantNots
from .structuring.structurer_nodes import (
MultiNode,
EmptyBlockNotice,
SequenceNode,
CodeNode,
SwitchCaseNode,
from .region_overlay import RegionOverlay
from .structurer_nodes import (
BreakNode,
CascadingConditionNode,
CodeNode,
ConditionalBreakNode,
LoopNode,
ConditionNode,
ContinueNode,
CascadingConditionNode,
EmptyBlockNotice,
IncompleteSwitchCaseNode,
LoopNode,
MultiNode,
SequenceNode,
SwitchCaseNode,
)
from .graph_region import GraphRegion
from .utils import peephole_optimize_expr
from .variable_map import variable_map_of
if TYPE_CHECKING:
from angr.ailment import Manager
@ -80,9 +81,15 @@ class AILExprIdAnnotation(claripy.Annotation):
return True
@property
def relocateable(self):
def relocatable(self):
return False
def __hash__(self):
return 1
def __eq__(self, other):
return isinstance(other, AILExprIdAnnotation)
#
# Util methods and mapping used during AIL AST to claripy AST conversion
@ -321,7 +328,6 @@ class ConditionProcessor:
edge_cond_left = self.recover_edge_condition(graph, src, dst0)
edge_cond_right = self.recover_edge_condition(graph, src, dst1)
cond = claripy.Not(edge_cond_left) == edge_cond_right
# call claripy.simplify() just in case there are annotations
return claripy.is_true(claripy.simplify(cond)) # type: ignore
def recover_edge_condition(self, graph: networkx.DiGraph, src, dst):
@ -644,7 +650,7 @@ class ConditionProcessor:
return None
if type(block) is IncompleteSwitchCaseNode:
return None
if type(block) is GraphRegion:
if isinstance(block, RegionOverlay):
# normally this should not happen. however, we have test cases that trigger this case.
return None
@ -736,7 +742,7 @@ class ConditionProcessor:
for case in block.cases:
s.extend(cls.get_last_statements(case))
return s
if type(block) is GraphRegion:
if isinstance(block, RegionOverlay):
# normally this should not happen. however, we have test cases that trigger this case.
return []
@ -757,8 +763,8 @@ class ConditionProcessor:
self.ail_manager.next_atom(),
"CmpEQ",
(
ailment.Expr.Register(self.ail_manager.next_atom(), None, self.EXC_COUNTER, 64),
ailment.Expr.Const(self.ail_manager.next_atom(), None, self.EXC_COUNTER, 64),
ailment.Expr.Register(self.ail_manager.next_atom(), self.EXC_COUNTER, 64),
ailment.Expr.Const(self.ail_manager.next_atom(), self.EXC_COUNTER, 64),
),
False,
),
@ -772,7 +778,7 @@ class ConditionProcessor:
return bool_var
return claripy.Not(bool_var)
if type(src_block) is GraphRegion:
if isinstance(src_block, RegionOverlay):
return claripy.true()
# sometimes the last statement is the conditional jump. sometimes it's the first statement of the block
@ -786,13 +792,13 @@ class ConditionProcessor:
if last_stmt is None:
return claripy.true()
if type(last_stmt) is ailment.Stmt.Jump:
if isinstance(last_stmt, ailment.Stmt.Jump):
if isinstance(last_stmt.target, ailment.Expr.Const):
return claripy.true()
# indirect jump
target_ast = self.claripy_ast_from_ail_condition(last_stmt.target, ins_addr=last_stmt.tags["ins_addr"])
return target_ast == dst_block.addr
if type(last_stmt) is ailment.Stmt.ConditionalJump:
if isinstance(last_stmt, ailment.Stmt.ConditionalJump):
bool_var = self.claripy_ast_from_ail_condition(
last_stmt.condition, must_bool=True, ins_addr=last_stmt.tags["ins_addr"]
)
@ -808,18 +814,29 @@ class ConditionProcessor:
#
def _convert_extract(self, hi, lo, expr, tags, memo=None):
# ailment does not support Extract. We translate Extract to Convert and shift.
if lo == 0:
return ailment.Expr.Convert(
# ailment does not support Extract. We translate Extract(hi, lo, expr) to a
# logical right shift by `lo` (dropping the low bits) followed by a Convert
# that truncates to the extracted width.
converted = self.convert_claripy_bool_ast(expr, memo=memo)
if lo != 0:
converted = ailment.Expr.BinaryOp(
self.ail_manager.next_atom(),
expr.size(),
hi + 1,
"Shr",
(
converted,
ailment.Expr.Const(self.ail_manager.next_atom(), lo, expr.size(), **tags),
),
False,
self.convert_claripy_bool_ast(expr, memo=memo),
**tags,
)
raise NotImplementedError("This case will be implemented once encountered.")
return ailment.Expr.Convert(
self.ail_manager.next_atom(),
expr.size(),
hi - lo + 1,
False,
converted,
**tags,
)
def convert_claripy_bool_ast(self, cond, memo=None):
"""
@ -844,7 +861,7 @@ class ConditionProcessor:
return cond
if cond.op in {"BoolS", "BoolV"} and claripy.is_true(claripy.simplify(cond)):
return ailment.Expr.Const(self.ail_manager.next_atom(), None, True, 1)
return ailment.Expr.Const(self.ail_manager.next_atom(), True, 1)
if cond in self._condition_mapping:
return self._condition_mapping[cond]
if cond.op in {"BVS", "BoolS"} and cond.args[0] in self._condition_mapping:
@ -902,12 +919,12 @@ class ConditionProcessor:
"__mod__": lambda cond_, tags: _binary_op_reduce("Mod", cond_.args, tags),
"LShR": lambda cond_, tags: _binary_op_reduce("Shr", cond_.args, tags),
"BVV": lambda cond_, tags: ailment.Expr.Const(
self.ail_manager.next_atom(), None, cond_.args[0], cond_.size(), **tags
self.ail_manager.next_atom(), cond_.args[0], cond_.size(), **tags
),
"BoolV": lambda cond_, tags: (
ailment.Expr.Const(self.ail_manager.next_atom(), None, True, 1, **tags)
ailment.Expr.Const(self.ail_manager.next_atom(), True, 1, **tags)
if cond_.args[0] is True
else ailment.Expr.Const(self.ail_manager.next_atom(), None, False, 1, **tags)
else ailment.Expr.Const(self.ail_manager.next_atom(), False, 1, **tags)
),
"Extract": lambda cond_, tags: self._convert_extract(*cond_.args, tags, memo=memo),
"ZeroExt": lambda cond_, tags: _binary_op_reduce(
@ -949,15 +966,16 @@ class ConditionProcessor:
)
if isinstance(condition, (ailment.Expr.Load, ailment.Expr.Register, ailment.Expr.VirtualVariable)):
# does it have a variable associated?
if condition.variable is not None:
condition_var = variable_map_of(self.ail_manager).variable(condition)
if condition_var is not None:
if condition.bits == 1:
var = claripy.BoolS(
f"ailexpr_{condition!r}-{condition.variable.ident}-{ins_addr:x}",
f"ailexpr_{condition!r}-{condition_var.ident}-{ins_addr:x}",
explicit_name=True,
)
else:
var = claripy.BVS(
f"ailexpr_{condition!r}-{condition.variable.ident}-{ins_addr:x}",
f"ailexpr_{condition!r}-{condition_var.ident}-{ins_addr:x}",
condition.bits,
explicit_name=True,
)
@ -977,7 +995,11 @@ class ConditionProcessor:
if not isinstance(condition.offset, ailment.expression.Const)
else condition.offset.value
)
var = claripy.BVS(f"ailexpr_Extract({offset_expr}, {hash(var_)})", condition.bits, explicit_name=True)
var = claripy.BVS(
f"ailexpr_Extract({condition.bits}, {condition.endness}, {offset_expr}, {hash(var_)})",
condition.bits,
explicit_name=True,
)
self._condition_mapping[var.args[0]] = condition
return var
if isinstance(condition, ailment.expression.Insert):
@ -1006,14 +1028,11 @@ class ConditionProcessor:
self._condition_mapping[var.args[0]] = condition
return var
if isinstance(condition, ailment.Expr.Const):
if condition.value is True or condition.value is False:
var = claripy.BoolV(condition.value)
else:
var = claripy.BVV(condition.value, condition.bits)
if condition.idx is not None:
# we do not want to lose track of this constant when it has idx
var = var.annotate(AILExprIdAnnotation())
self._condition_mapping[var] = condition
var = claripy.BVV(condition.value, condition.bits)
if condition.idx is not None:
# we do not want to lose track of this constant when it has idx
var = var.annotate(AILExprIdAnnotation())
self._condition_mapping[var] = condition
if isinstance(var, claripy.ast.Bits) and var.size() == 1:
var = claripy.true() if var.concrete_value == 1 else claripy.false()
return var

View file

@ -2,8 +2,8 @@ from __future__ import annotations
from .boolean_counter import BooleanCounter
from .call_counter import AILBlockCallCounter
from .expression_counters import OperatorCounter, RegisterExpressionCounter, SingleExpressionCounter
from .seq_cf_structure_counter import ControlFlowStructureCounter
from .expression_counters import SingleExpressionCounter, RegisterExpressionCounter, OperatorCounter
__all__ = (
"AILBlockCallCounter",

View file

@ -1,12 +1,13 @@
from __future__ import annotations
import typing
from angr.ailment import AILBlockViewer
if typing.TYPE_CHECKING:
from angr.ailment.block import Block
from angr.ailment.expression import BinaryOp
from angr.ailment.statement import Statement
from angr.ailment.block import Block
class BooleanCounter(AILBlockViewer):

View file

@ -1,10 +1,10 @@
from __future__ import annotations
from typing import TYPE_CHECKING
from angr.ailment import Block
from angr.ailment.statement import Label, ConditionalJump
from angr.ailment.block_walker import AILBlockViewer
from angr.ailment.statement import ConditionalJump, Label
from angr.analyses.decompiler.sequence_walker import SequenceWalker
if TYPE_CHECKING:
@ -29,10 +29,10 @@ class AILBlockCallCounter(AILBlockViewer):
return
super()._handle_ConditionalJump(stmt_idx, stmt, block)
def _handle_CallExpr(self, expr_idx: int, expr: Call, stmt_idx: int, stmt, block: Block | None):
def _handle_Call(self, expr_idx: int, expr: Call, stmt_idx: int, stmt, block: Block | None):
self.calls += 1
self.call_exprs.append((((block.addr, block.idx) if block is not None else None, stmt_idx), expr))
super()._handle_CallExpr(expr_idx, expr, stmt_idx, stmt, block)
super()._handle_Call(expr_idx, expr, stmt_idx, stmt, block)
class AILCallCounter(SequenceWalker):

View file

@ -1,12 +1,13 @@
from __future__ import annotations
from typing import Any, TYPE_CHECKING
from collections.abc import Iterable
from collections import defaultdict
from collections import defaultdict
from collections.abc import Iterable
from typing import TYPE_CHECKING, Any
from angr.ailment import Block
from angr.ailment.block_walker import AILBlockViewer
from angr.ailment.expression import Expression, Register
from angr.ailment.statement import Statement
from angr.ailment.block_walker import AILBlockViewer
from angr.ailment import Block
if TYPE_CHECKING:
from angr.ailment.expression import BinaryOp, UnaryOp

View file

@ -1,10 +1,10 @@
from __future__ import annotations
from collections import defaultdict
import angr.ailment as ailment
from angr import ailment
from angr.analyses.decompiler.sequence_walker import SequenceWalker
from angr.analyses.decompiler.structuring.structurer_nodes import LoopNode
from angr.analyses.decompiler.structurer_nodes import LoopNode
class ControlFlowStructureCounter(SequenceWalker):

View file

@ -1,56 +1,318 @@
from __future__ import annotations
from typing import Any, TYPE_CHECKING
import json
import time
from typing import TYPE_CHECKING, Any
from angr.protos import decompilation_cache_pb2
from angr.serializable import Serializable
from angr.utils.ail_serialization import (
pack_arg_vvars,
pack_ite_exprs,
pack_static_buffers,
pack_static_vvars,
parse_arg_vvars,
parse_ite_exprs,
parse_static_buffers,
parse_static_vvars,
)
from .clinic import Clinic
from .structured_codegen import BaseStructuredCodeGenerator
if TYPE_CHECKING:
from angr import ailment
from angr.analyses.decompiler.optimization_passes.expr_op_swapper import OpDescriptor
from angr.analyses.typehoon.typevars import TypeVariable, TypeConstraint
from angr.analyses.typehoon.typevars import TypeConstraint, TypeVariable
from angr.knowledge_plugins.cfg import CFGModel
from .notes import DecompilationNote
from .structured_codegen import BaseStructuredCodeGenerator
from .variable_map import VariableMap
class DecompilationCache:
# ---------------------------------------------------------------------------------------------------------------------
# Serialization helpers.
#
# Conventions:
# - Heavy sub-objects (``clinic``, ``codegen``) are embedded as already-serialized bytes (each manages its own format).
# - AIL-typed top-level slots (``arg_vvars``, ``ite_exprs``) use the typed messages from ``ail_types.proto``.
# - ``cfg`` is intentionally not serialized — it comes from the parent Project. Decompilation variables live on
# kb.dec_variables.
# - The 4 typehoon-typed slots are skipped entirely (typehoon is out of scope for now).
# ---------------------------------------------------------------------------------------------------------------------
def _simvar_to_bytes(v) -> bytes:
return type(v).__name__.encode("ascii") + b"\0" + v.serialize()
def _simvar_from_bytes(b: bytes):
import angr.sim_variable as sv_mod # pylint:disable=import-outside-toplevel
sep = b.index(b"\0")
return getattr(sv_mod, b[:sep].decode("ascii")).parse(b[sep + 1 :])
def _serialize_binop_operators(binop_operators, out_msg, set_flag=None) -> None:
if binop_operators is None:
return
if set_flag is not None:
setattr(set_flag[0], set_flag[1], True)
for op_desc, value in binop_operators.items():
entry = out_msg.add()
entry.key_json = op_desc.to_json()
entry.value = value
def _parse_binop_operators(entries):
from angr.analyses.decompiler.optimization_passes.expr_op_swapper import ( # pylint:disable=import-outside-toplevel
OpDescriptor,
)
return {OpDescriptor.from_json(e.key_json): e.value for e in entries}
def _serialize_parameters(params: dict, out_msg) -> None:
"""Translate the 15-key parameters dict into a DecompilationParameters cmessage."""
from angr.analyses.decompiler.optimization_pass_registry import ( # pylint:disable=import-outside-toplevel
pass_to_name,
)
if params.get("flavor") is not None:
out_msg.flavor = params["flavor"]
if "sp_tracker_track_memory" in params:
out_msg.sp_tracker_track_memory = bool(params["sp_tracker_track_memory"])
# Collection-typed parameters are never None (the Decompiler normalizes them to empty collections), so each is
# written directly; an empty collection is left unset and parses back to empty.
out_msg.vars_must_struct.extend(sorted(params.get("vars_must_struct") or ()))
out_msg.desired_variables.extend(sorted(params.get("desired_variables") or ()))
out_msg.inline_functions.extend(sorted(params.get("inline_functions") or ()))
for option, value in params.get("options") or ():
entry = out_msg.options.add()
entry.param = option.param
try:
entry.value_json = json.dumps(value)
except (TypeError, ValueError):
entry.value_json = json.dumps(None)
for cls in params.get("optimization_passes") or ():
out_msg.optimization_passes.append(pass_to_name(cls))
# peephole_optimizations is the one None-able collection: None means "use the default peephole set"
peepholes = params.get("peephole_optimizations")
out_msg.peephole_optimizations_use_default = peepholes is None
if peepholes is not None:
for cls in peepholes:
out_msg.peephole_optimizations.append(pass_to_name(cls))
for k, v in (params.get("expr_comments") or {}).items():
out_msg.expr_comments[k] = v
for k, v in (params.get("stmt_comments") or {}).items():
out_msg.stmt_comments[k] = v
_serialize_binop_operators(params.get("binop_operators") or {}, out_msg.binop_operators)
if params.get("ite_exprs"):
out_msg.ite_exprs.CopyFrom(pack_ite_exprs(params["ite_exprs"]))
if params.get("static_vvars"):
out_msg.static_vvars.CopyFrom(pack_static_vvars(params["static_vvars"]))
if params.get("static_buffers"):
out_msg.static_buffers.CopyFrom(pack_static_buffers(params["static_buffers"]))
out_msg.save_unoptimized_graph = bool(params.get("save_unoptimized_graph"))
def _parse_parameters(msg) -> dict:
"""Always populate every one of the 15 keys in the returned dict; scalar fields that were not set come back as
None and collection fields come back empty, except peephole_optimizations where None means "use the default
peephole set". This matches the decompiler's normalized _cache_parameters, which _can_use_decompilation_cache
compares key by key against the deserialized cache."""
from angr.analyses.decompiler.decompilation_options import PARAM_TO_OPTION # pylint:disable=import-outside-toplevel
from angr.analyses.decompiler.optimization_pass_registry import ( # pylint:disable=import-outside-toplevel
name_to_pass,
)
# Collection-typed values come back as empty collections (never None) so they match the Decompiler's normalized
# _cache_parameters during cache-validity comparison.
return {
"flavor": msg.flavor if msg.HasField("flavor") else None,
"sp_tracker_track_memory": msg.sp_tracker_track_memory if msg.HasField("sp_tracker_track_memory") else None,
"vars_must_struct": set(msg.vars_must_struct),
"desired_variables": frozenset(msg.desired_variables),
"inline_functions": frozenset(msg.inline_functions),
"options": {
(PARAM_TO_OPTION[e.param], json.loads(e.value_json) if e.value_json else None)
for e in msg.options
if e.param in PARAM_TO_OPTION
},
# unresolvable pass names (defining module not imported) drop out; the resulting shorter list will not match
# the live decompiler's parameters, so the cache falls through to a fresh decompilation
"optimization_passes": [cls for n in msg.optimization_passes if (cls := name_to_pass(n)) is not None],
"peephole_optimizations": (
None
if msg.peephole_optimizations_use_default
else [cls for n in msg.peephole_optimizations if (cls := name_to_pass(n)) is not None]
),
"expr_comments": dict(msg.expr_comments),
"stmt_comments": dict(msg.stmt_comments),
"binop_operators": _parse_binop_operators(msg.binop_operators),
"ite_exprs": parse_ite_exprs(msg.ite_exprs) if msg.HasField("ite_exprs") else set(),
"static_vvars": parse_static_vvars(msg.static_vvars) if msg.HasField("static_vvars") else {},
"static_buffers": parse_static_buffers(msg.static_buffers) if msg.HasField("static_buffers") else {},
"save_unoptimized_graph": msg.save_unoptimized_graph,
}
class DecompilationCache(Serializable):
"""
Caches key data structures that can be used later for refining decompilation results, such as retyping variables.
"""
# ``cfg`` is a decompile-time input used only for cache-validity checks. It is not serialized; after
# deserialization it is None until the caller re-attaches it.
__slots__ = (
"addr",
"arg_vvars",
"binop_operators",
"cfg",
"clinic",
"codegen",
"errors",
"func_typevar",
"function_summary",
"ite_exprs",
"max_tv_id",
"notes",
"parameters",
"stack_offset_typevars",
"stackvar_max_sizes",
"timestamp",
"type_constraints",
"var_to_typevar",
"variable_map",
"version",
)
def __init__(self, addr):
import angr # pylint:disable=import-outside-toplevel,cyclic-import
self.parameters: dict[str, Any] = {}
# angr version and creation time of this decompilation
self.version: str = angr.__version__
self.timestamp: int = int(time.time())
self.addr = addr
self.type_constraints: dict[TypeVariable, set[TypeConstraint]] | None = None
self.arg_vvars: dict | None = None
self.cfg: CFGModel | None = None
# Collection-typed fields default to empty containers rather than None, so serialization never has to
# distinguish None from empty.
self.type_constraints: dict[TypeVariable, set[TypeConstraint]] = {}
self.arg_vvars: dict = {}
self.func_typevar: TypeVariable | None = None
self.var_to_typevar: dict | None = None
self.stackvar_max_sizes: dict | None = None
self.stack_offset_typevars: dict | None = None
self.var_to_typevar: dict = {}
self.stackvar_max_sizes: dict = {}
self.stack_offset_typevars: dict = {}
self.codegen: BaseStructuredCodeGenerator | None = None
self.clinic: Clinic | None = None
self.ite_exprs: set[tuple[int, Any]] | None = None
self.binop_operators: dict[OpDescriptor, str] | None = None
self.variable_map: VariableMap | None = None
self.ite_exprs: set[tuple[int, ailment.Expression]] = set()
self.binop_operators: dict[OpDescriptor, str] = {}
self.errors: list[str] = []
self.function_summary: str | None = None
self.notes: dict[str, str] = {}
self.notes: dict[str, DecompilationNote] = {}
self.max_tv_id: int = 0
@property
def local_types(self):
if self.clinic is None or self.clinic.variable_kb is None:
if self.clinic is None or self.clinic.kb is None or self.addr not in self.clinic.kb.dec_variables:
return None
return self.clinic.variable_kb.variables[self.addr].types
return self.clinic.kb.dec_variables[self.addr].types
# -----------------------------------------------------------------------------------------------------------------
# Protobuf serialization. Heavy sub-objects (clinic, codegen) are embedded as already-serialized bytes; AIL-typed
# top-level fields (arg_vvars, ite_exprs) use the typed messages from ail_types.proto. The four typehoon-typed
# slots and the ``cfg`` input are not serialized and come back as None.
# -----------------------------------------------------------------------------------------------------------------
@classmethod
def _get_cmsg(cls):
return decompilation_cache_pb2.DecompilationCache() # pylint:disable=no-member
def serialize_to_cmessage(self):
msg = decompilation_cache_pb2.DecompilationCache(addr=self.addr) # pylint:disable=no-member
if self.clinic is not None:
msg.clinic = self.clinic.serialize()
if self.codegen is not None:
msg.codegen = self.codegen.serialize()
msg.errors.extend(self.errors)
if self.function_summary is not None:
msg.function_summary = self.function_summary
# Collection fields are never None; an empty collection is simply left unset and parses back to empty.
if self.arg_vvars:
msg.arg_vvars.CopyFrom(pack_arg_vvars(self.arg_vvars))
if self.ite_exprs:
msg.ite_exprs.CopyFrom(pack_ite_exprs(self.ite_exprs))
_serialize_binop_operators(self.binop_operators, msg.binop_operators)
for simvar, size in self.stackvar_max_sizes.items():
entry = msg.stackvar_max_sizes.add()
entry.simvar = _simvar_to_bytes(simvar)
entry.max_size = size
msg.version = self.version
msg.timestamp = self.timestamp
# An unset parameters message means "no recorded parameters"; cache-validity checks treat such a cache as
# always usable (matching runs with use_cache=False).
if self.parameters:
_serialize_parameters(self.parameters, msg.parameters)
for k, note in self.notes.items():
msg.notes_json[k] = note.to_json()
return msg
@classmethod
def parse_from_cmessage(
cls,
cmsg,
*,
project=None,
kb=None,
function=None,
cfg=None,
**_,
):
"""Parse a DecompilationCache from a cmessage. Runtime back-references (project, kb, function, cfg) are
passed through to the embedded Clinic / codegen parsers so the parsed cache is functional for cache-hit
validity checks. Decompilation variables live on kb.dec_variables."""
from .notes import DecompilationNote # pylint:disable=import-outside-toplevel
from .structured_codegen.c import CStructuredCodeGenerator # pylint:disable=import-outside-toplevel
cache = cls(cmsg.addr)
# cfg is not serialized; reattach from kwargs so cache-validity checks still work.
cache.cfg = cfg
if cmsg.HasField("clinic"):
cache.clinic = Clinic.parse(cmsg.clinic, project=project, kb=kb, function=function, cfg=cfg)
if cmsg.HasField("codegen"):
cache.codegen = CStructuredCodeGenerator.parse(cmsg.codegen, project=project, kb=kb, func=function)
cache.errors = list(cmsg.errors)
if cmsg.HasField("function_summary"):
cache.function_summary = cmsg.function_summary
# Collection fields default to empty (set in __init__); only assign when the message carries content.
if cmsg.HasField("arg_vvars"):
cache.arg_vvars = parse_arg_vvars(cmsg.arg_vvars)
if cmsg.HasField("ite_exprs"):
cache.ite_exprs = parse_ite_exprs(cmsg.ite_exprs)
cache.binop_operators = _parse_binop_operators(cmsg.binop_operators)
cache.stackvar_max_sizes = {_simvar_from_bytes(e.simvar): e.max_size for e in cmsg.stackvar_max_sizes}
# legacy blobs carry the proto3 defaults ""/0, meaning "unknown"; do not re-stamp them with current values
cache.version = cmsg.version
cache.timestamp = cmsg.timestamp
if cache.codegen is not None:
# mirror the stamps onto the codegen (a fresh decompile does the same in Decompiler._decompile)
cache.codegen.version = cache.version
cache.codegen.timestamp = cache.timestamp
if cmsg.HasField("parameters"):
cache.parameters = _parse_parameters(cmsg.parameters)
cache.notes = {k: DecompilationNote.from_json(v) for k, v in cmsg.notes_json.items()}
return cache

View file

@ -1,14 +1,12 @@
# decompilation options
from __future__ import annotations
from collections.abc import Callable
from collections import defaultdict
from typing import TypeVar
from collections.abc import Callable
from .structuring import structurer_class_from_name
from .structuring.phoenix import MultiStmtExprMode
T = TypeVar("T")
class DecompilationOption[T]:
"""
@ -47,6 +45,10 @@ class DecompilationOption[T]:
O = DecompilationOption
# Serialization contract for display options (cls="codegen"): to survive Codegen serialization, an option's param
# must have a matching optional scalar proto field, named identically, in the trailing display-option block of the
# Codegen message (protos/codegen.proto). Options without such a field are dropped on round-trip.
options = [
O(
"Aggressively remove dead memdefs",
@ -302,6 +304,17 @@ options = [
default_value=True,
clears_cache=False,
),
O(
"Indentation width",
"Number of space characters per indentation level in the pseudocode.",
int,
"codegen",
"indent_size",
category="Display",
default_value=4,
value_range=(1, 8),
clears_cache=False,
),
O(
"Multi-expression statements generation",
"Should the structuring algorithm generate multi-expression statements? If so, under what conditions?",
@ -325,6 +338,20 @@ options = [
default_value=False,
clears_cache=False,
),
O(
"Constrain and improve callee function prototypes based on call site information",
"When enabled, the decompiler will analyze call sites and constrain the types of callee function arguments "
"based on facts observed at the call sites. This may improve the accuracy of callee function prototypes and "
"consequently enhance the decompilation output of the current function. Note that enabling this option may "
"change the prototypes of callee functions in the knowledge base, which could affect subsequent decompilation "
"results.",
bool,
"clinic",
"constrain_callee_prototypes",
category="Types",
default_value=True,
clears_cache=True,
),
]
# NOTE: if you add a codegen option here, please add it to reapply_options

View file

@ -1,46 +1,60 @@
# pylint:disable=unused-import
# pylint:disable=unused-import,protected-access
from __future__ import annotations
import logging
from collections import defaultdict
from collections.abc import Iterable
from typing import Any, TYPE_CHECKING
from typing import TYPE_CHECKING, Any
import networkx
from cle import SymbolType
from angr import ailment
from angr.analyses.analysis import AnalysesHub, Analysis
from angr.analyses.cfg import CFGFast
from angr.analyses.s_propagator import sprop_cache_scope
from angr.analyses.typehoon.typehoon import Typehoon
from angr.analyses.typehoon.typevars import TypeVariableManager
from angr.errors import AngrAIError
from angr.knowledge_plugins.functions.function import Function
from angr.knowledge_base import KnowledgeBase
from angr.rust.optimization_passes import get_rust_optimization_passes
from angr.rust.typehoon.typehoon import RustTypehoon
from angr.sim_variable import SimMemoryVariable, SimRegisterVariable, SimStackVariable
from angr.utils import timethis
from angr.analyses import Analysis, AnalysesHub
from angr.sim_type import parse_type
from angr.errors import AngrAIError
from angr.analyses.typehoon.typehoon import Typehoon
from angr.rust.typehoon.typehoon import RustTypehoon
from angr.rust.optimization_passes import get_rust_optimization_passes
from .clinic import ClinicStage
from .structured_codegen.c import CStructuredCodeGenerator
from .structuring import RecursiveStructurer, PhoenixStructurer, DEFAULT_STRUCTURER
from .structuring.phoenix import MultiStmtExprMode
from .region_identifier import RegionIdentifier
from .optimization_passes.optimization_pass import OptimizationPassStage
from .ailgraph_walker import AILGraphWalker
from .clinic import ClinicStage
from .condition_processor import ConditionProcessor
from .decompilation_options import DecompilationOption, PARAM_TO_OPTION
from .decompilation_cache import DecompilationCache
from .utils import remove_edges_in_ailgraph
from .sequence_walker import SequenceWalker
from .structuring.structurer_nodes import SequenceNode
from .presets import DECOMPILATION_PRESETS, DecompilationPreset
from .decompilation_options import PARAM_TO_OPTION, DecompilationOption
from .edits import (
DecompilationEditError,
list_variable_names,
reflow_types,
rename_function,
rename_variable,
resolve_variable,
set_variable_type,
)
from .notes import DecompilationNote
from .optimization_passes.optimization_pass import OptimizationPassStage
from .presets import DECOMPILATION_PRESETS, DecompilationPreset
from .region_identifier import RegionIdentifier
from .sequence_walker import SequenceWalker
from .structured_codegen import DummyStructuredCodeGenerator
from .structured_codegen.c import CStructuredCodeGenerator
from .structured_codegen.rust import RustStructuredCodeGenerator
from .structurer_nodes import SequenceNode
from .structuring import DEFAULT_STRUCTURER, PhoenixStructurer, RecursiveStructurer
from .structuring.phoenix import MultiStmtExprMode
from .utils import remove_edges_in_ailgraph
from .variable_map import VariableMap
if TYPE_CHECKING:
from angr.analyses.typehoon.typevars import TypeConstraint, TypeVariable
from angr.knowledge_plugins.cfg.cfg_model import CFGModel
from .peephole_optimizations import PeepholeOptimizationExprBase, PeepholeOptimizationStmtBase
from angr.analyses.typehoon.typevars import TypeVariable, TypeConstraint
from .structured_codegen.base import BaseStructuredCodeGenerator
l = logging.getLogger(name=__name__)
@ -56,6 +70,15 @@ class Decompiler(Analysis):
Run this on a Function object for which a normalized CFG has been constructed.
The fully processed output can be found in result.codegen.text
AIL graphs exposed on the result (both on a fresh run and on a cache hit, including caches reloaded from
angrdb or the runtime-db spill):
- ``ail_graph`` (= ``clinic.cc_graph``): the simplified graph before region identification.
- ``clinic.graph``: the final graph after region identification and region simplification.
- ``unoptimized_ail_graph`` (= ``clinic.unoptimized_graph``): a snapshot before the first structure-altering
optimization pass; use it for an exact instruction-to-AIL mapping. Only built when
``save_unoptimized_graph=True`` is passed; otherwise this attribute is None on both fresh runs and cache hits.
"""
def __init__(
@ -66,7 +89,6 @@ class Decompiler(Analysis):
preset: str | DecompilationPreset | None = None,
optimization_passes=None,
sp_tracker_track_memory=True,
variable_kb=None,
peephole_optimizations: _PEEPHOLE_OPTIMIZATIONS_TYPE = None,
vars_must_struct: set[str] | None = None,
flavor="pseudocode",
@ -75,7 +97,7 @@ class Decompiler(Analysis):
ite_exprs=None,
binop_operators=None,
decompile=True,
regen_clinic=True,
regen_clinic=False,
inline_functions=None,
desired_variables=None,
update_memory_data: bool = True,
@ -92,6 +114,7 @@ class Decompiler(Analysis):
static_vvars: dict | None = None,
static_buffers: dict | None = None,
codegen_cls=CStructuredCodeGenerator,
save_unoptimized_graph: bool = False,
):
if not isinstance(func, Function):
func = self.kb.functions[func]
@ -132,7 +155,6 @@ class Decompiler(Analysis):
self._sp_tracker_track_memory = sp_tracker_track_memory
self._peephole_optimizations = peephole_optimizations
self._vars_must_struct = vars_must_struct
self._variable_kb = variable_kb
self._expr_comments = expr_comments
self._stmt_comments = stmt_comments
self._ite_exprs = ite_exprs
@ -145,24 +167,29 @@ class Decompiler(Analysis):
self._desired_variables = frozenset(desired_variables) if desired_variables else set()
self._static_vvars = static_vvars if static_vvars is not None else {}
self._static_buffers = static_buffers if static_buffers is not None else {}
self._save_unoptimized_graph = save_unoptimized_graph
# ``cfg`` is not in this dict: it is an input, not part of the decompilation result. Its identity is
# checked separately in :meth:`_can_use_decompilation_cache`.
# Collection-typed values are normalized to empty collections (never None) so the serialized cache does not
# need to distinguish None from empty. The exception is peephole_optimizations, where None means "use the
# default peephole set" and is distinct from an explicitly empty list.
self._cache_parameters = (
{
"cfg": self._cfg,
"variable_kb": self._variable_kb,
"options": {(o, v) for o, v in self._options if o.category != "Display" and v != o.default_value},
"optimization_passes": self._optimization_passes,
"sp_tracker_track_memory": self._sp_tracker_track_memory,
"peephole_optimizations": self._peephole_optimizations,
"vars_must_struct": self._vars_must_struct,
"vars_must_struct": self._vars_must_struct or set(),
"flavor": self._flavor,
"expr_comments": self._expr_comments,
"stmt_comments": self._stmt_comments,
"ite_exprs": self._ite_exprs,
"binop_operators": self._binop_operators,
"expr_comments": self._expr_comments or {},
"stmt_comments": self._stmt_comments or {},
"ite_exprs": self._ite_exprs or set(),
"binop_operators": self._binop_operators or {},
"inline_functions": self._inline_functions,
"desired_variables": self._desired_variables,
"static_vvars": self._static_vvars,
"static_buffers": self._static_buffers,
"save_unoptimized_graph": self._save_unoptimized_graph,
}
if use_cache
else None
@ -191,6 +218,18 @@ class Decompiler(Analysis):
self.use_cache = use_cache
self.update_cache = update_cache
self._variable_map = None
# structuring-specific parameters - will be reset in _decompile()
self._force_loop_single_exit = True
self._refine_loops_with_single_successor = False
self._expose_loop_head_backedges = False
self._recursive_structurer_params = {}
# cache of reusable AILBlockWalker instances that are shared by all SPropagator instances created during
# decompilation. Owned here so all walkers are released when this Decompiler instance is garbage-collected.
# SPropagator picks up this cache (see walker_cache_scope).
self._sprop_walker_cache: dict = {}
self._codegen_cls = CStructuredCodeGenerator
self._typehoon_cls = Typehoon
if self._flavor == "rust":
@ -199,7 +238,7 @@ class Decompiler(Analysis):
if decompile:
with self._resilience():
self._decompile()
self._decompile_with_cache()
if self.errors:
if self.update_cache:
if (self.func.addr, self._flavor) not in self.kb.decompilations:
@ -212,7 +251,7 @@ class Decompiler(Analysis):
self._optimization_passes = DECOMPILATION_PRESETS["basic"].get_optimization_passes(
self.project.arch, self.project.simos.name
)
self._decompile()
self._decompile_with_cache()
if self.update_cache:
for error in self.errors:
self.kb.decompilations[(self.func.addr, self._flavor)].errors.append(error.format())
@ -220,9 +259,14 @@ class Decompiler(Analysis):
def _can_use_decompilation_cache(self, cache: DecompilationCache) -> bool:
if self._cache_parameters is None or cache.parameters is None:
return False
# deserialized caches come back with cfg unset until the caller re-attaches it; unset is not a mismatch
if cache.cfg is not None and cache.cfg is not self._cfg:
return False
a, b = self._cache_parameters, cache.parameters
id_checks = {"cfg", "variable_kb"}
return all(a[k] is b[k] if k in id_checks else a[k] == b[k] for k in self._cache_parameters)
if not b:
# AngrDB-loaded caches carry no recorded parameters; there is nothing to validate against
return True
return all(k in b and a[k] == b[k] for k in a)
@staticmethod
def _parse_options(options: list[tuple[DecompilationOption | str, Any]]) -> list[tuple[DecompilationOption, Any]]:
@ -238,6 +282,34 @@ class Decompiler(Analysis):
converted_options.append((o, v))
return converted_options
def _decompile_with_cache(self):
with sprop_cache_scope(self._sprop_walker_cache):
self._decompile()
def _reuse_cached_decompilation(self, cache, clinic, codegen) -> None:
"""Full-reuse fast path: expose the cached clinic and codegen as this run's results without re-running the
pipeline. A live codegen's text is re-rendered to pick up in-place display edits; a freshly-deserialized
codegen (``_handlers is None``) keeps its stored text. The codegen inherits the cache's version and
timestamp."""
codegen.version = cache.version
codegen.timestamp = cache.timestamp
if codegen._handlers is not None:
codegen.regenerate_text()
self.cache = cache
self.clinic = clinic
self.codegen = codegen
self.seq_node = None
self.ail_graph = clinic.cc_graph
self.unoptimized_ail_graph = clinic.unoptimized_graph
self._variable_map = clinic.variable_map
self.vvar_id_start = clinic.vvar_id_start
self._copied_var_ids = clinic.copied_var_ids
if self.update_cache:
self.kb.decompilations[(self.func.addr, self._flavor)] = cache
self._finish_progress()
@timethis
def _decompile(self):
if self.func.is_simprocedure:
@ -262,10 +334,28 @@ class Decompiler(Analysis):
else:
old_codegen = None
old_clinic = None
ite_exprs = self._ite_exprs
binop_operators = self._binop_operators
# normalize to empty collections so the cache never stores None (passes treat None and empty the same)
ite_exprs = self._ite_exprs or set()
binop_operators = self._binop_operators or {}
l.debug("Decompilation cache miss")
# Full-reuse fast path: with use_cache and without regen_clinic (the default), a valid cache short-circuits
# the entire pipeline and hands back the cached clinic and codegen. Requires an AST-carrying codegen (not
# DummyStructuredCodeGenerator) and this function's variables in kb.dec_variables; anything else falls
# through to a fresh decompilation.
if (
self.use_cache
and not self._regen_clinic
and cache is not None
and old_clinic is not None
and old_codegen is not None
and not isinstance(old_codegen, DummyStructuredCodeGenerator)
and self.func.addr in self.kb.dec_variables
and self.func.prototype is not None
):
self._reuse_cached_decompilation(cache, old_clinic, old_codegen)
return
self.options_by_class = defaultdict(list)
if self._options:
@ -276,21 +366,13 @@ class Decompiler(Analysis):
self._set_global_variables()
self._update_progress(5.0, text="Converting to AIL")
variable_kb = self._variable_kb
# fall back to old codegen
if variable_kb is None and old_codegen is not None and isinstance(old_codegen, CStructuredCodeGenerator):
variable_kb = old_codegen._variable_kb
if variable_kb is None:
reset_variable_names = True
else:
reset_variable_names = self.func.addr not in variable_kb.variables.function_managers
reset_variable_names = self.func.addr not in self.kb.dec_variables.function_managers
# determine a few arguments according to the structuring algorithm
fold_callexprs_into_conditions = False
self._force_loop_single_exit = True
self._refine_loops_with_single_successor = False
self._complete_successors = False
self._expose_loop_head_backedges = False
self._recursive_structurer_params = self.options_to_params(self.options_by_class["recursive_structurer"])
if "structurer_cls" not in self._recursive_structurer_params:
self._recursive_structurer_params["structurer_cls"] = DEFAULT_STRUCTURER
@ -301,25 +383,36 @@ class Decompiler(Analysis):
if issubclass(self._recursive_structurer_params["structurer_cls"], PhoenixStructurer):
self._force_loop_single_exit = False
# self._refine_loops_with_single_successor = True
self._complete_successors = True
self._expose_loop_head_backedges = True
fold_callexprs_into_conditions = True
cache = DecompilationCache(self.func.addr)
cache.cfg = self._cfg
if self._cache_parameters is not None:
cache.parameters = self._cache_parameters
cache.ite_exprs = ite_exprs
cache.binop_operators = binop_operators
# The Decompiler owns the VariableMap. A fresh map is created before launching a new Clinic (re-linking
# populates it from scratch over freshly-allocated atom idx values). When a cached Clinic is reused without
# re-linking, its existing map is carried over below.
variable_map = VariableMap()
# convert function blocks to AIL blocks
def progress_callback(p, **kwargs):
return self._update_progress(p * (70 - 5) / 100.0 + 5, **kwargs)
if self._regen_clinic or old_clinic is None or self.func.prototype is None:
# a deserialized clinic whose function has no dec_variables cannot drive codegen; re-run Clinic instead
if (
self._regen_clinic
or old_clinic is None
or self.func.prototype is None
or self.func.addr not in self.kb.dec_variables
):
clinic = self.project.analyses.Clinic(
self.func,
kb=self.kb,
fail_fast=self._fail_fast,
variable_kb=variable_kb,
reset_variable_names=reset_variable_names,
optimization_passes=self._optimization_passes,
sp_tracker_track_memory=self._sp_tracker_track_memory,
@ -334,7 +427,7 @@ class Decompiler(Analysis):
optimization_scratch=self._optimization_scratch,
force_loop_single_exit=self._force_loop_single_exit,
refine_loops_with_single_successor=self._refine_loops_with_single_successor,
complete_successors=self._complete_successors,
expose_loop_head_backedges=self._expose_loop_head_backedges,
typehoon_cls=self._typehoon_cls,
ail_graph=self._clinic_graph,
arg_vvars=self._clinic_arg_vvars,
@ -344,18 +437,29 @@ class Decompiler(Analysis):
notes=self.notes,
static_vvars=self._static_vvars,
static_buffers=self._static_buffers,
save_unoptimized_graph=self._save_unoptimized_graph,
flavor=self._flavor,
variable_map=variable_map,
**self.options_to_params(self.options_by_class["clinic"]),
)
else:
clinic = old_clinic
# the deserialized clinic may carry peephole-optimization names that were unresolvable at parse time
# (their defining module was not imported then); retry resolving before its passes run again
clinic.resolve_peephole_optimizations()
# reuse the old, unaltered graph
clinic.graph = clinic.cc_graph
clinic.cc_graph = clinic.copy_graph()
# the SRDA model is tied to the previous run's graph; drop it so the simplification passes below
# regenerate it fresh for the reused graph
clinic.reaching_definitions = None
self.clinic = clinic
self.cache = cache
self._variable_kb = clinic.variable_kb
# Make the VariableMap available on the cache regardless of whether Clinic re-linked variables (a partial
# Clinic run, or the reuse-cached-Clinic path, may not repopulate cache.variable_map during linking).
cache.variable_map = clinic.variable_map
self._variable_map = clinic.variable_map
self._update_progress(70.0, text="Identifying regions")
self.vvar_id_start = clinic.vvar_id_start
self._copied_var_ids = clinic.copied_var_ids
@ -364,11 +468,13 @@ class Decompiler(Analysis):
# the function is empty
return
# expose a copy of the graph before any optimizations that may change the graph occur;
# use this graph if you need a reference of exact mapping of instructions to AIL statements
self.unoptimized_ail_graph = (
clinic.unoptimized_graph if clinic.unoptimized_graph is not None else clinic.copy_graph()
)
# expose a copy of the graph before any optimizations that may change the graph occur; use this graph if you
# need an exact instruction-to-AIL mapping. Only built when save_unoptimized_graph is set. clinic captured
# the snapshot iff a structure-altering pass ran; if none did, the current graph is itself unoptimized.
if self._save_unoptimized_graph:
self.unoptimized_ail_graph = (
clinic.unoptimized_graph if clinic.unoptimized_graph is not None else clinic.copy_graph()
)
cond_proc = ConditionProcessor(self.project.arch, clinic._ail_manager)
clinic.graph = self._run_graph_simplification_passes(
@ -426,8 +532,8 @@ class Decompiler(Analysis):
# simplify it
# Get variable manager for loop counter naming in RegionSimplifier
variable_manager = None
if clinic.variable_kb is not None and self.func.addr in clinic.variable_kb.variables:
variable_manager = clinic.variable_kb.variables[self.func.addr]
if self.func.addr in self.kb.dec_variables:
variable_manager = self.kb.dec_variables[self.func.addr]
region_simplifier_params = self.options_to_params(self.options_by_class["region_simplifier"])
# The Rust flavor forces if-else simplification off regardless of user options.
region_simplifier_params.pop("simplify_ifelse", None)
@ -450,7 +556,7 @@ class Decompiler(Analysis):
binop_operators=cache.binop_operators,
goto_manager=s.goto_manager,
graph=clinic.graph,
variable_kb=self._variable_kb,
kb=self.kb,
)
# rewrite the sequence node to remove phi expressions
@ -469,7 +575,7 @@ class Decompiler(Analysis):
ail_graph=clinic.graph,
flavor=self._flavor,
func_args=clinic.arg_list,
variable_kb=clinic.variable_kb,
variable_map=clinic.variable_map,
expr_comments=old_codegen.expr_comments if old_codegen is not None else None,
stmt_comments=old_codegen.stmt_comments if old_codegen is not None else None,
const_formats=old_codegen.const_formats if old_codegen is not None else None,
@ -484,6 +590,10 @@ class Decompiler(Analysis):
# save a copy of the AIL graph that is optimized but not modified by region identification
self.ail_graph = clinic.cc_graph
self.cache.codegen = codegen
if codegen is not None:
# copy the cache's version and timestamp onto the codegen
codegen.version = self.cache.version
codegen.timestamp = self.cache.timestamp
self.cache.clinic = self.clinic
# LLM refinement pass
@ -513,7 +623,7 @@ class Decompiler(Analysis):
update_graph=update_graph,
force_loop_single_exit=self._force_loop_single_exit,
refine_loops_with_single_successor=self._refine_loops_with_single_successor,
complete_successors=self._complete_successors,
expose_loop_head_backedges=self._expose_loop_head_backedges,
entry_node_addr=self.clinic.entry_node_addr,
**self.options_to_params(self.options_by_class["region_identifier"]),
)
@ -558,13 +668,13 @@ class Decompiler(Analysis):
blocks_by_addr=addr_to_blocks,
blocks_by_addr_and_idx=addr_and_idx_to_blocks,
graph=ail_graph,
variable_kb=self._variable_kb,
kb=self.kb,
reaching_definitions=reaching_definitions,
entry_node_addr=self.clinic.entry_node_addr,
scratch=self._optimization_scratch,
force_loop_single_exit=self._force_loop_single_exit,
refine_loops_with_single_successor=self._refine_loops_with_single_successor,
complete_successors=self._complete_successors,
expose_loop_head_backedges=self._expose_loop_head_backedges,
**kwargs,
)
@ -623,7 +733,7 @@ class Decompiler(Analysis):
blocks_by_addr=addr_to_blocks,
blocks_by_addr_and_idx=addr_and_idx_to_blocks,
graph=ail_graph,
variable_kb=self._variable_kb,
kb=self.kb,
arg_vvars=arg_vvars,
region_identifier=ri,
reaching_definitions=reaching_definitions,
@ -632,7 +742,7 @@ class Decompiler(Analysis):
scratch=self._optimization_scratch,
force_loop_single_exit=self._force_loop_single_exit,
refine_loops_with_single_successor=self._refine_loops_with_single_successor,
complete_successors=self._complete_successors,
expose_loop_head_backedges=self._expose_loop_head_backedges,
peephole_optimizations=self._peephole_optimizations,
avoid_vvar_ids=self._copied_var_ids,
**kwargs,
@ -707,18 +817,20 @@ class Decompiler(Analysis):
stack_offset_typevars = cache.stack_offset_typevars
stackvar_max_sizes = cache.stackvar_max_sizes
codegen = cache.codegen
max_tv_id = cache.max_tv_id
tv_manager = TypeVariableManager(self.func.addr, idx=max_tv_id + 1)
if codegen is None:
# nothing to reflow; but this should not happen
return None
var_kb = self._variable_kb if self._variable_kb is not None else KnowledgeBase(self.project)
var_kb = self.kb
if self.func.addr not in var_kb.variables:
if self.func.addr not in var_kb.dec_variables:
# for some reason variables for the current function don't really exist...
groundtruth = {}
else:
var_manager = var_kb.variables[self.func.addr]
var_manager = var_kb.dec_variables[self.func.addr]
# ground-truth types
groundtruth = {}
for variable in var_manager.variables_with_manual_types:
@ -755,16 +867,15 @@ class Decompiler(Analysis):
# Type inference
try:
tp = self.project.analyses.Typehoon(
tp = self.project.analyses[self._typehoon_cls].prep(kb=var_kb, fail_fast=self._fail_fast)(
type_constraints,
func_typevar,
kb=var_kb,
fail_fast=self._fail_fast,
var_mapping=var_to_typevar,
must_struct=must_struct,
ground_truth=groundtruth,
stack_offset_tvs=stack_offset_typevars,
stackvar_max_sizes=tv_max_sizes,
tv_manager=tv_manager,
)
tp.update_variable_types(
self.func.addr,
@ -782,7 +893,7 @@ class Decompiler(Analysis):
and isinstance(codegen, CStructuredCodeGenerator)
and codegen.cfunc is not None
):
var_manager = var_kb.variables[self.func.addr]
var_manager = var_kb.dec_variables[self.func.addr]
for i, arg in enumerate(codegen.cfunc.arg_list):
if i >= len(self.func.prototype.args):
break
@ -851,16 +962,24 @@ class Decompiler(Analysis):
:param ail_graph: The AIL graph to transform out of SSA form.
:return: The translated AIL graph.
"""
variable_kb = self._variable_kb
dephication = self.project.analyses.GraphDephication(
self.func, ail_graph, rewrite=True, variable_kb=variable_kb, kb=self.kb, fail_fast=self._fail_fast
self.func,
ail_graph,
rewrite=True,
variable_map=self._variable_map,
kb=self.kb,
fail_fast=self._fail_fast,
)
return dephication.output
def transform_seqnode_from_ssa(self, seq_node: SequenceNode) -> SequenceNode:
variable_kb = self._variable_kb
dephication = self.project.analyses.SeqNodeDephication(
self.func, seq_node, rewrite=True, variable_kb=variable_kb, kb=self.kb, fail_fast=self._fail_fast
self.func,
seq_node,
rewrite=True,
variable_map=self._variable_map,
kb=self.kb,
fail_fast=self._fail_fast,
)
return dephication.output
@ -917,29 +1036,10 @@ class Decompiler(Analysis):
if not code_text:
return False
# collect unified variables
varman = self._variable_kb.variables[self.func.addr]
unified_vars = varman.get_unified_variables(sort=None)
# also collect argument variables
arg_vars = []
if (
self.codegen
and isinstance(self.codegen, CStructuredCodeGenerator)
and self.codegen.cfunc
and self.codegen.cfunc.arg_list
):
for cvar in self.codegen.cfunc.arg_list:
v = cvar.unified_variable if cvar.unified_variable is not None else cvar.variable
if v not in unified_vars:
arg_vars.append(v)
all_vars = unified_vars + arg_vars
if not all_vars:
var_names = list_variable_names(self.codegen, self.kb, self.func.addr)
if not var_names:
return False
var_names = [v.name or str(v) for v in all_vars]
prompt = (
"You are a reverse engineering assistant. Given the following decompiled C code, suggest better, "
"more descriptive variable names. Only include variables that you want to rename. "
@ -954,27 +1054,27 @@ class Decompiler(Analysis):
if not result:
return False
# build name-to-variable lookup
name_to_var = {}
for v in all_vars:
key = v.name or str(v)
name_to_var[key] = v
changed = False
for rename in result.renames:
old_name = rename.old_name
new_name = rename.new_name
if not new_name:
old_name, new_name = rename.old_name, rename.new_name
if not new_name or old_name == new_name:
continue
var = name_to_var.get(old_name)
if var is None:
try:
edit = rename_variable(
self.project,
self.func,
old_name,
new_name,
kb=self.kb,
flavor=self._flavor,
rerender=False,
)
except DecompilationEditError as ex:
l.debug("LLM rename %s -> %s rejected: %s", old_name, new_name, ex)
continue
if old_name == new_name:
continue
var.name = new_name
var.renamed = True
changed = True
l.info("LLM renamed variable %s -> %s", old_name, new_name)
if edit.changed:
changed = True
l.info("LLM renamed variable %s -> %s", old_name, new_name)
return changed
@ -1018,13 +1118,15 @@ class Decompiler(Analysis):
if not new_name or new_name == current_name:
return False
l.info("LLM renamed function %s -> %s", current_name, new_name)
self.func.name = new_name
self.func.is_default_name = False
if self.codegen and isinstance(self.codegen, CStructuredCodeGenerator) and self.codegen.cfunc:
self.codegen.cfunc.name = new_name
try:
edit = rename_function(self.project, self.func, new_name, kb=self.kb, flavor=self._flavor, rerender=False)
except DecompilationEditError as ex:
l.debug("LLM function rename %s -> %s rejected: %s", current_name, new_name, ex)
return False
return True
if edit.changed:
l.info("LLM renamed function %s -> %s", current_name, new_name)
return edit.changed
def llm_suggest_variable_types(
self, llm_client=None, code_text: str | None = None, raise_exc: bool = False
@ -1047,19 +1149,20 @@ class Decompiler(Analysis):
if not code_text:
return False
varman = self._variable_kb.variables[self.func.addr]
unified_vars = varman.get_unified_variables(sort=None)
varman = self.kb.dec_variables[self.func.addr]
if not unified_vars:
return False
# build current type info
var_type_info = {}
for v in unified_vars:
name = v.name or str(v)
current_type = varman.get_variable_type(v)
for name in list_variable_names(self.codegen, self.kb, self.func.addr):
try:
rv = resolve_variable(self.kb, self.func.addr, name, codegen=self.codegen, flavor=self._flavor)
except DecompilationEditError:
continue
current_type = varman.get_variable_type(rv.variable)
var_type_info[name] = str(current_type) if current_type else "unknown"
if not var_type_info:
return False
prompt = (
"You are a reverse engineering assistant. Given the following decompiled C code and the current "
"variable types, suggest better C types for the variables. Only include variables whose types "
@ -1074,33 +1177,33 @@ class Decompiler(Analysis):
if not result:
return False
# build name-to-variable lookup
name_to_var = {}
for v in unified_vars:
key = v.name or str(v)
name_to_var[key] = v
changed = False
for type_change in result.type_changes:
var_name = type_change.variable_name
type_str = type_change.new_type
var_name, type_str = type_change.variable_name, type_change.new_type
if not type_str:
continue
var = name_to_var.get(var_name)
if var is None:
continue
try:
new_type = parse_type(type_str, arch=self.project.arch)
except Exception: # pylint:disable=broad-exception-caught
l.debug("LLM suggested unparseable type '%s' for %s", type_str, var_name)
# reflow once at the end rather than per variable: Typehoon is expensive
edit = set_variable_type(
self.project,
self.func,
var_name,
type_str,
kb=self.kb,
flavor=self._flavor,
reflow=False,
)
except DecompilationEditError as ex:
l.debug("LLM retype of %s to '%s' rejected: %s", var_name, type_str, ex)
continue
if edit.changed:
changed = True
l.info("LLM changed type of %s to %s", var_name, type_str)
varman.set_variable_type(var, new_type, mark_manual=True, all_unified=True)
changed = True
l.info("LLM changed type of %s to %s", var_name, type_str)
if changed and self.codegen:
self.codegen.reload_variable_types()
if changed:
new_codegen = reflow_types(self.project, self.func, kb=self.kb, flavor=self._flavor, rerender=False)
if new_codegen is not None:
self.codegen = new_codegen
return changed

View file

@ -1,6 +1,7 @@
from __future__ import annotations
from .graph_vvar_mapping import GraphDephicationVVarMapping
from .graph_dephication import GraphDephication
from .graph_vvar_mapping import GraphDephicationVVarMapping
from .seqnode_dephication import SeqNodeDephication
__all__ = ["GraphDephication", "GraphDephicationVVarMapping", "SeqNodeDephication"]

View file

@ -1,13 +1,13 @@
from __future__ import annotations
from typing import TYPE_CHECKING
from typing import Any
import logging
from collections import defaultdict
from typing import TYPE_CHECKING, Any
from angr.analyses import Analysis
from angr.analyses.analysis import Analysis
if TYPE_CHECKING:
from angr import KnowledgeBase
from angr.analyses.decompiler.variable_map import VariableMap
l = logging.getLogger(name=__name__)
@ -23,15 +23,15 @@ class DephicationBase(Analysis):
func,
vvar_to_vvar_mapping: dict[int, int] | None = None,
rewrite: bool = False,
variable_kb: KnowledgeBase | None = None,
variable_map: VariableMap | None = None,
):
if isinstance(func, str):
self._function = self.kb.functions[func]
else:
self._function = func
self.variable_kb = variable_kb
self.vvar_to_vvar_mapping = vvar_to_vvar_mapping if vvar_to_vvar_mapping is not None else None
self.variable_map = variable_map
self.rewrite = rewrite
self.output = None

View file

@ -1,19 +1,21 @@
from __future__ import annotations
from typing import TYPE_CHECKING
import logging
from collections import defaultdict
from typing import TYPE_CHECKING
import networkx
from angr.ailment.expression import Phi, VirtualVariable
from angr.ailment.statement import Assignment
from angr.analyses.analysis import register_analysis
from angr.knowledge_plugins.functions import Function
from angr.analyses import register_analysis
from .graph_rewriting import GraphRewritingAnalysis
from .dephication_base import DephicationBase
from .graph_rewriting import GraphRewritingAnalysis
if TYPE_CHECKING:
from angr import KnowledgeBase
from angr.analyses.decompiler.variable_map import VariableMap
l = logging.getLogger(name=__name__)
@ -31,7 +33,7 @@ class GraphDephication(DephicationBase): # pylint:disable=abstract-method
ail_graph,
vvar_to_vvar_mapping: dict[int, int] | None = None,
rewrite: bool = False,
variable_kb: KnowledgeBase | None = None,
variable_map: VariableMap | None = None,
):
"""
:param func: The subject of the analysis: a function, or a single basic block
@ -40,7 +42,12 @@ class GraphDephication(DephicationBase): # pylint:disable=abstract-method
self._graph = ail_graph
super().__init__(func, vvar_to_vvar_mapping=vvar_to_vvar_mapping, rewrite=rewrite, variable_kb=variable_kb)
super().__init__(
func,
vvar_to_vvar_mapping=vvar_to_vvar_mapping,
rewrite=rewrite,
variable_map=variable_map,
)
self._analyze()
@ -62,7 +69,12 @@ class GraphDephication(DephicationBase): # pylint:disable=abstract-method
def _rewrite_container(self) -> networkx.DiGraph:
# replace all vvars with phi variables in the graph
rewriter = GraphRewritingAnalysis(
self.project, self._function, self._graph, self.vvar_to_vvar_mapping, variable_kb=self.variable_kb
self.project,
self._function,
self._graph,
self.vvar_to_vvar_mapping,
kb=self.kb,
variable_map=self.variable_map,
)
return rewriter.out_graph

View file

@ -1,14 +1,15 @@
from __future__ import annotations
from typing import Any
import logging
from typing import Any
import networkx
from angr import ailment
from angr.utils.ail import is_phi_assignment
from angr.analyses import ForwardAnalysis
from angr.analyses.forward_analysis import ForwardAnalysis, FunctionGraphVisitor
from angr.analyses.forward_analysis.visitors.graph import NodeType
from angr.analyses.forward_analysis import FunctionGraphVisitor
from angr.utils.ail import is_phi_assignment
from .rewriting_engine import SimEngineDephiRewriting
l = logging.getLogger(__name__)
@ -19,11 +20,20 @@ class GraphRewritingAnalysis(ForwardAnalysis[None, NodeType, object, object, obj
This analysis traverses the AIL graph and rewrites virtual variables accordingly.
"""
def __init__(self, project, func, ail_graph, vvar_to_vvar: dict[int, int], variable_kb=None):
def __init__(
self,
project,
func,
ail_graph,
vvar_to_vvar: dict[int, int],
kb=None,
variable_map=None,
):
self.project = project
self._function = func
self._graph_visitor = FunctionGraphVisitor(self._function, ail_graph)
self.variable_kb = variable_kb
self._dvars_kb = kb
self.variable_map = variable_map
ForwardAnalysis.__init__(
self, order_jobs=False, allow_merging=False, allow_widening=False, graph_visitor=self._graph_visitor
@ -31,7 +41,11 @@ class GraphRewritingAnalysis(ForwardAnalysis[None, NodeType, object, object, obj
self._graph = ail_graph
self._vvar_to_vvar = vvar_to_vvar
self._engine_ail = SimEngineDephiRewriting(
self.project, self._vvar_to_vvar, func_addr=self._function.addr, variable_kb=self.variable_kb
self.project,
self._vvar_to_vvar,
func_addr=self._function.addr,
kb=self._dvars_kb,
variable_map=self.variable_map,
)
self._visited_blocks: set[Any] = set()

Some files were not shown because too many files have changed in this diff Show more