* knowledge_plugins, code_location: recompute hashes after unpickling
Atom, Definition, and CodeLocation cache their hash in a `_hash` slot. The
default pickling persists that slot, but the hash folds in per-process-salted
hashes (e.g. of register-name strings), so a value pickled in one process is
stale when unpickled in another -- equal objects then hash differently, which
breaks sets/dicts (and assertCountEqual) rebuilt from a pickle.
Add `__getstate__`/`__setstate__` that drop `_hash` from the pickled state and
reset it to None on load, so it is recomputed lazily in-process. `__dict__`
and all slots are otherwise preserved, and the existing default-format
pickles still load.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Disable protected-member for hash checks
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Enable ruff isort rule
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix typos
This mostly fixes comments, but in a handful of places fixes bugs due to typos.
One example:
```
- insn_op_idx=None if cmsg.operand_idx == -1 else cmsg.opearnd_idx,
+ insn_op_idx=None if cmsg.operand_idx == -1 else cmsg.operand_idx,
```
* more typo fixes
* address lint issues
* more import fixes?
* address feedback from PR
---------
Co-authored-by: Brian Caswell <bcaswell@microsoft.com>
* Attempt to provide an interface for writing RDA function handlers easily
* Aggressively touch up call and const atoms
* Totally rewrite function handlers
* Imports
* so many renames and fixes
* weh
* better errors
* Attempt to make decompiler work
* better behavior
* Fix a bunch of shit. thanks mypy
* Add DepGraph.find_path
* Use callsite insaddr for codeloc of function outputs
* Add LiveDefinitions: get_stack_values
* Don't require handle_impl_<func>
* wip: add logic for matching RDA definitions against SimVariables
* _narrow_exprs: Do not update call.ret_expr if ret_expr is None.
* Propagator: Fix an incorrect access to None in the AIL engine.
* RDA: Fix an incorrect CodeLocation definition in function handler.
* Clarify docs for CodeLocation
* fix handle_impl_ and clarify warning message
* Support pseudo function calls.
* Separate function effects applied at callsites and the ones applied inside callees.
* AILSimplifier._narrow_exprs: Filter away all definitions in callees.
* DepGraph no longer takes a reference to Project.
* Fix function argument inference for tail jumps.
* FunctionHandler: Use args. Support args_values.
* Tidy interfaces
* AILSimplifier._unify_local_variables: Ignore variables defined in callees.
* Fix CCA by enabling the function handler.
* Properly handle CallExprs.
* Fix call expression handling. Fix the test case.
* f
* DefinitionMatchPredicate: Fix default initializers
* Overhaul codelocs
* Fix context handling
* Fix values iter
* Sort some shit
* Add logic for handling incomplete prototypes in FunctionHandler
* FunctionHandler: apply effects from dependencies first.
* Properly handle Call.ret_expr in AIL. Fix a in VRA.
* Propagator: Be less strict about contatenation when the high bytes are concrete.
* oops
* Move uses to first pass of function handler effects
* Aggressive type hinting
* FunctionHandler: Try harder to resolve function address
* Type annotation
* Typing
* Adjust codegen to smooth over casts
* Tweak the function argument uses for the nth time
* ail simplifier: check that expr corresponding to use is not None
* Typing
* Objectively a correct bugfix
* Correctly generate args_atoms based on args_values
* in case of fire break glass
* Function handler: Add args uses even if there is no return value
* Add data.ret_values even if there is no ret atom (e.g. ccalls)
* Add the missing with_arch() call.
* function handler: hook prototypes don't have an arch
* Revert "Add the missing with_arch() call."
This reverts commit 319eb8f826f634c89106221e33359a2eb1009a48.
* Implement better is_testing detection.
* Uncomment an important fire extinguisher.
* RDA: AIL engine overwrites the whole register when the return value only occupies the partial register.
* FunctionHandler: Properly handle clobbered partial registers.
* Fix a type hint in atoms.Register.
* Do not add duplicated uses for values that are in args_values.
* copy guessed_prototype from hook to function data
* AILSimplifier: Ignore unused conflicting defs when unifying variables.
* c_args_as_atoms: Handle variadic functions.
* RDA: AILEngine: Do not redefine locals if call args are available.
* VRA: EngineAIL: Follow RDA's logic when handling call.ret_expr.
* Respect block_idx a few places; add caller-handled ret_defns to callsite info
* CallSiteMaker: Mark argument definitions in call stmts so RDA knows what to erase.
* RDA: AILEngine.handle_register: Use the optimal define location if possible.
* oops
* AILSimplifier._narrow_exprs: Fix the callee function check.
* Remove unused imports.
* docs
* Adapt StaticObjectFinder to the new function handler model.
* Make vex rda tests pass
* function handler: Apply bootstrap return value definition at callsite
* lint
* lint 2
---------
Co-authored-by: Audrey Dutcher <audrey@rhelmot.io>
Co-authored-by: Matt Borgerson <contact@mborgerson.com>
* SimMemoryObject: Use slots.
* Add LabeledMemory.
* CodeLocation: stmt_idx can be None.
* Propagator: Switch to LabeledMemory and claripy ASTs - Strike One.
* Implement SimLabeledMemoryObject.
* Propagator: Switch to LabeledMemory and claripy ASTs - Strike Two.
* Bug fixes. Migrate RDA live_definitions to LabeledMemory.
* Call the test case.
* Add MVListPage.
* some effort
* Update the test case.
* More progress
* More work done.
* Implement page merging.
* Implement add and sub for RDEngineVEX.
* More test cases passed
* More progress. Update test cases.
* RDA: `SimEngineRDVEX._handle_function_core` to use `MultiValues`
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* Fix _process_block_end().
* Update _handle_CMPNE
* Update _handle_CMPLT
* Update _handle_CMPORD
* Update _handle_Sar
* Add none safety to some _handle_* functions
* Fix _handle_Not update
* Update _handle_Not1
* Update _handle_CCall
* Add exception checking during shifting
* Convert SimEngineRDAIL.
* bug fixes
* Fix the endianness bug.
* Several bug fixes.
* Do not resimplify a block if it is not changed.
* RDA: Do not reload register values.
* Fix DefinitionAnnotation.
* Fix MultiValues.__len__().
* Fix a test case.
* Propagator: Properly deal with stack addresses in Load.
* Migrate most of variable recovery analysis.
* Force AIL Store to use its own size.
* Fix incorrect propagation when bytes are extracted from an MO. Get rid of .processor_state in VRA.
* Get rid of ProcessorState.
* Minor bug fixes in _reference().
* bug fixes
* Propagator: Return 1-bit tops for CMPs.
* Cache TOPs.
* Fix variable indent for global variables.
* SimMemoryObject: Optimize __eq__().
* VRF: Fix four handlers in the AIL engine.
* Remove a breakpoint.
* MVListPage: Do not calculate changed bytes twice during merging.
* MVListPage: Reduce unnecessary scans of None in content.
* ListPage/MVListPage: Support custom SimMemoryObject comparators.
* MVListPage: Reduce calls to _contains().
* Propagator: Remove TOPs in replacements after analysis.
* Reduce max iterations of RDA and Propagator to 2.
* Bug fix in propagator.
* Fix an incorrect type annotation.
* Unify _post_analysis in Propagator.
* LiveDefinitions: Fix _mo_cmp.
* Revert "Reduce max iterations of RDA and Propagator to 2."
This reverts commit 536ea4d59f80095dc37ec9055ed25aab19ed7d83.
* _replace_and_build(): Returns if the replacement really happened.
* Propagator: Fix stack variable load/store.
* Compute less propagation and RDAs in AILSimplifier.
* RDA: Use cached and less-accurate dummy definitions for kill_definitions().
* Fix LiveDefinitions._mo_cmp.
* Propagator: Fix a minor issue in the Add handler in AIL engine.
* Propagator: Attempt to propagate everything coming out of tmps.
* Migrate variable recovery.
* RDA: Get rid of references to DataSet. Fix support for floating point values.
* LightEngineVEX: _handle_Const supports floating points.
* CallSiteMaker: Handle SimMemoryMissingError.
* extract_offset_to_sp: Handle __add__ with only one argument.
* Propagator: Do not store if the data is None.
* Implement get_all_definitions(). Fix bugs in SimEngineRDVEX.
* Fix more broken test cases.
* VariableRecovery: Fix a StopIteration bug.
* Fix more bugs.
* Lint the code.
* Fix test cases. Drop DataSet.
Co-authored-by: Pamplemousse <xav.maso@gmail.com>
Co-authored-by: Clasm <wfgibbs@asu.edu>
* RDA: Don't duplicate `get_sp` implementation
... accross `ReachingDefinitionsState` and `LiveDefinitions`
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* RDA: `LiveDefinitions.get_sp()` not to fail
...when there are several SP definitions.
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* RDA: Utilitary to get the size of several `DataSet`s
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* RDA: More operations for `UnknownSize`
As the size of `DataSet`s represents a number of bits, the following
operation often happens in the code: `dataset._bits // 8` .
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* RDA: Minor missing documention addition
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* RDA: Favor `unittest` assertions to `nose`
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* RDA: Basic heap memory modelisation
* have a `HeapAddress` dedicated type
* uber-basic simulation of an allocator behavior (addresses
generation)
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* RDA: Add heap modelisation to the state
* content as part of the `LiveDefinitions`
* memory management using `HeapAllocator` and `HeapAddress`es
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* RDA: Allow sum between `int` and `HeapAddress`
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
* RDA: Make `HeapAddress` hashable
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
