Flags are sorted into the name hashtable with their realnames as well.
Refcounting is used to prevent double-free and similar issues that would
be caused by this.
* Added build files [Capstone to Zydis]
* x86 Analysis [Capstone to Zydis]
* Changed x86 RzIL [Capstone to Zydis]
* Changed asm and arch files [Capstone to Zydis]
* Compilation, build error and test fixes [Capstone to Zydis]
* Test changes [Capstone to Zydis]
* Remaining changes [Capstone to Zydis]
* Added BE support [Capstone to Zydis]
---------
Co-authored-by: tushar3q34 <tushar3q34@gmail.com>
* Replace OpenBSD regex library with PCRE2.
PCRE2 has way better performance than the OpenBSD
library (something around 20 times faster).
The following flags are enabled for every pattern:
- PCRE2_UTF
- PCRE2_MATCH_INVALID_UTF
- PCRE2_NO_UTF_CHECK
All the others are optional.
Changes made:
- Adds PCRE2 as subproject.
- Changes the API away from POSIX to PCRE2.
- Edits many regex patterns because:
- ' ' is skipped in patterns, if the EXTENDED flag is set for matching. '\s' must be set now.
- '.' doesn't match newlines by default.
- Changes the API so matches and their groups are bundled into PVectors.
- Moves the regex component to rz_util.
* Fix cross build - add copy of PCRE2 dependecy
Meson currently doesn't support subprojects to be native and non-native at the same time.
See: https://github.com/mesonbuild/meson/issues/10947
Unfortunately, sdb depends on rz_util which in turn depends on PCRE2.
Excluding PCRE2 from the native build makes linking of rz_util not possible anymore.
Adding it, will make Meson complain that the dependencies cannot be mixed.
Hence, we compile a copy of PCRE2 for the native build if required.
Variables on the stack are not identified by bp/sp+<offset> anymore, but
by their address from the bottom of the stack frame (RzStackAddr),
independent of how they are accessed.
So now there are only two kinds of variables: stack and register.
This required some major refactoring and other changes:
* RzAnalysisVar.isarg was removed. Whether a variable is an argument is
now specified implicitly by its storage location.
* Varsub of struct fields had to be rewritten so fields can be queried
by arbitrary stack addresses using the recently introduced sp
tracking, as the old approach to fill a list with all fields would not
work anymore.
* analysis.vars.stackname was removed, new behavior is more similar to
this being true before.
* Variables will not be created at stack+0 now, because the return
address is there. Before, vars were only created sometimes in such
cases.
* Variables created from bp offsets in x86 are not deleted anymore if
the function's bp_frame is false (see removed
rz_analysis_function_delete_vars_by_kind(fcn,
RZ_ANALYSIS_VAR_KIND_BPV); calls). This may lead to some
false-positive detected variables. Whether this really is a practical
issue is yet to be seen. At least there are no meaningful tests that
are broken by this.
* Applying variables from dwarf needed some fixes for determining the
correct stack locations of variables in order to write meaningful
tests. The handling is still not entirely correct for all
possibilities of dwarf info, but at least the changed/added test cases
are right and serve as a reference for future changes.
* Projects version 11 is introduced.
* afvb commands have been removed, afvs now handles all stack vars.
This introduces the ability to split asm strings into tokens and assign a type to them.
This means:
* Each asm strings is assigned a list of tokens. Each token points to a sub-string.
* A token has a type attached which describes it (e.g. register token, operator token, mnemonic token etc.)
* A generic parsing method is introduced which splits an asm string into tokens. The pattern it parses is `<mnem> <op> <op> ...`.
* Instead of the generic method, plugins can create the token strings on their own.
* An API method was introduced to help plugins implement their own parsing methods via regex patterns.
* Custom token strings are stored in `RzAsmOp.asm_toks`
Additionally the coloring of asm strings is now exclusively done via the token strings.
* For this a function was added to colorize tokenized asm strings according to their token types.
* CI: install sigdb in static version of rizin
* RzCore: use bool2status whenever possible
* RzCore: add flirt.sigdb.load vars to select which signatures to load
* test: adjust testing to not consider system/home flirt db
* sign: use a RzSigDb structure instead of a raw list to store signatures
* sign: switch RzSigDb to use a hashtable and avoid duplicates
* RzBin/ELF: avoid the creation of useless flags at 0
Use UT64_MAX as an error value for addresses like symbols, imports,
sections, etc., and do not create flags for those because they are not
meaningful. This is useful for imports that do not have an address in
the virtual/file space and for non-alloc sections (e.g. .comment).
- Removed oldshell version of `t` (types) commands
- Removed `tk` command because types storage switched to the hashtable
- Added `tfc` command to show the calling convention of the function or set it
- Added `tsl` (RZ_OUTPUT_MODE_LONG) to show structure members' sizes and offsets
- Added `tul` (RZ_OUTPUT_MODE_LONG) to show union members' sizes
- Switched the types storage to the hashtable instead of SDB
- Added load and export from/to SDB for standard type libraries and serialization for Projects
- Changed all `char *type` occurences to the proper `RzType`
- Added `RzCallable` type for the function/class method types
- Changed all function types API to use the `RzCallable` API
- Switched the function types (`RzCallable`) storage to the hashtable instead of SDB
- Added a connection between `RzAnalysisFunction` and `RzCallable` - `rz_analysis_function_derive_type()` API function to derive new `RzCallable` type from the `RzAnalysisFunction` even if it doesn't exist in the RzTypeDB.
- Moved type links to the RzAnalysis where they belong
- Removed TCC and MPC-based parsers
- Added Tree-Sitter-based parser for C grammar (see `librz/type/parser/*`)
- Reworked type databases in `librz/analysis/d/` to be consistent with C standard, libc, platform-specific definitions
- Switched the type links from `char *` to `RzType`
- Switched the type links from SDB to the hashtable
- Added concept of the "type paths", e.g. `a.b[20].c` where `b` is a member of `a` (`a` is either structure or a union) and `c` is a member of `b`, and `b` is an array where we take 20-th element.
- Added concept of the typeclasses (a bit similar to Haskell) for numerical types: `None` (the most generic one), `Num` (includes `Integral` and `Floating`), `Integral` (includes `Signed Integral` and `Unsigned Integral`), `Floating`.
- Added concept of type equality (`rz_types_equal(a, b)` API function)
- Added Doxygen documentation to every API function in the `librz/type/*`.
- Updated the project format from **V2** to **V3**, added migration.
* RzType introduction
* Move format processing to RzTypes from RzUtil
* Rename RzType to RzTypeDB
* Join RzType and RzAnalysis noreturn functions
* Add noreturn column in `aflt` output
* Remove D (disassembly) pf tests
* Fix `tn` test
* Add function noreturn serialization
* Modify meaningless type test
* Fix wrongly written `Cf` test
* Fix `za t` (zignature type) command
* Fix signature type test
* Disable `asm.bytes` by default
* Fix tests by adding `e asm.bytes=true` (1/n)
* Fix tests by adding (2/n)
* Fix tests by adding `e asm.bytes=true` (3/n)
* Fix tests by adding `asm.bytes=true` (4/n)
* Fix test by adding `asm.bytes=true` (5/n)
* Fix mostly all broken tests
* Attemp to fix the rzpipe test
* Attempt to fix the tests in `db/cmd/cmd_pipe`
* Fix tests again
* Fix cmd_pipe test
* Add a test where `asm.bytes` is false
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
