init_static_types_internal() walks the CTF type section that comes
directly from an object file's .ctf section. During the first counting
pass it treats a CTF_K_FORWARD record's ctt_type as the CTF_K_* kind of
the forwarded tag and bumps the corresponding population count:
if (kind == CTF_K_FORWARD)
pop[tp->ctt_type]++;
pop[] is a fixed-size stack array with CTF_K_MAX + 1 (64) entries, but
ctt_type is a 32-bit value read straight from the file and is never
validated. A crafted dict whose forward record carries a ctt_type
greater than CTF_K_MAX therefore causes an out-of-bounds write to the
stack at an attacker-controlled index.
The type walk is reachable from ctf_bufopen()/ctf_open(), i.e. whenever
libctf opens a dict: ld while linking CTF, and objdump/nm --ctf. The
second pass already tolerates an out-of-range ctt_type (ctf_name_table()
has a default case), so only this first-pass index was unguarded.
Reject a ctt_type outside the valid kind range as ECTF_CORRUPT, matching
the existing corruption handling in the same loop.
Reproduced with a 65-byte in-memory dict (one CTF_K_FORWARD record whose
ctt_type is 64) passed to ctf_bufopen(). Before the fix, AddressSanitizer
reports a stack-buffer-overflow at ctf-open.c:759 overflowing pop[64];
after the fix ctf_bufopen() returns ECTF_CORRUPT. Valid forwards
(ctt_type of CTF_K_STRUCT/UNION/ENUM) and the boundary value CTF_K_MAX
still open successfully.
Signed-off-by: Naveed Khan <naveed@digiscrypt.com>
ctf-link.c has a labs that clang finds suspect when compiling for a
32-bit host, warning about the signed/unsigned comparison. (labs
return is 32-bit signed, 0xffffffffe is 32-bit unsigned on a 32-bit
host.) I think the labs is bogus, apparently there because
ctf_link_deduplicating_count_inputs returns a ssize_t rather than a
size_t to cover error status returns. However, the error status isn't
a negative count but only -1. So remove the labs.
* ctf-link.c (ctf_link_deduplicating_per_cu): Avoid 32-bit
host warning; don't use labs, cast "inputs" instead.
Style/formatting fix. Report "too many inputs" using %lu.
This patch removes the constraint that a CTF_F_ARRNELEMS flag has to be
present in order to reverse the elements of an array when dumping. The
flag was never added to GCC, and having this requirement causes more
problems than it solves. A quick recap of the issue:
Given an array int foo[1][2][3], the expected graph is:
int foo[1][2][3] -> int foo[2][3] -> int foo[3] -> int foo
Prior to GCC PR114186, the emitted graph would be:
int foo[1][2][3] -> int foo[1][2] -> int foo[1] -> int foo
Following GCC PR114186, before the libctf fix, the output was:
int foo[3][2][1] -> int foo[3][2] -> int foo[3] -> int foo
So the underlying type graph was correct, but the ordering of elements
was incorrect.
With this fix, we emit correct ordering of the type graph, with no
requirements for the compiler to signal that it has GCC PR114186.
include/
* ctf.h (CTF_F_ARRNELEMS): Delete.
(CTF_F_MAX): Adjust.
libctf/
* ctf-decl.c (ctf_decl_push): Act as if CTF_F_ARRNELEMS is
always set.
* ctf-dump.c (ctf_dump_header): No longer dump its value.
* testsuite/libctf-lookup/multidim-array.c: No longer detect
compilers not emitting this flag (none do).
The recent libctf fix for ctf_arc_bufpreamble missed a case:
what if the input is exactly sizeof (ctf_archive_t) in size (which can
happen if the archive has no members at all, so returning the preamble
from one of the members is in any case impossible?). In this case
it'll return an off-the-end pointer, and its caller will overrun.
(This can also happen with fuzzed input which has a valid magic
number.)
Allow it to fail in this case, returning NULL, and adjust its sole
caller. The caller's conclusions in this case will be wrong (it will
conclude that the archive is connected to .symtab), but the incorrect
conclusions are harmless because the lack of archive members will
immediately cause a failure in ctf_arc_bufopen(), and an error return.
Thanks to Alan Modra for the original fix this soups up.
libctf/
PR libctf/33548
* ctf-archive.c (ctf_arc_bufpreamble): Fail if the archive is
too short (or empty, with no dicts to contain preambles),
returning NULL.
* ctf-open-bfd.c (ctf_bfdopen_ctfsect): Handle a NULL return.
"Section is indexed" and "No symbol table" are generated at the
beginning of ctf_dump_objts but not output, resulting in an asan
complaint about leaked memory. Clearly the messages should either not
be generated, or they should be output. I chose to do the latter in
this patch, which requires some testsuite adjustment.
libctf/
* ctf-dump.c (ctf_dump_objts): Output "Section is indexed" and
"No symbol table" strings, enclosed in parentheses and
indented.
ld/
* testsuite/ld-ctf/data-func-conflicted-vars.d: Update.
* testsuite/ld-ctf/data-func-conflicted.d: Update.
Avoid warnings about invalid escapes in etc/update-copyright.py by
using raw strings, add BinutilsFilter to skip psql.rc and add
"Kalray SA." as another copyright holder.
Existing code checks that the first uint64_t ctfa_magic field is
available before reading but neglects to check that the last uint64_t
ctfa_ctfs is available before reading it in ctf_arc_bufpreamble.
ctf_arc_bufopen sets up a pointer to the struct ctf_archive in
ctf_new_archive_internal. Extend the check to cover the entire struct.
PR 33548
PR 33549
* ctf-archive.c (ctf_arc_bufpreamble, ctf_arc_bufopen): Check
that buffer contains at least an entire struct ctf_archive
when accessing such a struct.
In CLANG_PLUGIN_FILE it is possible for plugin_file to be non-NULL
when LLVMgold.so does not exist.
configure output is messy, with results not printed against their
"checking.." line, eg.
checking for clang... (cached) yes
checking for clang plugin file... checking for x86_64-pc-linux-gnu-ar... (cached) ar --plugin /usr/lib/llvm-20/lib/clang/20/../../LLVMgold.so
/usr/lib/llvm-20/lib/clang/20/../../LLVMgold.so
This patch fixes those problems, and a similar interposition of other
configure output between AC_MSG_CHECKING and AC_MSG_RESULT in
gcc-plugin.m4. It also tidies some of the message text, and makes
similar code in gcc-plugin.m4 and clang-plugin.m4 a little more
consistent.
config/
* clang-plugin.m4 (CLANG_PLUGIN_FILE): Don't place checks for
tools (llvm-config, ar) inside AC_MSG_CHECKING..AC_MSG_RESULT
for clang plugin file. Clear plugin_file before loop exit.
(CLANG_PLUGIN_FILE_FOR_TARGET): Similarly.
* gcc-plugin.m4 (GCC_PLUGIN_OPTION): Similarly.
(GCC_PLUGIN_OPTION_FOR_TARGET): Correct AC_MSG_CHECKING. Tidy
return code.
binutils/
* testsuite/lib/binutils-common.exp <llvm_plug_opt>: Set for
non-native.
* configure: Regenerate.
/
* configure: Regenerate.
bfd/
* configure: Regenerate.
gas/
* configure: Regenerate.
gdb/
* configure: Regenerate.
gprof/
* configure: Regenerate.
gprofng/
* configure: Regenerate.
* libcollector/configure: Regenerate.
ld/
* configure: Regenerate.
libbacktrace/
* configure: Regenerate.
libctf/
* configure: Regenerate.
libiberty/
* configure: Regenerate.
libsframe/
* configure: Regenerate.
opcodes/
* configure: Regenerate.
sim/
* configure: Regenerate.
zlib/
* configure: Regenerate.
The warning is spurious (the variable is unused in this situation),
but clang cannot know that.
libctf/ChangeLog:
* testsuite/libctf-writable/symtypetab-nonlinker-writeout.c:
Initialize dummy variable.
libctf-regression/open-error-free.c #defines _GNU_SOURCE at its top.
This is already done by config.h if necessary: drop it.
libctf/ChangeLog:
* testsuite/libctf-regression/open-error-free.c: Drop
unnecessary system-dependent #define.
This patch inverts the ordering of nelems on multidimensional arrays on
versions of gcc without the CTF_F_ARRNELEMS flag. This allows those
systems which run older gcc to pass the test without modification to the
multidim-array.lk file.
libctf/ChangeLog:
* testsuite/libctf-lookup/multidim-array.c: Test fixes.
Running the libctf testsuite currently leaves big-struct-ctf.o in
libctf/testsuite/libctf-lookup/.
It ought to be possible to make the binutils source dir read-only,
and putting compiler output in the source dir leads to interesting
effects when testing multiple binutils targets in parallel.
* lib/ctf-lib.exp (run_lookup_test): For "link: objects" compile
objects in tmpdir.
Before GCC PR114186, all looked good in the land of
multidimensional arrays: you wrote
int a[5][10];
and ctf_type_aname() et al would print it as
int [5][10]
Unfortunately this was two bugs in one. GCC was emitting the array as if
it were int a[10][5], i.e. as this:
a -> [10] -> [5] -> int
rather than
a -> [5] -> [10] -> int
as it should be. libctf was hiding this by printing them in the wrong
order, concealing the bug from anyone using objdump --ctf or anything
but actual type graph traversal. Once this was fixed for GCC, the bug
was visible in libctf: multidimensional arrays were printed backwards!
(But this is just a print-time bug: the underlying bug, that something
traversing the type graph would see the array in backwards order, was
fixed by the fix to GCC.)
Fix this libctf bug, printing the arrays the right way round. In a
possibly futile attempt to retain some vestige of backwards
compatibility, introduce a new bug-compat flag CTF_F_ARRNELEMS, which,
if on, indicates that PR114186 is fixed and GCC is emitting array
elements the right way round. (Unfortunately, the fix went in without
this flag, so some GCCs will still emit CTF that will cause libctf to
print them wrong, even with this fix -- but it's no wronger than it was
before, and new GCC and new binutils, as well as GCC older than any fix
for PR114186 and new binutils, will print things properly. Someone
traversing the type graph will see things right after the GCC fix, wrong
before it, and there isn't really any reliable way to tell which you
have, though if CTF_F_ARRNELEMS is set, you definitely have a fixed GCC.
The test checks for this, but it's not something we expect actual users
to ever do -- CTF dict flags are an internal implementation detail with
no user-visible API for a reason.)
[nca: log message, test compat with older compilers]
include/
* ctf.h (CTF_F_ARRNELEMS): New bug-compat flag.
(CTF_F_MAX): Adjust.
libctf/
PR libctf/32161
* ctf-decl.c (ctf_decl_push): Prepend if this is an array and
the bug-compat flag is set.
* ctf-dump.c (ctf_dump_header): Dump the new bug-compat flag.
* testsuite/libctf-lookup/multidim-array*: New test.
After GCC PR 121411 is fixed, large structures (with offsets > 512MiB)
are written correctly... but libctf cannot query them properly unless
they are even bigger (> 4GiB), because it checks to see if the ctt_size
is CTF_LSIZE_SENT to decide whether to use a ctf_lmember_t or a
ctf_member_t to encode the structure members. But the structure member
offsets are in *bits*, not bytes: the right value to check is
CTF_LSTRUCT_THRESH, which is 1/8th the size.
(Thanks to Martin Pirker <martin.pirker1@chello.at> for the diagnosis
and fix.)
Testing this is a bit fun, because we don't want to emit an error if the
compiler is broken: but we cannot tell whether the compiler is broken
using the existing lookup harness, because its input is passed through
the linker (and thus the broken ld). So add another sort of link mode,
"objects", which keeps the constituent object files around and passes
both the final linker output and the object files that make it up to the
lookup program. Our testcase can then check the linker input to see if
the compiler is buggy, and only if it isn't check the linker output and
fail if things aren't right.
libctf/
PR libctf/33339
* ctf-types.c (ctf_struct_member): Check CTF_LSTRUCT_THRESH, not
CTF_LSIZE_SENT.
* testsuite/lib/ctf-lib.exp (run_lookup_test): New 'objects'
link option.
* testsuite/libctf-lookup/big-struct-corruption.*: New test.
* testsuite/libctf-lookup/big-struct-ctf.c: New test input.
The ctf_link_add_ctf API function has a 'lazy opening' feature whereby,
if you pass in the file but not a CTF archive, the archive is opened
as late as possible during links. This is valuable mostly in
cu-mapped links (a feature not accessible via GNU ld), where it
ensures that, rather than eventually needing memory for the original
link inputs, the smushed-together cu-mapped intermediate outputs,
*and* the final output, we only need enough memory for the smushed-
together outputs, the final output, and one input, since the inputs
can be closed immediately after they are smushed together.
(In GNU ld, the feature is useless because it loads all sections into
memory anyway.)
The lazy-opening feature uses libctf's ctf_open function, which uses
BFD: so it is not available in libctf-nobfd -- except that I thought I
had a cunning trick, and used a weak symbol so that if you linked
libctf-nobfd into your program and then also linked in bfd, the feature
stayed enabled.
This is silly -- if your program is licensed such that you can link in
BFD, you can just link in libctf.so and not bother with libctf-nobfd.so
in the first place. Worse, the weak symbol usage broke MacOS builds,
since MacOS's system compiler uses a different means of introducing weak
symbols. We could test for and use it, but this is the only place in
libctf to use weak symbols at all, and the feature of lazy-opening with
libctf-nobfd is so marginal we might as well drop it: it's almost
certain there are zero users, certainly fewer users than users of MacOS
with the system compiler.
While we're at it, simplify things by deleting the never-implemented
feature (not exposed in the API) to allow linking together raw buffers
of CTF data. If we need it we can bring it back, but all it's doing
right now is complicating the code to no end at all.
libctf/
PR libctf/33194
* ctf-link.c (ctf_open): Delete weak pragma.
(ctf_link_add): Fuse with...
(ctf_link_add_ctf): ... this function. Drop BUF, N args
and corresponding unimplemented feature warnings. Only check
NOBFD to see whether lazy loading is available, not PIC as
well.
(ctf_link_lazy_open): Likewise.
Before now, we were relying on gcc -B to repoint GCC at our freshly-built
linker, and assuming we could use this to run tests that do linker dedup.
This is, alas, not reliable (building --with-ld=/an/absolute/path bypasses
-B searches for the linker), and we would like the libctf tests in
particular to not fail if the compiler uses a linker that cannot deduplicate
(simply skipping such tests).
Add a new testsuite function that tries to link one of the lookup tests'
testcases (which is known to produce conflicting types, i.e. a dict with
per-translation-unit children) and then does an objdump and greps it for
signs of the per-translation-unit children. If there are none, the linker
probably just concatenated the sections: i.e., it is not a linker that
can deduplicate CTF.
libctf/
PR libctf/33162
* testsuite/config/default.exp (objdump): Introduce.
(OBJDUMPFLAGS): Likewise.
* testsuite/lib/ctf-lib.exp (check_ctf_linker_dedup): New.
* testsuite/libctf-lookup/lookup.exp: Use it.
* testsuite/libctf-regression/libctf-repeat-cu.exp: Likewise.
* testsuite/libctf-regression/regression.exp: Likewise.
If we're skipping _BEGIN_ and _END_ we should certainly skip all the other
ABI-required always-emitted symbols given in ld/emultempl/solaris2*em as
well. (This fixes a couple of diagnostics tests, but that's just because
the tests are quite sensitive to CTF section sizes, and introducing any
symtypetab entries for those tests perturbs those.)
Some of these are usually STT_NOTYPE, but not always: if programs already
emitted the symbol they might end up with any type, in particular
STT_OBJECT, and appear in the symtypetabs.
[nca: added commit log, added more symbols]
libctf/
PR libctf/33162
* ctf-serialize.c (ctf_symtab_skippable): Skip
more always-emitted Solaris symbols.
We generate de-commented version scripts for Solaris ld (whose
-z gnu-version-script linker option does not support comments),
but after we switched to generating libctf-nobfd.ver itself, we
failed to adjust this code to pick it up from the new location.
libctf/
PR libctf/33161
* configure.ac: Fix libctf-nobfd.ver path on Solaris.
* configure: Regenerated.
Writable mmaps and read are inconsistent with each other on Solaris 11.4,
which breaks archive writing. Simply assert that Solaris 2.11 doesn't have
mmap, falling back to read(): the archive format is being bumped shortly, to
one with completely different writeout code, so this workaround need not
stay for long.
[nca: added commit log]
libctf/
PR libctf/29292
* configure.ac: Turn off mmap on Solaris 2.11.
* configure: Regenerated.
This patch adds two additional distinct types (__float128 and the SIMD
vector type generated from the vector_size attribute) to the umbrella of
two existing types (long double and array, respectively). These types
were previously invalid, producing CTF_K_UNKNOWN in the case of
__float128 or a float in the case of the SIMD vector. This patch will
cleanly allow these types to be represented more accurately without
breaking back-compat.
Signed-off-by: Bruce McCulloch <bruce.mcculloch@oracle.com>
Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
The root-visible flag we're dealing with here is directly out of the dict,
not a flag passed in to the API, so it does not have the values CTF_ADD_ROOT
or CTF_ADD_NONROOT: instead it's simply zero for non-root-visible, nonzero
otherwise. Fix the test.
libctf/
* ctf-create.c (ctf_add_enumerator): Fix root-visibility test.
If you add a non-root type to a dict, you should always get a new, unique
type ID back, even if a root-visible type with the same name already exists.
Unfortunately, if the root-visible type is a forward, and you're adding a
non-root-visible struct, union, or enum, the machinery to detect forwards
and promote them to the concrete type fires in this case and returns the
root-visible type! If this is an enum being inserted hidden because its
enumerands conflict with some other enum, this will lead to failure later
on: in any case, it's seriously counterintuitive to add a non-root- visible
type and get a root-visible one instead.
Fix this by checking the root-visible flag properly and only checking for
forwards if this type is root-visible. (This may lead to a certain degree
of proliferation of non-root-visible forwards: we can add a cleanup pass for
those later if needed.)
libctf/
* ctf-create.c (ctf_add_struct_sized): Check the root-visible flag when
doing forward promotion.
(ctf_add_union_sized): Likewise.
(ctf_add_enum): Likewise.
Reviewed-by: Bruce McCulloch <bruce.mcculloch@oracle.com>
We don't use any GNU-specific printf args, but this prevents warnings about
%z, observed on MinGW even though every libc anyone is likely to use there
supports %z perfectly well, and we're not stopping using it just because
MinGW complains. Doing this means we stand more chance of seeing *actual*
problems on such platforms without them being drowned in noise.
We turn this off on clang, which doesn't support __gnu_printf__.
Suggested by Eli Zaretskii.
libctf/
PR libctf/31863
* ctf-impl.h (_libctf_printflike_): Use __gnu_printf__.
In normal deduplicating links, we insert every type (identified by its
unique hash) precisely once. But conflicting types appear in multiple
dicts, so for those, we loop, inserting them into every target dict
in turn (each corresponding to an input dict that type appears in).
But in cu-mapped links, some of those dicts may have been merged into
one: now that we are hiding duplicate conflicting types more
aggressively in such links, we are getting duplicate identical hidden
types turning up in large numbers.
Fix this by eliminating them in cu-mapping phase 1 (the phase in which this
merging takes place), by checking to see if a type with this hash has
already been inserted in this dict and skipping it if so. This is
redundant and a waste of time in other cu-mapping phases and in normal
links, but in cu-mapped links it saves a few tens to hundreds of kilobytes
in kernel-sized links.
libctf/
PR libctf/33047
* ctf-dedup.c (ctf_dedup_emit_type): Check for already-emitted
types in cu-mapping phase 1.
The previous commits dropped preservation of the non-root flag in ctf_link
and arranged to use it somewhat differently to track conflicting types in
cu-mapped CUs when doing cu-mapped links. This was necessary to prevent
entirely spuriously hidden types from appearing on the output of such links.
Bring it (and the test for it) back. The problem with the previous design
was that it implicitly assumed that the non-root flag it saw on the input
was always meant to be preserved (when in the final phase of cu-mapped links
it merely means that conflicting types were found in intermediate links),
and also that it could figure out what the non-root flag on the input was by
sucking in the non-root flag of the input type corresponding to an output in
the output mapping (which maps type hashes to a corresponding type on some
input).
This method of getting properties of the input type *does* work *if* that
property was one of those hashed by the ctf_dedup_hash_type process. In
that case, every type with a given hash will have the same value for all
hashed-in properties, so it doesn't matter which one is consulted (the
output mapping points at an arbitrary one of those input types). But the
non-root flag is explicitly *not* hashed in: as a comment in
ctf_dedup_rhash_type notes, being non-root is not a property of a type, and
two types (one non-root, one not) can perfectly well be the same type even
though one is visible and one isn't. So just copying the non-root flag from
the output mapping's idea of the input type will copy in a value that is not
stabilized by the hash, so is more-or-less random!
So we cannot do that. We have to do something else, which means we have to
decide what to do if two identical types with different nonroot flag values
pop up. The most sensible thing to do is probably to say that if all
instances of a type are non-root-visible, the linked output should also be
non-root-visible: any root-visible types in that set, and the output type is
root-visible again.
We implement this with a new cd_nonroot_consistency dynhash, which maps type
hashes to the value 0 ("all instances root-visible"), 1 ("all instances
non-root-visible") or 2 ("inconsistent"). After hashing is over, we save a
bit of memory by deleting everything from this hashtab that doesn't have a
value of 1 ("non-root-visible"), then use this to decide whether to emit any
given type as non-root-visible or not.
However... that's not quite enough. In cu-mapped links, we want to
disregard this whole thing because we just hide everything -- but in phase
2, when we take the smushed-together CUs resulting from phase 1 and
deduplicate them against each other, we want to do what the previous commits
implemented and ignore the non-root flag entirely, instead falling back to
preventing clashes by hiding anything that would be considered conflicting.
We extend the existing cu_mapped parameter to various bits of ctf_dedup so
that it is now tristate: 0 means a normal link, 1 means the smush-it-
together phase of cu-mapped links, and 2 means the final phase of cu-mapped
links. We do the hide-conflicting stuff only in phase 2, meaning that
normal links by GNU ld can always respect the value of the nonroot flag put
on types in the input.
(One extra thing added as part of this: you can now efficiently delete the
last value returned by ctf_dynhash_next() by calling
ctf_dynhash_next_remove.)
We bring back the ctf-nonroot-linking test with one tweak: linking now works
on mingw as long as you're using the ucrt libc, so re-enable it for better
test coverage on that platform.
libctf/
PR libctf/33047
* ctf-hash.c (ctf_dynhash_next_remove): New.
* ctf-impl.h (struct ctf_dedup) [cd_nonroot_consistency]: New.
* ctf-link.c (ctf_link_deduplicating): Differentiate between
cu-mapped and non-cu-mapped links, even in the final phase.
* ctf-dedup.c (ctf_dedup_hash_type): Callback prototype addition.
Get the non-root flag and pass it down.
(ctf_dedup_rhash_type): Callback prototype addition. Document
restrictions on use of the nonroot flag.
(ctf_dedup_populate_mappings): Populate cd_nonroot_consistency.
(ctf_dedup_hash_type_fini): New function: delete now-unnecessary
values from cd_nonroot_consistency.
(ctf_dedup_init): Initialize it.
(ctf_dedup_fini): Destroy it.
(ctf_dedup): cu_mapping is now cu_mapping_phase. Call
ctf_dedup_hash_type_fini.
(ctf_dedup_emit_type): Use cu_mapping_phase and
cd_nonroot_consistency to propagate the non-root flag into outputs
for normal links, and to do name-based conflict checking only for
phase 2 of cu-mapped links.
(ctf_dedup_emit): cu_mapping is now cu_mapping_phase. Adjust
assertion accordingly.
* testsuite/libctf-writable/ctf-nonroot-linking.c: Bring back.
* testsuite/libctf-writable/ctf-nonroot-linking.lk: Likewise.
If types are conflicting, they are usually moved into separate child dicts
-- but not always. If they are added to the same dict by the cu-mapping
mechanism (as used e.g. for multi-TU kernel modules), we can easily end
up adding multiple conflicting types with the same name to the same dict.
The mechanism used for turning on the non-root-visible flag in order to do
this had a kludge attached which always hid types with the same name,
whether or not they were conflicting. This is unnecessary and can hide
types that should not be hidden, as well as hiding bugs. Remove it, and
replace it with two different approaches:
- for everything but cu-mapped links (the in-memory first phase of a link
with ctf_link_add_cu_mapping in force), check for duplicate names if
types are conflicting, and mark them as hidden if the names are found.
This will never happen in normal links (in an upcoming commit we will
suppress doing even this much in such cases).
- for cu-mapped links, the only case that merges multiple distinct target
dicts into one, we apply a big hammer and simply hide everything! The
non-root flag will be ignored in the next link phase anyway (which dedups
the cu-mapped pieces against each other), and this way we can be sure
that merging multiple types cannot incur name clashes at this stage.
The result seems to work: the only annoyance is that when enums with
conflicting enumerators are found in a single cu-mapped child (so, really
multiple merged children), you may end up with every instance of that enum
being hidden for reasons of conflictingness. I don't see a real way to
avoid that.
libctf/
PR libctf/33047
* ctf-dedup.c (ctf_dedup_emit_type): Only consider non
conflicting types. Improve type hiding in the presence of clashing
enumerators. Hide everything when doing a cu-mapped link: they will
be unhidden by the next link pass if nonconflicting.
This reverts commit 87b2f67310.
It is based on a misconception, that hidden types in the deduplicator
input should always be hidden in the output. For cu-mapped links,
and final links following cu-mapped links, this is not true: we want
to hide inputs if they were conflicting on the output and no more.
We will reintroduce the testcase once a better fix is found.
libctf/
PR libctf/33047
* ctf-dedup.c (ctf_dedup_emit_type): Don't respect the nonroot flag.
* testsuite/libctf-writable/ctf-nonroot-linking.c: Removed.
* testsuite/libctf-writable/ctf-nonroot-linking.lk: Removed.
ctf_arc_import_parent, called by the cached-opening machinery used by
ctf_archive_next and archive-wide lookup functions like
ctf_arc_lookup_symbol, has an err-pointer parameter like all other opening
functions. Unfortunately it unconditionally initializes it whenever
provided, even if there was no error, which can lead to its being
initialized to an uninitialized value. This is not technically an
API-contract violation, since we don't define what happens to the error
value except when an error happens, but it is still unpleasant.
Initialize it only when there is an actual error, so we never initialize it
to an uninitialized value.
While we're at it, improve all the opening pathways: on success, set errp to
0, rather than leaving it what it was, reducing the likelihood of
uninitialized error param returns in callers too. (This is inconsistent
with the treatment of ctf_errno(), but the err value being a parameter
passed in from outside makes the divergence acceptable: in open functions,
you're never going to be overwriting some old error value someone might want
to keep around across multiple calls, some of which are successful and some
of which are not.)
Soup up existing tests to verify all this.
Thanks to Bruce McCulloch for the original patch, and Stephen Brennan for
the report.
libctf/
PR libctf/32903
* ctf-archive.c (ctf_arc_open_internal): Zero errp on success.
(ctf_dict_open_sections): Zero errp at the start.
(ctf_arc_import_parent): Intialize err.
* ctf-open.c (ctf_bufopen): Zero errp at the start.
* testsuite/libctf-lookup/add-to-opened.c: Make sure one-element
archive opens update errp.
* testsuite/libctf-writable/ctf-compressed.c: Make sure real archive
opens update errp.
GCC permits not only unnamed structs and unions, but cv-qualified ones.
Our earlier fix in 6c3a38777b supported
unnamed structs and unions, but only unqualified ones.
Resolving away cvr-quals of nameless fields (and, irrelevantly, typedefs)
is easy and fixes this problem.
Tests adjusted accordingly.
libctf/
PR libctf/32746
* ctf-types.c (ctf_member_next): Resolve away cv-quals.
(ctf_member_info): Likewise.
* testsuite/libctf-lookup/struct-iteration-ctf.c: Add a cv-qualified
type or two: make sure to keep a non-qualified one.
* testsuite/libctf-lookup/struct-iteration.c: Verify consistency
of ctf_member_next and ctf_member_info.
* testsuite/libctf-lookup/struct-iteration.lk: Adjust.
Tested-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Slices had a bunch of horrible usability problems. In particular, while
towers of cv-quals are resolved away by functions that need to do it, towers
of cv-quals with slices in the middle are not resolved away by functions
like ctf_enum_value that can see through slices: resolving volatile -> slice
-> const -> enum will leave it with a 'const', which will error pointlessly,
annoying callers, who reasonably expect slices to be more invisible than
this. (The user-callable ctf_type_resolve still does not resolve away
slices, because this is the only way users can see that the slices are there
at all.)
This is induced by a fix for another wart: ctf_add_enumerator does not
resolve anything away at all, so you can't even add enumerators to const or
volatile enums -- and more problematically, you can't add enumerators to
enums with an explicit encoding without resolving away the types by hand,
since ctf_add_enum_encoded works by returning a slice! ctf_add_enumerator
now resolves away all of those, so any cvr-or-typedef-or-slice-qual
terminating in an enum can be added to, exactly as callers likely expect.
(New tests added.)
libctf/
* ctf-create.c (ctf_add_enumerator): Resolve away cvr-qualness.
* ctf-types.c (ctf_type_resolve_unsliced): Don't terminate at
the first slice.
* testsuite/libctf-writable/slice-of-slice.*: New test.
<memory.h> is not needed and not standardized and is just an alias for
<string.h>.
<sys/param.h> is not needed and not standardized and contains a kitchen
sink of various unportable definitions not agreed upon and best done
manually or through other headers.
These fixes are needed to compile binutils on Sortix and other operating
systems with a strict POSIX.1-2024 libc without obsolete features.
Signed-off-by: Jonas 'Sortie' Termansen <sortie@maxsi.org>
When building gdb, I run into:
...
ctf-spec.texi:809: warning: @xref should not appear on @multitable line
...
The line in question is:
...
@multitable {Kind} {@code{CTF_K_VOLATILE}} {Indicates a type that cannot be represented in CTF, or that} {@xref{Pointers typedefs and cvr-quals}}
...
which defines a prototype row with 4 columns.
However, the table only has 3 colums:
...
@headitem Kind @tab Macro @tab Purpose
...
Fix the warning by removing the item in the prototype row representing a fourth column.
Tested on aarch64-linux.
PR libctf/32044
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32044
Previous code included the full $srcdir pathnames in the individual
subtest PASS/FAIL names, which makes it difficult to compute
comparisons or regressions between test runs on different machines.
This version switches to the basename only, which are common.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
This failed to properly byteswap its return value.
The ctf_archive format predates the idea of "just write natively and
flip on open", and byteswaps all over the place. It's too easy to
forget one. The next revision of the archive format (not versioned,
so we just tweak the magic number instead) should be native-endianned
like the dicts inside it are.
libctf/
* ctf-archive.c (ctf_archive_count): Byteswap return value.
If you asprintf something and then use it only as input to another asprintf,
it helps to free it afterwards.
libctf/
* ctf-dump.c (ctf_dump_header): Free the flagstr after use.
(ctf_dump): Make a NULL return slightly clearer.
A bug in ctf_dtd_delete led to refs in the string table to the
names of non-root-visible types not being removed when the DTD
was. This seems harmless, but actually it would lead to a write
down a pointer into freed memory if such a type was ctf_rollback()ed
over and then the dict was serialized (updating all the refs as the
strtab was serialized in turn).
Bug introduced in commit fe4c2d5563
("libctf: create: non-root-visible types should not appear in name tables")
which is included in binutils 2.35.
libctf/
* ctf-create.c (ctf_dtd_delete): Remove refs for all types
with names, not just root-visible ones.
The dict and archive opening code in libctf is somewhat unusual, because
unlike everything else, it cannot report errors by setting an error on the
dict, because in case of error there isn't one. They get passed an error
integer pointer that is set on error instead.
Inside ctf_bufopen this is implemented by calling ctf_set_open_errno and
passing it a positive error value. In turn this means that most things it
calls (including init_static_types) return zero on success and a *positive*
ECTF_* or errno value on error.
This trickles down to ctf_dynhash_insert_type, which is used by
init_static_types to add newly-detected types to the name tables. This was
returning the error value it received from a variety of functions without
alteration. ctf_dynhash_insert conformed to this contract by returning a
positive value on error (usually OOM), which is unfortunate for multiple
reasons:
- ctf_dynset_insert returns a *negative* value
- ctf_dynhash_insert and ctf_dynset_insert don't take an fp, so the value
they return is turned into the errno, so it had better be right, callers
don't just check for != 0 here
- more or less every single caller of ctf_dyn*_insert in libctf other than
ctf_dynhash_insert_type (and there are a *lot*, mostly in the
deduplicator) assumes that ctf_dynhash_insert returns a negative value
on error, even though it doesn't. In practice the only possible error is
OOM, but if OOM does happen we end up with a nonsense error value.
The simplest fix for this seems to be to make ctf_dynhash_insert and
ctf_dynset_insert conform to the usual interface contract: negative
values are errors. This in turn means that ctf_dynhash_insert_type
needs to change: let's make it consistent too, returning a negative
value on error, putting the error on the fp in non-negated form.
init_static_types_internal adapts to this by negating the error return from
ctf_dynhash_insert_type, so the value handed back to ctf_bufopen is still
positive: the new call site in ctf_track_enumerator does not need to change.
(The existing tests for this reliably detect when I get it wrong.
I know, because they did.)
libctf/
* ctf-hash.c (ctf_dynhash_insert): Negate return value.
(ctf_dynhash_insert_type): Set de-negated error on the dict:
return negated error.
* ctf-open.c (init_static_types_internal): Adapt to this change.
The recent change to detect duplicate enum values and return ECTF_DUPLICATE
when found turns out to perturb a great many callers. In particular, the
pahole-created kernel BTF has the same problem we historically did, and
gleefully emits duplicated enum constants in profusion. Handling the
resulting duplicate errors from BTF -> CTF converters reasonably is
unreasonably difficult (it amounts to forcing them to skip some types or
reimplement the deduplicator).
So let's step back a bit. What we care about mostly is that the
deduplicator treat enums with conflicting enumeration constants as
conflicting types: programs that want to look up enumeration constant ->
value mappings using the new APIs to do so might well want the same checks
to apply to any ctf_add_* operations they carry out (and since they're
*using* the new APIs, added at the same time as this restriction was
imposed, there is likely to be no negative consequence of this).
So we want some way to allow processes that know about duplicate detection
to opt into it, while allowing everyone else to stay clear of it: but we
want ctf_link to get this behaviour even if its caller has opted out.
So add a new concept to the API: dict-wide CTF flags, set via
ctf_dict_set_flag, obtained via ctf_dict_get_flag. They are not bitflags
but simple arbitrary integers and an on/off value, stored in an unspecified
manner (the one current flag, we translate into an LCTF_* flag value in the
internal ctf_dict ctf_flags word). If you pass in an invalid flag or value
you get a new ECTF_BADFLAG error, so the caller can easily tell whether
flags added in future are valid with a particular libctf or not.
We check this flag in ctf_add_enumerator, and set it around the link
(including on child per-CU dicts). The newish enumerator-iteration test is
souped up to check the semantics of the flag as well.
The fact that the flag can be set and unset at any time has curious
consequences. You can unset the flag, insert a pile of duplicates, then set
it and expect the new duplicates to be detected, not only by
ctf_add_enumerator but also by ctf_lookup_enumerator. This means we now
have to maintain the ctf_names and conflicting_enums enum-duplication
tracking as new enums are added, not purely as the dict is opened.
Move that code out of init_static_types_internal and into a new
ctf_track_enumerator function that addition can also call.
(None of this affects the file format or serialization machinery, which has
to be able to handle duplicate enumeration constants no matter what.)
include/
* ctf-api.h (CTF_ERRORS) [ECTF_BADFLAG]: New.
(ECTF_NERR): Update.
(CTF_STRICT_NO_DUP_ENUMERATORS): New flag.
(ctf_dict_set_flag): New function.
(ctf_dict_get_flag): Likewise.
libctf/
* ctf-impl.h (LCTF_STRICT_NO_DUP_ENUMERATORS): New flag.
(ctf_track_enumerator): Declare.
* ctf-dedup.c (ctf_dedup_emit_type): Set it.
* ctf-link.c (ctf_create_per_cu): Likewise.
(ctf_link_deduplicating_per_cu): Likewise.
(ctf_link): Likewise.
(ctf_link_write): Likewise.
* ctf-subr.c (ctf_dict_set_flag): New function.
(ctf_dict_get_flag): New function.
* ctf-open.c (init_static_types_internal): Move enum tracking to...
* ctf-create.c (ctf_track_enumerator): ... this new function.
(ctf_add_enumerator): Call it.
* libctf.ver: Add the new functions.
* testsuite/libctf-lookup/enumerator-iteration.c: Test them.
We set this flag at the top of ctf_link_write (to tell ctf_serialize, way
down under the archive file writing functions, to do the various link- time
serialization things like symbol filtering and the like), but we never
remember to clear it except on error. This is probably bad if you want to
serialize the dict yourself directly in the future after linking it (which
is... definitely a *possible* use of the API, if rather strange).
libctf/
* ctf-link.c (ctf_link_write): Clear LCTF_LINKING before exit.
