The current implementation of gdb_py_tp_name() leaks a reference to the
object returned by PyType_GetFullyQualifiedName() for Python >= 3.13, and
PyType_GetQualName() for Python >= 3.11.
Managing the strong reference on the returned PyObject* with a gdbpy_ref<>
fixes the reference count, but also causes the temporary object to be
deallocated on function exit. As a consequence, the 'const char *' returned
by PyUnicode_AsUTF8AndSize() becomes dangling and can no longer safely be
returned.
The proposed approach consists in changing gdb_py_tp_name() to return a
std::string, and forcing a copy of the 'const char *' value, statically
or dynamically allocated, stored in a temporary or non-temporary PyObject,
depending on the version of Python it was compiled with.
An unfortunate side effect of this fix is that every call sites where the
tp_name is printed, must now use `.c_str()' because PyErr_Format()
and its siblings cannot handle std::string.
Approved-By: Andrew Burgess <aburgess@redhat.com>
I noticed that while gdb_python_initialized is an int, there's also an
assignment using false:
...
gdb_python_initialized = false;
...
Fix this by converting gdb_python_initialized to bool type.
Approved-by: Kevin Buettner <kevinb@redhat.com>
Result of:
...
$ sed -i 's/Py_RETURN_NONE/return py_none ().release ()/' gdb/python/*.{c,h}
...
with the changes in py_none reverted.
Likewise for Py_RETURN_TRUE,Py_RETURN_FALSE and Py_RETURN_NOTIMPLEMENTED.
Approved-By: Tom Tromey <tom@tromey.com>
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=34145
GDB currently initializes CPython using the PyConfig C API introduced in
Python 3.8 (PEP 597). From an ABI stability perspective, this API has a
major drawback: it exposes a plain structure whose fields may be added or
removed between Python versions. As a result, it was excluded from the
Python limited API.
Python 3.14 introduced a new configuration API (PEP 741) that avoids
exposing plain structures and instead, operates via opaque pointers.
This design makes it much more suitable for inclusion in the Python
limited API. Indeed, this was the original intent of the PEP-741 author.
However, CPython maintainers ultimately decided otherwise.
Since GDB aims at using the CPython stable API to avoid rebuilding for
each Python version, the absence of a configuration API in the limited
C API constitutes a blocker. Nevertheless, this can be worked around by
using PEP-741 configuration API, whose design is compatible with the
limited C API. It is relatively safe to assume that this API will stay
around for some time.
In this perspective, this patch adds support for using the PEP-741 config
API starting from Python 3.14. When Py_LIMITED_API is defined, the
required functions are exposed as external symbols via a workaround header.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
Approved-By: Tom Tromey <tom@tromey.com>
This changes a number of error messages in gdb to use command_style.
In some places I've added double quotes around the command name for
consistency with other messages.
Approved-By: Andrew Burgess <aburgess@redhat.com>
Py_TYPE (self)->tp_name is the traditional idiomatic way to get a Python
type's fully qualified name. However, in the context of the Python
limited API, PyTypeObject is opaque, so the 'tp_name' attribute is no
longer accessible. Additionally, retrieving the type of a Python object
requires Py_TYPE, which is only available as part of the stable API
starting with Python 3.14.
This patch increases minimal Python limited API version from 3.11 to 3.14.
It also introduces two new helpers to retrieve a type's fully qualified
name: gdb_py_tp_name() and gdbpy_py_obj_tp_name(), which extract the fully
qualified name from a PyTypeObject and a PyObject, respectively. Ifdefery
allows these wrappers to select the appropriate API depending on the Python
version and whether the Python limited API is enabled. For any Python
version less than 3.13, gdb_py_tp_name() fallbacks using __qualname__
instead. However, the result may differ slightly in some cases, e.g. the
module name may be missing.
Finally, this patch adapts the existing code to use these wrappers, and
adjusts some test expectations to use the fully qualified name (or
__qualname__ for versions <= 3.13) where it was not previously used.
Note that the corner case where the module name would be missing does not
appear in the testsuite.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
Approved-By: Tom Tromey <tom@tromey.com>
Replace all occurrences of:
strcmp (...) == 0
strcmp (...) != 0
!strcmp (...)
0 == strcmp (...)
strcmp (...) directly used as a boolean predicate
with the equivalent expression using streq.
This is for consistency (we already use streq as some places in the
testsuite) but also for clarity. I think that streq is clearer on the
intent than strcmp. It's also a bit shorter.
Change-Id: Ibbf5261b1872c240bc0c982c147f6a5477275a91
Approved-By: Andrew Burgess <aburgess@redhat.com>
* PyTuple_GET_ITEM -> PyTuple_GetItem
* PyTuple_SET_ITEM -> PyTuple_SetItem
* PyTuple_GET_SIZE -> PyTuple_Size
Unlike PyTuple_SET_ITEM(), PyTuple_SetItem() returns an integer: 0 on
success and -1 on error (e.g. IndexError). The existing code must therefore
be updated to handle this new behaviour.
Since processing now stops when PyTuple_SetItem() returns an error, some
resources (such as newly allocated tuples) must be properly deallocated in
error paths. To address this, this patch replaces the use of raw 'PyObject *'
pointers with gdbpy_ref<>, a reference-counted wrapper around 'PyObject *',
which automatically decrements the reference count on early exit.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
Approved-By: Tom Tromey <tom@tromey.com>
This changes gdbarch_to_arch_object to return a gdbpy_ref<>,
using the type system to convey that a new reference is always
returned.
Approved-By: Simon Marchi <simon.marchi@efficios.com>
This changes type_to_type_object to return a gdbpy_ref<>,
using the type system to convey that a new reference is always
returned.
Approved-By: Simon Marchi <simon.marchi@efficios.com>
This changes value_to_value_object to return a gdbpy_ref<>,
using the type system to convey that a new reference is always
returned.
Approved-By: Simon Marchi <simon.marchi@efficios.com>
This changes symtab_and_line_to_sal_object to return a gdbpy_ref<>,
using the type system to convey that a new reference is always
returned.
Approved-By: Simon Marchi <simon.marchi@efficios.com>
This patch replaces PyImport_ExtendInittab () with its limited C
API equivalent, PyImport_AppendInittab (), a convenience wrapper
around PyImport_ExtendInittab ().
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
Approved-By: Tom Tromey <tom@tromey.com>
This patch replaces Py_CompileStringExFlags () with its limited C API
equivalent, Py_CompileString (). The eval_python_command () helper is
now exposed through the private GDB Python API as a utility function.
PyRun_SimpleString () is replaced with eval_python_command () to avoid
code duplication.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
Approved-By: Tom Tromey <tom@tromey.com>
This updates the copyright headers to include 2026. I did this by
running gdb/copyright.py and then manually modifying a few files as
noted by the script.
In C++20, "module" is an "identifier with special meaning". While it
can be used as a variable name, it highlights as a keyword in Emacs,
and I think it's nicer to just avoid it.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32660
Add gdb_rl_tilde_expand, a wrapper around readline's tilde_expand that
returns a gdb::unique_xmalloc_ptr<char>. Change all callers of
tilde_expand to use gdb_rl_tilde_expand (even the couple of spots that
release it immediatly, for consistency). This simplifies a few callers.
The name gdb_tilde_expand is already taken by a home-made implementation
in gdbsupport/gdb_tilde_expand.{h.cc}. I wonder if we could just use
that one instead of readline's tilde_expand, but that's an orthogonal
question. I don't know how they differ, and I don't want to introduce
behavior changes in this patch.
Change-Id: I6d34eef19f86473226df4ae56d07dc01912e3131
Approved-By: Tom Tromey <tom@tromey.com>
The GCC plugin that implements the Python API checker does not appear
to really be maintained. And, as far as I know, it never really
worked for C++ code anyway. Considering those factors, and that no
one has tried to run it in years, I think it's time to remove the
macros from the gdb source.
Approved-By: Simon Marchi <simon.marchi@efficios.com>
On ppc64le-linux (AlmaLinux 9.6) with python 3.9 and test-case
gdb.python/py-failed-init.exp I run into:
...
builtin_spawn $gdb -nw -nx -q -iex set height 0 -iex set width 0 \
-data-directory $build/gdb/data-directory -iex set interactive-mode on^M
Python path configuration:^M
PYTHONHOME = 'foo'^M
PYTHONPATH = (not set)^M
program name = '/usr/bin/python'^M
isolated = 0^M
environment = 1^M
user site = 1^M
import site = 1^M
sys._base_executable = '/usr/bin/python'^M
sys.base_prefix = 'foo'^M
sys.base_exec_prefix = 'foo'^M
sys.platlibdir = 'lib64'^M
sys.executable = '/usr/bin/python'^M
sys.prefix = 'foo'^M
sys.exec_prefix = 'foo'^M
sys.path = [^M
'foo/lib64/python39.zip',^M
'foo/lib64/python3.9',^M
'foo/lib64/python3.9/lib-dynload',^M
]^M
Fatal Python error: init_fs_encoding: failed to get the Python codec of the \
filesystem encoding^M
Python runtime state: core initialized^M
ModuleNotFoundError: No module named 'encodings'^M
^M
Current thread 0x00007fffabe18480 (most recent call first):^M
<no Python frame>^M
ERROR: (eof) GDB never initialized.
Couldn't send python print (1) to GDB.
UNRESOLVED: gdb.python/py-failed-init.exp: gdb-command<python print (1)>
Couldn't send quit to GDB.
UNRESOLVED: gdb.python/py-failed-init.exp: quit
...
The test-case expects gdb to present a prompt, but instead gdb calls exit
with this back trace:
...
(gdb) bt
#0 0x00007ffff6e4bfbc in exit () from /lib64/glibc-hwcaps/power10/libc.so.6
#1 0x00007ffff7873fc4 in fatal_error.lto_priv () from /lib64/libpython3.9.so.1.0
#2 0x00007ffff78aae60 in Py_ExitStatusException () from /lib64/libpython3.9.so.1.0
#3 0x00007ffff78c0e58 in Py_InitializeEx () from /lib64/libpython3.9.so.1.0
#4 0x0000000010b6cab4 in py_initialize_catch_abort () at gdb/python/python.c:2456
#5 0x0000000010b6cfac in py_initialize () at gdb/python/python.c:2540
#6 0x0000000010b6d104 in do_start_initialization () at gdb/python/python.c:2595
#7 0x0000000010b6eaac in gdbpy_initialize (extlang=0x11b7baf0 <extension_language_python>)
at gdb/python/python.c:2968
#8 0x000000001069d508 in ext_lang_initialization () at gdb/extension.c:319
#9 0x00000000108f9280 in captured_main_1 (context=0x7fffffffe870)
at gdb/main.c:1100
#10 0x00000000108fa3cc in captured_main (context=0x7fffffffe870)
at gdb/main.c:1372
#11 0x00000000108fa4d8 in gdb_main (args=0x7fffffffe870) at gdb/main.c:1401
#12 0x000000001001d1d8 in main (argc=3, argv=0x7fffffffece8) at gdb/gdb.c:38
...
This may be a python issue [1].
The problem doesn't happen if we use the PyConfig approach instead of the
py_initialize_catch_abort approach.
Fix this by using the PyConfig approach starting 3.9 (previously, starting
3.10 to avoid Py_SetProgramName deprecation in 3.11).
It's possible that we have the same problem and need the same fix for 3.8, but
I don't have a setup to check that. Add a todo in a comment.
Tested on ppc64le-linux.
Approved-By: Tom Tromey <tom@tromey.com>
[1] https://github.com/python/cpython/issues/107827
It is already possible to produce styled output from Python by
converting the gdb.Style to its escape code sequence, and writing that
to the output stream.
But this commit adds an alternative option to the mix by extending the
existing gdb.write() function to accept a 'style' argument. The value
of this argument can be 'None' to indicate no style change should be
performed, this is the default, and matches the existing behaviour.
Or the new 'style' argument can be a gdb.Style object, in which case
the specified style is applied only for the string passed to
gdb.write, after which the default style is re-applied.
Using gdb.write with a style object more closely matches how GDB
handles styling internally, and has the benefit that the user doesn't
need to remember to restore the default style when they are done.
Reviewed-By: Eli Zaretskii <eliz@gnu.org>
Approved-By: Tom Tromey <tom@tromey.com>
PR ada/33217 points out that gdb incorrectly calls the <ctype.h>
functions. In particular, gdb feels free to pass a 'char' like:
char *str = ...;
... isdigit (*str)
This is incorrect as isdigit only accepts EOF and values that can be
represented as 'unsigned char' -- that is, a cast is needed here to
avoid undefined behavior when 'char' is signed and a character in the
string might be sign-extended. (As an aside, I think this API seems
obviously bad, but unfortunately this is what the standard says, and
some systems check this.)
Rather than adding casts everywhere, this changes all the code in gdb
that uses any <ctype.h> API to instead call the corresponding c-ctype
function.
Now, c-ctype has some limitations compared to <ctype.h>. It works as
if the C locale is in effect, so in theory some non-ASCII characters
may be misclassified. This would only affect a subset of character
sets, though, and in most places I think ASCII is sufficient -- for
example the many places in gdb that check for whitespace.
Furthermore, in practice most users are using UTF-8-based locales,
where these functions aren't really informative for non-ASCII
characters anyway; see the existing workarounds in gdb/c-support.h.
Note that safe-ctype.h cannot be used because it causes conflicts with
readline.h. And, we canot poison the <ctype.h> identifiers as this
provokes errors from some libstdc++ headers.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=33217
Approved-By: Simon Marchi <simon.marchi@efficios.com>
For background, see this thread:
https://inbox.sourceware.org/gdb-patches/20250612144607.27507-1-tdevries@suse.de
Tom describes the issue clearly in the above thread, here's what he
said:
Once in a while, when running test-case gdb.base/bp-cmds-continue-ctrl-c.exp,
I run into:
...
Breakpoint 2, foo () at bp-cmds-continue-ctrl-c.c:23^M
23 usleep (100);^M
^CFAIL: $exp: run: stop with control-c (unexpected) (timeout)
FAIL: $exp: run: stop with control-c
...
This is PR python/32167, observed both on x86_64-linux and powerpc64le-linux.
This is not a timeout due to accidental slowness, gdb actually hangs.
The backtrace at the hang is (on cfarm120 running AlmaLinux 9.6):
...
(gdb) bt
#0 0x00007fffbca9dd94 in __lll_lock_wait () from
/lib64/glibc-hwcaps/power10/libc.so.6
#1 0x00007fffbcaa6ddc in pthread_mutex_lock@@GLIBC_2.17 () from
/lib64/glibc-hwcaps/power10/libc.so.6
#2 0x000000001067aee8 in __gthread_mutex_lock ()
at /usr/include/c++/11/ppc64le-redhat-linux/bits/gthr-default.h:749
#3 0x000000001067afc8 in __gthread_recursive_mutex_lock ()
at /usr/include/c++/11/ppc64le-redhat-linux/bits/gthr-default.h:811
#4 0x000000001067b0d4 in std::recursive_mutex::lock ()
at /usr/include/c++/11/mutex:108
#5 0x000000001067b380 in std::lock_guard<std::recursive_mutex>::lock_guard ()
at /usr/include/c++/11/bits/std_mutex.h:229
#6 0x0000000010679d3c in set_quit_flag () at gdb/extension.c:865
#7 0x000000001066b6dc in handle_sigint () at gdb/event-top.c:1264
#8 0x00000000109e3b3c in handler_wrapper () at gdb/posix-hdep.c:70
#9 <signal handler called>
#10 0x00007fffbcaa6d14 in pthread_mutex_lock@@GLIBC_2.17 () from
/lib64/glibc-hwcaps/power10/libc.so.6
#11 0x000000001067aee8 in __gthread_mutex_lock ()
at /usr/include/c++/11/ppc64le-redhat-linux/bits/gthr-default.h:749
#12 0x000000001067afc8 in __gthread_recursive_mutex_lock ()
at /usr/include/c++/11/ppc64le-redhat-linux/bits/gthr-default.h:811
#13 0x000000001067b0d4 in std::recursive_mutex::lock ()
at /usr/include/c++/11/mutex:108
#14 0x000000001067b380 in std::lock_guard<std::recursive_mutex>::lock_guard ()
at /usr/include/c++/11/bits/std_mutex.h:229
#15 0x00000000106799cc in set_active_ext_lang ()
at gdb/extension.c:775
#16 0x0000000010b287ac in gdbpy_enter::gdbpy_enter ()
at gdb/python/python.c:232
#17 0x0000000010a8e3f8 in bpfinishpy_handle_stop ()
at gdb/python/py-finishbreakpoint.c:414
...
What happens here is the following:
- the gdbpy_enter constructor attempts to set the current extension language
to python using set_active_ext_lang
- set_active_ext_lang attempts to lock ext_lang_mutex
- while doing so, it is interrupted by sigint_wrapper (the SIGINT handler),
handling a SIGINT
- sigint_wrapper calls handle_sigint, which calls set_quit_flag, which also
tries to lock ext_lang_mutex
- since std::recursive_mutex::lock is not async-signal-safe, things go wrong,
resulting in a hang.
The hang bisects to commit 8bb8f83467 ("Fix gdb.interrupt race"), which
introduced the lock, making PR python/32167 a regression since gdb 15.1.
Commit 8bb8f83467 fixes PR dap/31263, a race reported by ThreadSanitizer:
...
WARNING: ThreadSanitizer: data race (pid=615372)
Read of size 1 at 0x00000328064c by thread T19:
#0 set_active_ext_lang(extension_language_defn const*) gdb/extension.c:755
#1 scoped_disable_cooperative_sigint_handling::scoped_disable_cooperative_sigint_handling()
gdb/extension.c:697
#2 gdbpy_interrupt gdb/python/python.c:1106
#3 cfunction_vectorcall_NOARGS <null>
Previous write of size 1 at 0x00000328064c by main thread:
#0 scoped_disable_cooperative_sigint_handling::scoped_disable_cooperative_sigint_handling()
gdb/extension.c:704
#1 fetch_inferior_event() gdb/infrun.c:4591
...
Location is global 'cooperative_sigint_handling_disabled' of size 1 at 0x00000328064c
...
SUMMARY: ThreadSanitizer: data race gdb/extension.c:755 in \
set_active_ext_lang(extension_language_defn const*)
...
The problem here is that gdb.interrupt is called from a worker thread, and its
implementation, gdbpy_interrupt races with the main thread on some variable.
The fix presented here is based on the fix that Tom proposed, but
fills in the missing Mingw support.
The problem is basically split into two: hosts that support unix like
signals, and Mingw, which doesn't support signals.
For signal supporting hosts, I've adopted the approach that Tom
suggests, gdbpy_interrupt uses kill() to send SIGINT to the GDB
process. This is then handled in the main thread as if the user had
pressed Ctrl+C. For these hosts no locking is required, so the
existing lock is removed. However, everywhere the lock currently
exists I've added an assert:
gdb_assert (is_main_thread ());
If this assert ever triggers then we're setting or reading the quit
flag on a worker thread, this will be a problem without the mutex.
For Mingw, the current mutex is retained. This is fine as there are
no signals, so no chance of the mutex acquisition being interrupted by
a signal, and so, deadlock shouldn't be an issue.
To manage the complexity of when we need an assert, and when we need
the mutex, I've created 'struct ext_lang_guard', which can be used as
a RAII object. This object either performs the assertion check, or
acquires the mutex, depending on the host.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32167
Co-Authored-By: Tom de Vries <tdevries@suse.de>
Approved-By: Tom Tromey <tom@tromey.com>
This commit allows gdb.Color objects to be used to style output from
GDB commands written in Python, and the styled output should work
correctly with pagination.
There are two parts to fixing this:
First, GDB needs to be able to track the currently applied style
within the page_file class. This means that style changes need to be
achieved with calls to pager_file::emit_style_escape.
Now usually, GDB does this by calling something like fprintf_styled,
which takes care to apply the style for us. However, that's not
really an option here as a gdb.Color isn't a full style, and as the
gdb.Color object is designed to be converted directly into escape
sequences that can then be printed, we really need a solution that
works with this approach.
However pager_file::puts already has code in place to handle escape
sequences. Right now all this code does is spot the escape sequence
and append it to the m_wrap_buffer. But in this commit I propose that
we go one step further, parse the escape sequence back into a
ui_file_style object in pager_file::puts, and then we can call
pager_file::emit_style_escape.
If the parsing doesn't work then we can just add the escape sequence
to m_wrap_buffer as we did before.
But wait, how can this work if a gdb.Color isn't a full style? Turns
out that's not a problem. We only ever emit the escape sequence for
those parts of a style that need changing, so a full style that sets
the foreground color will emit the same escape sequence as a gdb.Color
for the foreground. When we convert the escape sequence back into a
ui_file_style, then we get a style with everything set to default,
except the foreground color.
I had hoped that this would be all that was needed. But unfortunately
this doesn't work because of the second problem...
... the implementation of the Python function gdb.write() calls
gdb_printf(), which calls gdb_vprintf(), which calls ui_file::vprintf,
which calls ui_out::vmessage, which calls ui_out::call_do_message, and
finally we reach cli_ui_out::do_message. This final do_message
function does this:
ui_file *stream = m_streams.back ();
stream->emit_style_escape (style);
stream->puts (str.c_str ());
stream->emit_style_escape (ui_file_style ());
If we imagine the case where we are emitting a style, triggered from
Python like this:
gdb.write(gdb.Color('red').escape_sequence(True))
the STYLE in this case will be the default ui_file_style(), and STR
will hold the escape sequence we are writing.
After the first change, where pager_file::puts now calls
pager_file::emit_style_escape, the current style of STREAM will have
been updated. But this means that the final emit_style_escape will
now restore the default style.
The fix for this is to avoid using the high level gdb_printf from
gdb.write(), and instead use gdb_puts instead. The gdb_puts function
doesn't restore the default style, which means our style modification
survives.
There's a new test included. This test includes what appears like a
pointless extra loop (looping over a single value), but this makes
sense given the origin of this patch. I've pulled this commit from a
longer series:
https://inbox.sourceware.org/gdb-patches/cover.1755080429.git.aburgess@redhat.com
I want to get this bug fix merged before GDB 17 branches, but the
longer series is not getting reviews, so for now I'm just merging this
one fix. Once the rest of the series gets merged, I'll be extending
the test, and the loop (mentioned above) will now loop over more
values.
This patch introduces a new macro, INIT_GDB_FILE. This is used to
replace the current "_initialize_" idiom when introducing a per-file
initialization function. That is, rather than write:
void _initialize_something ();
void
_initialize_something ()
{
...
}
... now you would write:
INIT_GDB_FILE (something)
{
...
}
The macro handles both the declaration and definition of the function.
The point of this approach is that it makes it harder to accidentally
cause an initializer to be omitted; see commit 2711e475 ("Ensure
cooked_index_entry self-tests are run"). Specifically, the regexp now
used by make-init-c seems harder to trick.
New in v2: un-did some erroneous changes made by the script.
The bulk of this patch was written by script.
Regression tested on x86-64 Fedora 41.
This commit adds a new gdb.warning() function. This function takes a
string and then calls GDB's internal warning() function. This will
display the string as a warning.
Using gdb.warning() means that the message will get the new emoji
prefix if the user has that feature turned on. Also, the message will
be sent to gdb.STDERR without the user having to remember to print to
the correct stream.
Reviewed-By: Eli Zaretskii <eliz@gnu.org>
Approved-By: Tom Tromey <tom@tromey.com>
This updates the copyright headers to include 2025. I did this by
running gdb/copyright.py and then manually modifying a few files as
noted by the script.
Approved-By: Eli Zaretskii <eliz@gnu.org>
Consider the following scenario:
...
$ cat hello
int
main (void)
{
printf ("hello\n");
return 0;
}
$ gcc -x c hello -g
$ gdb -q -iex "maint set gnu-source-highlight enabled off" a.out
Reading symbols from a.out...
(gdb) start
Temporary breakpoint 1 at 0x4005db: file hello, line 6.
Starting program: /data/vries/gdb/a.out
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Temporary breakpoint 1, main () at hello:6
6 printf ("hello\n");
...
This doesn't produce highlighting for line 6, because:
- pygments is used for highlighting instead of source-highlight, and
- pygments guesses the language for highlighting only based on the filename,
which in this case doesn't give a clue.
Fix this by:
- adding a language parameter to the extension_language_ops.colorize interface,
- passing the language as found in the debug info, and
- using it in gdb.styling.colorize to pick the pygments lexer.
The new test-case gdb.python/py-source-styling-2.exp excercises a slightly
different scenario: it compiles a c++ file with a .c extension, and checks
that c++ highlighting is done instead of c highlighting.
Tested on x86_64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
PR cli/30966
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30966
Currently, gdb.execute emits styled output when the command is sending
its output to GDB's stdout, and produces unstyled output when the
output is going to a string.
But it is not unreasonable that a user might wish to capture styled
output from a gdb.execute call, for example, the user might want to
display the styled output are part of some larger UI output block.
At the same time, I don't think it makes sense to always produce
styled output when capturing the output in a string; if what the user
wants is to parse the output, then the style escape sequences make
this far harder.
I propose that gdb.execute gain a new argument 'styling'. When False
we would always produce unstyled output, and when True we would
produce styled output if styling is not disabled by some other means.
For example, if GDB's 'set style enabled' is off, then I think
gdb.execute() should respect that. My assumption here is that
gdb.execute() might be executed by some extension. If the extension
thinks "styled output world work here", but the user hates styled
output, and has turned it off, then the extension should not be
forcing styled output on the user.
I chose 'styling' instead of 'styled' as the Python argument name
because we already use 'styling' in gdb.Value.format_string, and we
don't use 'styled' anywhere else. This is only a little bit of
consistency, but I still think it's a good thing.
The default for 'styling' will change depending on where the output is
going. When gdb.execute is sending the output to GDB's stdout then
the default for 'styling' is True. When the output is going to a
string, then the default for 'styling' will be False. Not only does
this match the existing behaviour, but I think this makes sense. By
default we assume that output captured in a string is going to be
parsed, and therefore styling markup is unhelpful, while output going
to stdout should receive styling.
This fixes part of the problem described in PR gdb/32676. That bug
tries to capture styled source listing in a string, which wasn't
previously possible.
There are some additional issues with capturing source code; GDB
caches the source code in the source code cache. However, GDB doesn't
check if the cached content is styled or not. As a consequence, if
the first time the source of a file is shown it is unstyled, then the
cached will hold the unstyled source code, and future requests will
return that unstyled source. I'll address this issue in a separate
patch.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32676
Approved-By: Tom Tromey <tom@tromey.com>
I noticed that it was not possible to return a string containing non
utf-8 characters using gdb.execute(). For example, using the binary
from the gdb.python/py-source-styling.exp test:
(gdb) file ./gdb/testsuite/outputs/gdb.python/py-source-styling/py-source-styling
Reading symbols from ./gdb/testsuite/outputs/gdb.python/py-source-styling/py-source-styling...
(gdb) set style enabled off
(gdb) list 26
21 int some_variable = 1234;
22
23 /* The following line contains a character that is non-utf-8. This is a
24 critical part of the test as Python 3 can't convert this into a string
25 using its default mechanism. */
26 char c[] = "�"; /* List this line. */
27
28 return 0;
29 }
(gdb) python print(gdb.execute('list 26', to_string=True))
Python Exception <class 'UnicodeDecodeError'>: 'utf-8' codec can't decode byte 0xc0 in position 250: invalid start byte
Error occurred in Python: 'utf-8' codec can't decode byte 0xc0 in position 250: invalid start byte
It is necessary to disable styling before the initial 'list 26',
otherwise the source will be passed through GNU source highlight, and
GNU source highlight seems to be smart enough to figure out the
character encoding, and convert it to UTF-8. This conversion is then
cached in the source cache, and the later Python gdb.execute call will
get back a pure UTF-8 string.
If source styling is disabled, then GDB caches the string without the
conversion to UTF-8, now the gdb.execute call gets back the string
with a non-UTF-8 character within it, and Python throws an error
during its attempt to create a string object.
I'm not, at this point, proposing a solution that tries to guess the
source file encoding, though I guess such a thing could be done.
Instead, I think we should make use of the host_charset(), as set by
the user with 'set host-charset ....' during the creation of the
Python string.
To do this, in execute_gdb_command, we should switch from
PyUnicode_FromString, which requires the input be a UTF-8 string, to
using PyUnicode_Decode, which allows GDB to specify the string
encoding. We will use host_charset().
With this done, it is now possible to list the file contents using
gdb.execute(), with the contents passing through a string:
(gdb) set host-charset ISO-8859-1
(gdb) python print(gdb.execute('list 26', to_string=True), end='')
21 int some_variable = 1234;
22
23 /* The following line contains a character that is non-utf-8. This is a
24 critical part of the test as Python 3 can't convert this into a string
25 using its default mechanism. */
26 char c[] = "À"; /* List this line. */
27
28 return 0;
29 }
(gdb)
There are already plenty of other places in GDB's Python code where we
use PyUnicode_Decode to create a string from something that might
contain user generated content, so I believe this is the correct
approach.
I noticed a
// namespace selftests
comment, which doesn't follow our comment formatting convention. I did
a find & replace to fix all the offenders.
Change-Id: Idf8fe9833caf1c3d99e15330db000e4bab4ec66c
In AIX a recent commit caused a build break with the error as shown below.
In file included from python/py-color.h:23,
from python/python.c:39:
python/python-internal.h:86:10: fatal error: Python.h: No such file or directory
86 | #include <Python.h>
In AIX, we run builds with and without python for our internal CI's.
A feature development made by the recent commit https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=6447969d0ac774b6dec0f95a0d3d27c27d158690
missed to guard Python.h in HAVE_PYTHON macro.
This commit is a fix for the same.
Approved-By: Tom Tromey <tom@tromey.com>
Colors can be specified as "none" for terminal's default color, as a name of
one of the eight standard colors of ISO/IEC 6429 "black", "red", "green", etc.,
as an RGB hexadecimal tripplet #RRGGBB for 24-bit TrueColor, or as an
integer from 0 to 255. Integers 0 to 7 are the synonyms for the standard
colors. Integers 8-15 are used for the so-called bright colors from the
aixterm extended 16-color palette. Integers 16-255 are the indexes into xterm
extended 256-color palette (usually 6x6x6 cube plus gray ramp). In
general, 256-color palette is terminal dependent and sometimes can be
changed with OSC 4 sequences, e.g. "\033]4;1;rgb:00/FF/00\033\\".
It is the responsibility of the user to verify that the terminal supports
the specified colors.
PATCH v5 changes: documentation fixed.
PATCH v6 changes: documentation fixed.
PATCH v7 changes: rebase onto master and fixes after review.
PATCH v8 changes: fixes after review.
A common problem is that python may fail to initialize if PYTHONHOME is
set incorrectly, or points to incompatible default libraries.
Likewise if PYTHONPATH points to incompatible modules.
For instance, say PYTHONHOME is foo, then we get:
...
$ gdb -q
Python path configuration:
PYTHONHOME = 'foo'
PYTHONPATH = (not set)
program name = '/usr/bin/python'
isolated = 0
environment = 1
user site = 1
safe_path = 0
import site = 1
is in build tree = 0
stdlib dir = 'foo/lib64/python3.12'
sys._base_executable = '/usr/bin/python'
sys.base_prefix = 'foo'
sys.base_exec_prefix = 'foo'
sys.platlibdir = 'lib64'
sys.executable = '/usr/bin/python'
sys.prefix = 'foo'
sys.exec_prefix = 'foo'
sys.path = [
'foo/lib64/python312.zip',
'foo/lib64/python3.12',
'foo/lib64/python3.12/lib-dynload',
]
Python Exception <class 'ModuleNotFoundError'>: No module named 'encodings'
Python not initialized
$
...
In this case, it might be easy to figure out what went wrong because of the
obviously incorrect pathnames, but that might not be the case if PYTHONHOME
points to an incompatible python installation.
Fix this by adding a warning with a description of the possible cause and what
to do about it:
...
Python initialization failed: \
failed to get the Python codec of the filesystem encoding
gdb: warning: Python failed to initialize with PYTHONHOME set. Maybe because \
it is set incorrectly? Maybe because it points to incompatible standard \
libraries? Consider changing or unsetting it, or ignoring it using "set \
python ignore-environment on" at early initialization.
...
Likewise for PYTHONPATH:
...
Python initialization failed: \
failed to get the Python codec of the filesystem encoding
gdb: warning: Python failed to initialize with PYTHONPATH set. Maybe because \
it points to incompatible modules? Consider changing or unsetting it, or \
ignoring it using "set python ignore-environment on" at early \
initialization.
...
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
PR python/32379
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32379
When using PYTHONDONTWRITEBYTECODE with an empty string we get:
...
$ PYTHONDONTWRITEBYTECODE= gdb -q -batch -ex "show python dont-write-bytecode"
Python's dont-write-bytecode setting is auto (currently on).
...
This is incorrect, it should be off.
The actual setting is correct, that was already fixed in commit 24d2cbc42c
("set/show python dont-write-bytecode fixes"), in function
python_write_bytecode.
Fix this by:
- factoring out new function env_python_dont_write_bytecode out of
python_write_bytecode, and
- using it in show_python_dont_write_bytecode.
Tested on x86_64-linux, using test-case gdb.python/py-startup-opt.exp and:
- PYTHONDONTWRITEBYTECODE=
- PYTHONDONTWRITEBYTECODE=1
- unset PYTHONDONTWRITEBYTECODE
Approved-By: Tom Tromey <tom@tromey.com>
PR python/32389
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32389
Configuration flags "python dont-write-bytecode" and
"python ignore-environment" have effect only at Python initialization.
For instance, setting "python dont-write-bytecode" here has no effect:
...
$ gdb -q
(gdb) show python dont-write-bytecode
Python's dont-write-bytecode setting is auto (currently off).
(gdb) python import sys
(gdb) python print (sys.dont_write_bytecode)
False
(gdb) set python dont-write-bytecode on
(gdb) python print (sys.dont_write_bytecode)
False
...
This is not clear in the code: we set Py_DontWriteBytecodeFlag and
Py_IgnoreEnvironmentFlag in set_python_ignore_environment and
set_python_dont_write_bytecode. Fix this by moving the setting of those
variables to py_initialization.
Furthermore, this is not clear to the user: after Python initialization, the
user can still modify the configuration flags, and observe the changed setting:
...
$ gdb -q
(gdb) show python ignore-environment
Python's ignore-environment setting is off.
(gdb) set python ignore-environment on
(gdb) show python ignore-environment
Python's ignore-environment setting is on.
(gdb)
...
Fix this by emitting a warning when trying to set these configuration flags
after Python initialization:
...
$ gdb -q
(gdb) set python ignore-environment on
warning: Setting python ignore-environment after Python initialization has \
no effect, try setting this during early initialization
(gdb) set python dont-write-bytecode on
warning: Setting python dont-write-bytecode after Python initialization has \
no effect, try setting this during early initialization, or try setting \
sys.dont_write_bytecode
...
and by keeping the values constant after Python initialization.
Since the auto setting for python dont-write-bytecode depends on the current
value of environment variable PYTHONDONTWRITEBYTECODE, we simply avoid it
after Python initialization:
...
$ gdb -q -batch \
-eiex "show python dont-write-bytecode" \
-iex "show python dont-write-bytecode"
Python's dont-write-bytecode setting is auto (currently off).
Python's dont-write-bytecode setting is off.
...
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
PR python/32388
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32388
I added ATTRIBUTE_UNUSED to py_initialize_catch_abort as a quick fix to deal
with it being unused for PY_VERSION_HEX >= 0x030a0000, but forgot to fix this
before committing.
Fix this now, by removing the attribute and using
'#if PY_VERSION_HEX < 0x030a0000' instead.
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
Function do_start_initialization has a large part dedicated to initializing
the python interpreter, as opposed to the rest of the function where
gdb-specific python support is initialized.
Factor out this part, as new function py_initialize, and rename the existing
py_initialize to py_initialize_catch_abort.
Refactor the new function py_initialize by getting rid of the nested:
...
#ifdef WITH_PYTHON_PATH
#if PY_VERSION_HEX < 0x030a0000
#else
#endif
#else
#endif
...
In particular, this changes behaviour for the "!defined (WITH_PYTHON_PATH)"
case.
For the "defined (WITH_PYTHON_PATH)" case, we've started using
Py_InitializeFromConfig () for PY_VERSION_HEX >= 0x030a0000 to deal with the
deprecation of Py_SetProgramName in 3.11.
For the "!defined (WITH_PYTHON_PATH)" case, we don't use Py_SetProgramName so
we stuck with Py_Initialize ().
However, in 3.12 Py_DontWriteBytecodeFlag and Py_IgnoreEnvironmentFlag got
deprecated and also here we need Py_InitializeFromConfig () to deal with this,
but the "!defined (WITH_PYTHON_PATH)" case didn't get updated.
This should be taken care of, now that we have this behavior:
- for PY_VERSION_HEX < 0x030a0000 we use Py_Initialize
- for PY_VERSION_HEX >= 0x030a0000 we use Py_InitializeFromConfig
I'm not sure how to test the "!defined (WITH_PYTHON_PATH)" though.
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
I tried out making python initialization fail by passing an incorrect
PYTHONHOME, and got:
...
$ PYTHONHOME=foo ./gdb.sh -q
Python path configuration:
PYTHONHOME = 'foo'
...
Python initialization failed: \
failed to get the Python codec of the filesystem encoding
Python not initialized
$
...
The relevant part of the code is:
...
static void
gdbpy_initialize (const struct extension_language_defn *extlang)
{
if (!do_start_initialization () && py_isinitialized && PyErr_Occurred ())
gdbpy_print_stack ();
gdbpy_enter enter_py;
...
What happens is:
- gdbpy_enter::gdbpy_enter () is called, where we run into:
'if (!gdb_python_initialized) error (_("Python not initialized"));'
- the error propagates to gdb's toplevel
- gdb print the error and exits.
It seems unnecesssary that we exit gdb. We could continue the
session without python support.
Fix this by:
- bailing out of gdbpy_initialize if !do_start_initialization
- bailing out of finalize_python if !gdb_python_initialized
This gets us instead:
...
$ PYTHONHOME=foo gdb -q
Python path configuration:
PYTHONHOME = 'foo'
...
Python initialization failed: \
failed to get the Python codec of the filesystem encoding
(gdb) python print (1)
Python not initialized
(gdb)
...
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
I tried out making python initialization fail by passing an incorrect
PYTHONHOME with python 3.6, and got:
...
$ PYTHONHOME=foo gdb -q
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'
Current thread 0x0000ffff89269c80 (most recent call first):
Fatal signal: Aborted
...
Aborted (core dumped)
$
...
This is as per spec: when Py_Initialize () fails, a fatal error is raised
using Py_FatalError.
This can be worked around using:
...
$ PYTHONHOME=foo gdb -q -eiex "set python ignore-environment on"
(gdb)
...
but it would be better if gdb didn't abort.
I found an article [1] describing two solutions:
- try out Py_Initialize in a separate process, and
- catch the abort using a signal handler.
This patch implements the latter solution.
Obviously we cannot call into python anymore after the abort, so we avoid
calling Py_IsInitialized (), and instead use a new variable py_isinitialized.
This gets us instead:
...
$ PYTHONHOME=foo gdb -q
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'
Current thread 0x0000fffecfd49c80 (most recent call first):
Python not initialized
$
...
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
PR python/32379
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32379
[1] https://stackoverflow.com/questions/7688374/how-to-i-catch-and-handle-a-fatal-error-when-py-initialize-fails
I tried out making python initialization fail by passing an incorrect
PYTHONHOME, and got:
...
$ PYTHONHOME=foo gdb -q
Python path configuration:
PYTHONHOME = 'foo'
...
Python Exception <class 'ModuleNotFoundError'>: No module named 'encodings'
Python not initialized
$
...
The relevant part of the code is:
...
static void
gdbpy_initialize (const struct extension_language_defn *extlang)
{
if (!do_start_initialization () && PyErr_Occurred ())
gdbpy_print_stack ();
gdbpy_enter enter_py;
...
What happens is that:
- do_start_initialization returns false because Py_InitializeFromConfig fails,
leaving us in the !Py_IsInitialized () state
- PyErr_Occurred () returns true
- gdbpy_print_stack is called, which prints
"Python Exception <class 'ModuleNotFoundError'>: No module named 'encodings"
The problem is that in the Py_IsInitialized () == false state, very few
functions can be called, and PyErr_Occurred is not one of them [1], and
likewise functions in gdbpy_print_stack.
Fix this by:
- guarding the PyErr_Occurred / gdbpy_print_stack part with Py_IsInitialized ().
- handling the !Py_IsInitialized () case by printing the failure PyStatus
in do_start_initialization
This gets us instead:
...
$ PYTHONHOME=foo ./gdb.sh -q
Python path configuration:
PYTHONHOME = 'foo'
...
Python initialization failed: failed to get the Python codec of the filesystem encoding
Python not initialized
$
...
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
[1] https://docs.python.org/3/c-api/init.html#before-python-initialization
I noticed in do_start_initialization:
...
std::string oldloc = setlocale (LC_ALL, NULL);
setlocale (LC_ALL, "");
...
if (count == (size_t) -1)
{
fprintf (stderr, "Could not convert python path to string\n");
return false;
}
setlocale (LC_ALL, oldloc.c_str ());
...
that the old locale is not restored if the "return false" is triggered.
Fix this by using SCOPE_EXIT.
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
In this review:
https://inbox.sourceware.org/gdb-patches/87wmirfzih.fsf@tromey.com
Tom pointed out that using PyObject_IsTrue as I was doing, though
technically fine, at least appears to be missing an error check, and
that it would be better to compare to Py_True directly. I made that
change in the patch Tom was commenting on, but I'd actually copied
that code from elsewhere in python/python.c, so this commit updates
the original code inline with Tom's review feedback.
There should be no functional change after this commit.
Approved-By: Tom Tromey <tom@tromey.com>
Implement extension_language_ops::find_objfile_from_buildid within
GDB's Python API. Doing this allows users to write Python extensions
that can help locate missing objfiles when GDB opens a core file. A
handler might perform some project- or site-specific actions to find a
missing objfile. Or might provide some project- or site-specific
advice to the user on how they can obtain the missing objfile.
The implementation is very similar to the approach taken in:
commit 8f6c452b5a
Date: Sun Oct 15 22:48:42 2023 +0100
gdb: implement missing debug handler hook for Python
The following new commands are added as commands implemented in
Python, this is similar to how the Python missing debug and unwinder
commands are implemented:
info missing-objfile-handlers
enable missing-objfile-handler LOCUS HANDLER
disable missing-objfile-handler LOCUS HANDLER
To make use of this extension hook a user will create missing objfile
handler objects, and registers these handlers with GDB. When GDB
opens a core file and encounters a missing objfile each handler is
called in turn until one is able to help. Here is a minimal handler
that does nothing useful:
import gdb
import gdb.missing_objfile
class MyFirstHandler(gdb.missing_objfile.MissingObjfileHandler):
def __init__(self):
super().__init__("my_first_handler")
def __call__(self, pspace, build_id, filename):
# This handler does nothing useful.
return None
gdb.missing_objfile.register_handler(None, MyFirstHandler())
Returning None from the __call__ method tells GDB that this handler
was unable to find the missing objfile, and GDB should ask any other
registered handlers.
Possible return values from a handler:
- None: This means the handler couldn't help. GDB will call other
registered handlers to see if they can help instead.
- False: The handler has done all it can, but the objfile couldn't
be found. GDB will not call any other handlers, and will
continue without the objfile.
- True: The handler has installed the objfile into a location where
GDB would normally expect to find it. GDB should repeat its
normal lookup process and the objfile should now be found.
- A string: The handler can return a filename, which is the missing
objfile. GDB will load this file.
Handlers can be registered globally, or per program space. GDB checks
the handlers for the current program space first, and then all of the
global handles. The first handler that returns a value that is not
None, has "handled" the missing objfile, at which point GDB continues.
The implementation of this feature is mostly straight forward. I have
reworked some of the missing debug file related code so that it can be
shared with this feature. E.g. gdb/python/lib/gdb/missing_files.py is
mostly content moved from gdb/python/lib/gdb/missing_debug.py, but
updated to be more generic. Now gdb/python/lib/gdb/missing_debug.py
and the new file gdb/python/lib/gdb/missing_objfile.py both call into
the missing_files.py file.
For gdb/python/lib/gdb/command/missing_files.py this is even more
extreme, gdb/python/lib/gdb/command/missing_debug.py is completely
gone now and gdb/python/lib/gdb/command/missing_files.py provides all
of the new commands in a generic way.
I have made one change to the existing Python API, I renamed the
attribute Progspace.missing_debug_handlers to
Progspace.missing_file_handlers. I don't see this as too
problematic. This attribute was only used to implement the missing
debug feature and was never documented beyond the fact that it
existed. There was no reason for users to be touching this attribute.
Reviewed-By: Eli Zaretskii <eliz@gnu.org>
In preparation for later commits in this series, rename
ext_lang_missing_debuginfo_result to ext_lang_missing_file_result.
A later commit will add additional Python APIs to handle different
types of missing files beyond just debuginfo.
This is just a rename commit, there should be no functional changes
after this commit.
Approved-By: Tom Tromey <tom@tromey.com>
Result of:
...
$ search="GDB_PY_HANDLE_EXCEPTION ("
$ replace="return gdbpy_handle_gdb_exception (nullptr, "
$ sed -i \
"s/$search/$replace/" \
gdb/python/*.c
...
Also remove the now unused GDB_PY_HANDLE_EXCEPTION.
No functional changes.
Tested on x86_64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
PR python/32163 points out that various types provided by gdb are not
added to the gdb module, so they aren't available for interactive
inspection. I think this is just an oversight.
This patch fixes the problem by introducing a new helper function that
both readies the type and then adds it to the appropriate module. The
patch also poisons PyType_Ready, the idea being to avoid this bug in
the future.
v2:
* Fixed a bug in original patch in gdb.Architecture registration
* Added regression test for the types mentioned in the bug
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32163
Reviewed-By: Alexandra Petlanova Hajkova <ahajkova@redhat.com>
