When enabling the Python limited API, pointers to Python C extension
objects can no longer be implicitly converted to 'PyObject *' by the
compiler.
gdbpy_ref_policy is a templated class that provides a generic interface
for incrementing and decrementing the reference counter on the given
object. It is used as a specialisation of the policy parameter in
gdb::ref_ptr, together with PyObject as the parameter type. As a result,
gdbpy_ref_policy always expects an argument derived from PyObject.
This patch fixes the resulting compilation issue by adding an explicit
static_cast to 'PyObject *' before passing the value to Py_INCREF and
Py_DECREF. As a side effect, these casts enforce, at compile time, that
the template type passed to gdbpy_ref_policy is a subclass of PyObject.
To provide a clearer diagnostic when an incorrect type is used, a
static_assert is added to gdbpy_ref_policy, avoiding obscure errors
originating from the static_cast. Finally, all C Python extension types
passed to gdbpy_ref_policy are updated to inherit from PyObject.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
Approved-By: Tom Tromey <tom@tromey.com>
GDB is currently using the Python unlimited API. Migrating the codebase
to the Python limited API would have for benefit to make a GDB build
artifacts compatible with older and newer versions of Python that they
were built with.
This patch prepares the ground for migrating the existing C extension
types from static types to heap-allocated ones, by removing the
dependency on tp_dictoffset, which is unavailable when using the limited
API.
One of the most common incompatibilities in the current static type
declarations is the tp_dictoffset slot, which specifies the dictionary
offset within the instance structure. Historically, the unlimited
API has provided two approaches to supply a dictionary for __dict__:
* A managed dictionary.
Setting Py_TPFLAGS_MANAGED_DICT in tp_flags indicates that the
instances of the type have a __dict__ attribute, and that the
dictionary is managed by Python.
According to the Python documentation, this is the recommended approach.
However, this flag was introduced in 3.12, together with
PyObject_VisitManagedDict() and PyObject_ClearManagedDict(), neither
of which is part of the limited API (at least for now). As a result,
this recommended approach is not viable in the context of the limited
API.
* An instance dictionary, for which the offset in the instance is
provided via tp_dictoffset.
According to the Python documentation, this "tp slot" is on the
deprecation path, and Py_TPFLAGS_MANAGED_DICT should be used instead.
Given the age of the GDB codebase and the requirement to support older
Python versions (>= 3.4), no need to argue that today, the implementation
of __dict__ relies on tp_dictoffset. However, in the context of the
limited API, PyType_Slot does not provide a Py_tp_dictoffset member, so
another approach is needed to provide __dict__ to instances of C extension
types.
Given the constraints of the limited API, the proposed solution consists
in providing a dictionary through a common base class, gdbpy__dict__wrapper.
This helper class owns a dictionary member corresponding to __dict__, and
any C extension type requiring a __dict__ must inherit from it. Since
extension object must also be convertible to PyObject, this wrapper class
publicly inherits from PyObject as well.
Access to the dictionary is provided via a custom getter defined in a
PyGetSetDef, similarily to what was previously done with gdb_py_generic_dict().
Because __dict__ participates in attribute look-up, and since this dictionary
is neither managed by Python nor exposed via tp_dictoffset, custom
implementations of tp_getattro and tp_setattro are required to correctly
redirect attribute look-ups to the dictionary. These custom implementations
— equivalent to PyObject_GenericGetAttr() and PyObject_GenericSetAttr() —
must be installed via tp_getattro / tp_setattro for static types, or
Py_tp_getattro / Py_tp_setattro for heap-allocated types.
- gdbpy__dict__wrapper: a base class for C extension objects that own a
__dict__.
- gdb_py_generic_dict_getter: a __dict__ getter for extension types
derived from gdbpy__dict__wrapper.
- gdb_py_generic_getattro: equivalent of PyObject_GenericGetAttr, but
fixes the look-up of __dict__.
- gdb_py_generic_setattro: equivalent of PyObject_GenericSetAttr, but
fixes the look-up of __dict__.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
Approved-By: Tom Tromey <tom@tromey.com>
When enabling the Python limited API, pointers to Python C extension
objects can no longer be implicitly converted to 'PyObject *' by the
compiler.
The lookup() method of gbdpy_registry returns a new reference to the
type object of the looked-up entry. It does so by calling Py_XINCREF()
to increment the reference counter of the returned type object. The
template parameter obj_type corresponds to the type of C extension
object type. With the Python limited API enabled, obj_type can no longer
be implicitly converted to 'PyObject *' when passed to Py_XINCREF().
This patch fixes the resulting compilation issue by adding an explicit
static_cast to 'PyObject *' before passing the value to Py_XINCREF().
As a side effect, this cast enforces, at compile time, that the template
type 'Storage::obj_type' passed to gdbpy_registry is a subclass of
PyObject. To provide a clearer diagnostic when an incorrect type is used,
a static_assert is added to gdbpy_registry, avoiding obscure errors
originating from the static_cast. Finally, the relevant C extension types
passed to gdbpy_registry are updated to inherit publicly from PyObject.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
Approved-By: Tom Tromey <tom@tromey.com>
This patch replaces Py_CompileStringExFlags () with its limited C API
equivalent, Py_CompileString (). The eval_python_command () helper is
now exposed through the private GDB Python API as a utility function.
PyRun_SimpleString () is replaced with eval_python_command () to avoid
code duplication.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23830
Approved-By: Tom Tromey <tom@tromey.com>
This updates the copyright headers to include 2026. I did this by
running gdb/copyright.py and then manually modifying a few files as
noted by the script.
In C++20, "module" is an "identifier with special meaning". While it
can be used as a variable name, it highlights as a keyword in Emacs,
and I think it's nicer to just avoid it.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32660
The GCC plugin that implements the Python API checker does not appear
to really be maintained. And, as far as I know, it never really
worked for C++ code anyway. Considering those factors, and that no
one has tried to run it in years, I think it's time to remove the
macros from the gdb source.
Approved-By: Simon Marchi <simon.marchi@efficios.com>
According to 'git annotate', the Py_TPFLAGS_CHECKTYPES was added to
python-internal.h way back when gdb was first ported to Python 3. It
was a compatibility fix for Python 2.
This is not needed any more, because Python 2 is no longer supported.
This patch removes the vestiges.
Approved-By: Simon Marchi <simon.marchi@efficios.com>
I noticed my IDE (VSCode) starting to automatically trim trailing
whitespaces on save, despite the setting for it being disabled. I
realized that this is because the .editorconfig file now has
trim_trailing_whitespace = true
for many file types. If we have this EditorConfig setting forcing
editors to trim trailing whitespaces, I think it would make sense to
clean up trailing whitespaces from our files. Otherwise, people will
always get spurious whitespace changes when editing these files.
I did a mass cleanup using this command:
$ find gdb gdbserver gdbsupport -type f \( \
-name "*.c" -o \
-name "*.h" -o \
-name "*.cc" -o \
-name "*.texi" -o \
-name "*.exp" -o \
-name "*.tcl" -o \
-name "*.py" -o \
-name "*.s" -o \
-name "*.S" -o \
-name "*.asm" -o \
-name "*.awk" -o \
-name "*.ac" -o \
-name "Makefile*" -o \
-name "*.sh" -o \
-name "*.adb" -o \
-name "*.ads" -o \
-name "*.d" -o \
-name "*.go" -o \
-name "*.F90" -o \
-name "*.f90" \
\) -exec sed -ri 's/[ \t]+$//' {} +
I then did an autotools regen, because we don't actually want to change
the Makefile and Makefile.in files that are generated.
Change-Id: I6f91b83e3b8c4dc7d5d51a2ebf60706120efe691
This commit starts adding some core file related features to the
Python API.
In this initial commit I've tried to keep the changes as small as
possible for easy review.
There's a new Python class gdb.Corefile, which represents a loaded
core file. This API doesn't allow the user to create their own
gdb.Corefile objects, a core file must be loaded using the 'core-file'
command, then a gdb.Corefile object can be obtained by querying the
inferior in which the core file was loaded.
There's a new attribute gdb.Inferior.corefile, this is None when no
core file is loaded, or contains a gdb.Corefile object if a core file
has been loaded.
Currently, the gdb.Corefile object has one attribute, and one method,
these are:
gdb.Corefile.filename -- the file name of the loaded core file.
gdb.Corefile.is_valid() -- indicates if a gdb.Corefile object is
valid or not. See notes below.
A gdb.Corefile object is only valid while the corresponding core file
is loaded into an inferior. Unloading the core file, or loading a
different one will cause a gdb.Corefile object to become invalid. For
example:
(gdb) core-file /tmp/core.54313
... snip ...
(gdb) python core=gdb.selected_inferior().corefile
(gdb) python print(core)
<gdb.Corefile inferior=1 filename='/tmp/core.54313'>
(gdb) python print(core.is_valid())
True
(gdb) core-file
No core file now.
(gdb) python print(core)
<gdb.Corefile (invalid)>
(gdb) python print(core.is_valid())
False
(gdb)
In order to track changes to the core file, there is a new observable
'core_file_changed', which accounts for the changes in corelow.c,
observable,c, and observable.h. Currently, this observable is not
visible as a Python event.
I chose to access the core file via the inferior even though the core
file BFD object is actually stored within the program_space. As such,
it might seem that the natural choice would be to add the attribute as
gdb.Progspace.corefile.
For background reading on my choice, please see:
https://inbox.sourceware.org/gdb-patches/577f2c47793acb501c2611c0e6c7ea379f774830.1668789658.git.aburgess@redhat.com
This patch was never merged, it is still on my backlog, but the
observation in that work is that some targets are not really
shareable. For example, the core_target (corelow.c) stores
information about the loaded core file within the target instance. As
such, each target instance represents a single loaded core file.
Except that the BFD part of the core file is stored in the
program_space, which is a little weird.
During review, Tom made the observation, that maybe we should
investigate moving the core file BFD into the core_target. I'm
inclined to agree with this as a direction of travel.
All this leaves us with two observations:
1. Currently, loading a core file into an inferior, then using
'add-inferior' will try to share the core_target between
inferiors. This is broken, and can trigger GDB crashes. The
obvious fix, without reworking core_target, is just to prevent
this sharing, making core_target per-inferior.
2. Having the core file information split between the core_target
instance, and the BFD stored in the program_space is a little
weird, and is really just historical. Planning for a future
where the BFD is also stored in the core_target might be wise.
So, if we imagine that the BFD is (one day) moved into the
core_target, and that the core_target really becomes non-shareable,
then it is, I think, clearer that the corefile attribute should live
on the gdb.Inferior object, not the gdb.Progspace object.
There's testing for all the functionality added in this commit.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32844
Reviewed-By: Eli Zaretskii <eliz@gnu.org>
Approved-By: Tom Tromey <tom@tromey.com>
It is already possible to produce styled output from Python by
converting the gdb.Style to its escape code sequence, and writing that
to the output stream.
But this commit adds an alternative option to the mix by extending the
existing gdb.write() function to accept a 'style' argument. The value
of this argument can be 'None' to indicate no style change should be
performed, this is the default, and matches the existing behaviour.
Or the new 'style' argument can be a gdb.Style object, in which case
the specified style is applied only for the string passed to
gdb.write, after which the default style is re-applied.
Using gdb.write with a style object more closely matches how GDB
handles styling internally, and has the benefit that the user doesn't
need to remember to restore the default style when they are done.
Reviewed-By: Eli Zaretskii <eliz@gnu.org>
Approved-By: Tom Tromey <tom@tromey.com>
Update gdbpy_create_ptid_object (python/py-infthread.c) to return a
gdbpy_ref<> rather than a 'PyObject *'. This reduces the chances that
a caller will leak an object, though no such memory leaks are fixed in
this commit, this is just a code improvement patch.
There should be no user visible changes after this commit.
Approved-By: Simon Marchi <simon.marchi@efficios.com>
A commit I recently pushed:
commit 0b5023cc71
Date: Sat Apr 12 09:15:53 2025 +0100
gdb/python/guile: user created prefix commands get help list
can trigger a segfault if a user tries to create nested prefix
commands. For example, this will trigger a crash:
(gdb) python gdb.ParameterPrefix("prefix-1", gdb.COMMAND_NONE)
(gdb) python gdb.ParameterPrefix("prefix-1 prefix-2", gdb.COMMAND_NONE)
Fatal signal: Segmentation fault
... etc ...
If the user adds an actual parameter under 'prefix-1' before creating
'prefix-2', then everything is fine:
(gdb) python gdb.ParameterPrefix("prefix-1", gdb.COMMAND_NONE)
(gdb) python gdb.Parameter('prefix-1 param-1', gdb.COMMAND_NONE, gdb.PARAM_BOOLEAN)
(gdb) python gdb.ParameterPrefix("prefix-1 prefix-2", gdb.COMMAND_NONE)
The mistake in the above patch is in how gdbpy_parse_command_name is
used. The BASE_LIST output argument from this function points to the
list of commands for the prefix, not to the prefix command itself.
So when gdbpy_parse_command_name is called for 'prefix-1 prefix-2',
BASE_LIST points to the list of commands associated with 'prefix-1',
not to the actual 'prefix-1' cmd_list_element.
Back in cmdpy_init, from where gdbpy_parse_command_name was called, I
was walking back from the first entry in BASE_LIST to figure out if
this was a "show" prefix command or not. However, if BASE_LIST is
empty then there is no first item, and this would trigger the
segfault.
The solution it to extend gdbpy_parse_command_name to also return the
prefix cmd_list_element in addition to the existing values. With this
done, and cmdpy_init updated, the segfault is now avoided.
There's a new test that would trigger the crash without the patch.
And, of course, the above commit also broke guile in the exact same
way. And the fix is exactly the same. And there's a guile test too.
NOTE: We should investigate possibly sharing some of this boiler plate
helper code between Python and Guile. But not in this commit.
Approved-By: Tom Tromey <tom@tromey.com>
I believe we previously agreed that the minimum supported Python
version should be 3.4. This patch makes this change, harmonizing the
documentation (which was inconsistent about the minimum version) and
the code.
New in v2: rebased, and removed a pre-3.4 workaround from __init__.py.
Reviewed-By: Eli Zaretskii <eliz@gnu.org>
Approved-by: Kevin Buettner <kevinb@redhat.com>
Acked-By: Tom de Vries <tdevries@suse.de>
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31870
This updates the copyright headers to include 2025. I did this by
running gdb/copyright.py and then manually modifying a few files as
noted by the script.
Approved-By: Eli Zaretskii <eliz@gnu.org>
Since commit a691853148 ("gdb/python: introduce gdbpy_registry"), when
building gdb with gcc 9, I run into:
...
In file included from gdb/varobj.c:38:0:
gdb/python/python-internal.h:1211:47: error: expected ‘;’ before ‘<’ token
using StorageKey = typename registry<O>::key<Storage>;
^
...
due to this code:
...
template <typename Storage>
class gdbpy_registry
{
...
template<typename O>
using StorageKey = typename registry<O>::key<Storage>;
template<typename O>
Storage *get_storage (O *owner, const StorageKey<O> &key) const
{ ... }
...
}
...
As an experiment, I tried out eliminating the type alias:
...
template<typename O>
Storage *get_storage (O *owner,
const typename registry<O>::key<Storage> &key) const
{ ... }
...
and got instead:
...
In file included from gdb/varobj.c:38:0:
gdb/python/python-internal.h:1211:63: error: non-template ‘key’ used as template
Storage *get_storage (O *owner,
const typename registry<O>::key<Storage> &key) const
^~~
gdb/python/python-internal.h:1211:63: note: use ‘registry<O>::template key’ \
to indicate that it is a template
...
Following that suggestion, I tried:
...
template<typename O>
Storage *
get_storage (O *owner,
const typename registry<O>::template key<Storage> &key) const
{ ... }
...
which fixed the problem.
Likewise, adding the template keyword in the type alias fixes the original
problem, so fix it like that.
Tested on x86_64-linux.
This commit introduces new template class gdbpy_registry to simplify
Python object lifecycle management. As of now, each of the Python
object implementations contain its own (copy of) lifecycle management
code that is largely very similar. The aim of gdbpy_registry is to
factor out this code into a common (template) class in order to simplify
the code.
Approved-By: Tom Tromey <tom@tromey.com>
This patch is the result of running check-include-guards.py on the
current tree. Running it a second time causes no changes.
Reviewed-By: Tom de Vries <tdevries@suse.de>
This converts the type copying code to use the new hash map.
Change-Id: I35f0a4946dcc5c5eb84820126cf716b600f3302f
Co-Authored-By: Tom Tromey <tom@tromey.com>
Approved-By: Tom Tromey <tom@tromey.com>
Result of:
...
$ search="GDB_PY_SET_HANDLE_EXCEPTION ("
$ replace="return gdbpy_handle_gdb_exception (-1, "
$ sed -i \
"s/$search/$replace/" \
gdb/python/*.c
...
Also remove the now unused GDB_PY_SET_HANDLE_EXCEPTION.
No functional changes.
Tested on x86_64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
Result of:
...
$ search="GDB_PY_HANDLE_EXCEPTION ("
$ replace="return gdbpy_handle_gdb_exception (nullptr, "
$ sed -i \
"s/$search/$replace/" \
gdb/python/*.c
...
Also remove the now unused GDB_PY_HANDLE_EXCEPTION.
No functional changes.
Tested on x86_64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
I've recently committed two patches:
- commit 2f8cd40c37 ("[gdb/python] Use GDB_PY_HANDLE_EXCEPTION more often")
- commit fbf8e4c35c ("[gdb/python] Use GDB_PY_SET_HANDLE_EXCEPTION more often")
which use the macros GDB_PY_HANDLE_EXCEPTION and GDB_PY_SET_HANDLE_EXCEPTION
more often, with the goal of making things more consistent.
Having done that, I wondered if a better approach could be possible.
Consider GDB_PY_HANDLE_EXCEPTION:
...
/* Use this in a 'catch' block to convert the exception to a Python
exception and return nullptr. */
#define GDB_PY_HANDLE_EXCEPTION(Exception) \
do { \
gdbpy_convert_exception (Exception); \
return nullptr; \
} while (0)
...
The macro nicely codifies how python handles exceptions:
- setting an error condition using some PyErr_Set* variant, and
- returning a value implying that something went wrong
presumably with the goal that using the macro will mean not accidentally:
- forgetting to return on error, or
- returning the wrong value on error.
The problems are that:
- the macro hides control flow, specifically the return statement, and
- the macro hides the return value.
For example, when reading somewhere:
...
catch (const gdb_exception &except)
{
GDB_PY_HANDLE_EXCEPTION (except);
}
...
in order to understand what this does, you have to know that the macro
returns, and that it returns nullptr.
Add a template gdbpy_handle_gdb_exception:
...
template<typename T>
[[nodiscard]] T
gdbpy_handle_gdb_exception (T val, const gdb_exception &e)
{
gdbpy_convert_exception (e);
return val;
}
...
which can be used instead:
...
catch (const gdb_exception &except)
{
return gdbpy_handle_gdb_exception (nullptr, except);
}
...
[ Initially I tried this:
...
template<auto val>
[[nodiscard]] auto
gdbpy_handle_gdb_exception (const gdb_exception &e)
{
gdbpy_convert_exception (e);
return val;
}
...
with which the usage is slightly better looking:
...
catch (const gdb_exception &except)
{
return gdbpy_handle_gdb_exception<nullptr> (except);
}
...
but I ran into trouble with older gcc compilers. ]
While still a single statement, we now have it clear:
- that the statement returns,
- what value the statement returns.
[ FWIW, this could also be handled by say:
...
- GDB_PY_HANDLE_EXCEPTION (except);
+ GDB_PY_HANDLE_EXCEPTION_AND_RETURN_VAL (except, nullptr);
...
but I still didn't find the fact that it returns easy to spot.
Alternatively, this is the simplest form we could use:
...
return gdbpy_convert_exception (e), nullptr;
...
but the pairing would not necessarily survive a copy/paste/edit cycle. ]
Also note how making the value explicit makes it easier to check for
consistency:
...
catch (const gdb_exception &except)
{
return gdbpy_handle_gdb_exception (-1, except);
}
if (PyErr_Occurred ())
return -1;
...
given that we do use the explicit constants almost everywhere else.
Compared to using GDB_PY_HANDLE_EXCEPTION, there is the burden now to specify
the return value, but I assume that this will be generally copy-pasted and
therefore present no problem.
Also, there's no longer a guarantee that there's an immediate return, but I
assume that nodiscard making sure that the return value is not silently
ignored is sufficient mitigation.
For now, re-implement GDB_PY_HANDLE_EXCEPTION and GDB_PY_SET_HANDLE_EXCEPTION
in terms of gdbpy_handle_gdb_exception.
Follow-up patches will eliminate the macros.
No functional changes.
Tested on x86_64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
PR python/32163 points out that various types provided by gdb are not
added to the gdb module, so they aren't available for interactive
inspection. I think this is just an oversight.
This patch fixes the problem by introducing a new helper function that
both readies the type and then adds it to the appropriate module. The
patch also poisons PyType_Ready, the idea being to avoid this bug in
the future.
v2:
* Fixed a bug in original patch in gdb.Architecture registration
* Added regression test for the types mentioned in the bug
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32163
Reviewed-By: Alexandra Petlanova Hajkova <ahajkova@redhat.com>
By default GDB will be printing the hex payload of the ptwrite package as
auxiliary information. To customize this, the user can register a ptwrite
filter function in python, that takes the payload and the PC as arguments and
returns a string which will be printed instead. Registering the filter
function is done using a factory pattern to make per-thread filtering easier.
Approved-By: Markus Metzger <markus.t.metzger@intel.com>
C++ 11 has a built-in attribute for this, no need to use a compat macro.
Change-Id: I90e4220d26e8f3949d91761f8a13cd9c37da3875
Reviewed-by: Lancelot Six <lancelot.six@amd.com>
In commit 764af87825 ("[gdb/python] Add typesafe wrapper around
PyObject_CallMethod") I added poisoning of PyObject_CallMethod:
...
/* Poison PyObject_CallMethod. The typesafe wrapper gdbpy_call_method should be
used instead. */
template<typename... Args>
PyObject *
PyObject_CallMethod (Args...);
...
The idea was that subsequent code would be forced to use gdbpy_call_method
instead of PyObject_CallMethod.
However, that caused build issues with gcc 14 and python 3.13:
...
/usr/bin/ld: python/py-disasm.o: in function `gdb::ref_ptr<_object, gdbpy_ref_policy<_object> > gdbpy_call_method<unsigned int, long long>(_object*, char const*, unsigned int, long long)':
/data/vries/gdb/src/gdb/python/python-internal.h:207:(.text+0x384f): undefined reference to `_object* PyObject_CallMethod<_object*, char*, char*, unsigned int, long long>(_object*, char*, char*, unsigned int, long long)'
/usr/bin/ld: python/py-tui.o: in function `gdb::ref_ptr<_object, gdbpy_ref_policy<_object> > gdbpy_call_method<int>(_object*, char const*, int)':
/data/vries/gdb/src/gdb/python/python-internal.h:207:(.text+0x1235): undefined reference to `_object* PyObject_CallMethod<_object*, char*, char*, int>(_object*, char*, char*, int)'
/usr/bin/ld: python/py-tui.o: in function `gdb::ref_ptr<_object, gdbpy_ref_policy<_object> > gdbpy_call_method<int, int, int>(_object*, char const*, int, int, int)':
/data/vries/gdb/src/gdb/python/python-internal.h:207:(.text+0x12b0): undefined reference to `_object* PyObject_CallMethod<_object*, char*, char*, int, int, int>(_object*, char*, char*, int, int, int)'
collect2: error: ld returned 1 exit status
...
Fix this by poisoning without using templates.
Tested on x86_64-linux.
The following recent change introduced a regression when building using
clang++:
commit 764af87825
Date: Wed Jun 12 18:58:49 2024 +0200
[gdb/python] Add typesafe wrapper around PyObject_CallMethod
The error message is:
../../gdb/python/python-internal.h:151:16: error: default initialization of an object of const type 'const char'
constexpr char gdbpy_method_format;
^
= '\0'
CXX python/py-block.o
1 error generated.
make[2]: *** [Makefile:1959: python/py-arch.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from ../../gdb/python/py-auto-load.c:25:
../../gdb/python/python-internal.h:151:16: error: default initialization of an object of const type 'const char'
constexpr char gdbpy_method_format;
^
= '\0'
1 error generated.
make[2]: *** [Makefile:1959: python/py-auto-load.o] Error 1
In file included from ../../gdb/python/py-block.c:23:
../../gdb/python/python-internal.h:151:16: error: default initialization of an object of const type 'const char'
constexpr char gdbpy_method_format;
^
= '\0'
1 error generated.
This patch fixes this by changing gdbpy_method_format to be a templated
struct, and only have its specializations define the static constexpr
member "format". This way, we avoid having an uninitialized constexpr
expression, regardless of it being instantiated or not.
Reviewed-By: Tom de Vries <tdevries@suse.de>
Change-Id: I5bec241144f13500ef78daea30f00d01e373692d
This changes gdbpy_call_method to return a gdbpy_ref<>. This is
slightly safer because it makes it simpler to correctly handle
reference counts.
Reviewed-By: Tom de Vries <tdevries@suse.de>
In gdb/python/py-tui.c we have code like this:
...
gdbpy_ref<> result (PyObject_CallMethod (m_window.get(), "hscroll",
"i", num_to_scroll, nullptr));
...
The nullptr is superfluous, the format string already indicates that there's
only one method argument.
OTOH, passing no method args does use a nullptr:
...
gdbpy_ref<> result (PyObject_CallMethod (m_window.get (), "render",
nullptr));
...
Furthermore, choosing the right format string chars can be tricky.
Add a typesafe wrapper around PyObject_CallMethod that hides these
details, such that we can use the more intuitive:
...
gdbpy_ref<> result (gdbpy_call_method (m_window.get(), "hscroll",
num_to_scroll));
...
and:
...
gdbpy_ref<> result (gdbpy_call_method (m_window.get (), "render"));
...
Tested on x86_64-linux.
Co-Authored-By: Tom de Vries <tdevries@suse.de>
Approved-By: Tom Tromey <tom@tromey.com>
If in gdb/python/python-internal.h, we pretend to have a platform that doesn't
support long long:
...
-#ifdef HAVE_LONG_LONG
+#if 0
...
I get on arm-linux:
...
(gdb) placement_candidate()
disassemble test^M
Dump of assembler code for function test:^M
0x004004d8 <+0>: push {r11} @ (str r11, [sp, #-4]!)^M
0x004004dc <+4>: Python Exception <class 'ValueError'>: \
Buffer returned from read_memory is sized 0 instead of the expected 4^M
^M
unknown disassembler error (error = -1)^M
(gdb) FAIL: $exp: memory source api: second disassembler pass
...
The problem is that gdb_py_longest is typedef-ed to long, but the
corresponding format character GDB_PY_LL_ARG is defined to "L", meaning
"long long" [1].
Fix this by using "l", meaning long instead. Likewise for GDB_PY_LLU_ARG.
Tested on arm-linux.
Approved-By: Tom Tromey <tom@tromey.com>
PR python/31845
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31845
[1] https://docs.python.org/3/c-api/arg.html
Starting with python 3.6, support for platforms without long long support
has been removed [1].
HAVE_LONG_LONG and PY_LONG_LONG are still defined, but only for compatibility,
so stop relying on them.
Tested on x86_64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
[1] https://github.com/python/cpython/issues/72148
This removes the embedded 'if' from GDB_PY_HANDLE_EXCEPTION and
GDB_PY_SET_HANDLE_EXCEPTION. I believe this 'if' was necessary with
the old gdb try/catch macros, but it no longer is: these should only
ever be called from a 'catch' block, where it's already known that an
exception was thrown.
Simon pointed out, though, that in a few spots, these were in facts
called outside of 'catch' blocks. This patch cleans up these spots.
I also found one spot where a redundant 'return nullptr' could be
removed.
Starting python version 3.12, PyErr_Fetch and PyErr_Restore are deprecated.
Use PyErr_GetRaisedException and PyErr_SetRaisedException instead, for
python >= 3.12.
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
With python 3.12, I run into:
...
(gdb) PASS: gdb.python/py-block.exp: check variable access
python print (block['nonexistent'])^M
Python Exception <class 'KeyError'>: 'nonexistent'^M
Error occurred in Python: 'nonexistent'^M
(gdb) FAIL: gdb.python/py-block.exp: check nonexistent variable
...
The problem is that that PyErr_Fetch returns a normalized exception, while the
test-case matches the output for an unnormalized exception.
With python 3.6, PyErr_Fetch returns an unnormalized exception, and the
test passes.
Fix this by:
- updating the test-case to match the output for a normalized exception, and
- lazily forcing normalized exceptions using PyErr_NormalizeException.
Tested on aarch64-linux.
Approved-By: Tom Tromey <tom@tromey.com>
Similar to gdbpy_err_fetch::value, add a getter gdbpy_err_fetch::type, and use
both consistently to get gdbpy_err_fetch members m_error_value and
m_error_type.
Tested on aarch64-linux.
We currently pass frames to function by value, as `frame_info_ptr`.
This is somewhat expensive:
- the size of `frame_info_ptr` is 64 bytes, which is a bit big to pass
by value
- the constructors and destructor link/unlink the object in the global
`frame_info_ptr::frame_list` list. This is an `intrusive_list`, so
it's not so bad: it's just assigning a few points, there's no memory
allocation as if it was `std::list`, but still it's useless to do
that over and over.
As suggested by Tom Tromey, change many function signatures to accept
`const frame_info_ptr &` instead of `frame_info_ptr`.
Some functions reassign their `frame_info_ptr` parameter, like:
void
the_func (frame_info_ptr frame)
{
for (; frame != nullptr; frame = get_prev_frame (frame))
{
...
}
}
I wondered what to do about them, do I leave them as-is or change them
(and need to introduce a separate local variable that can be
re-assigned). I opted for the later for consistency. It might not be
clear why some functions take `const frame_info_ptr &` while others take
`frame_info_ptr`. Also, if a function took a `frame_info_ptr` because
it did re-assign its parameter, I doubt that we would think to change it
to `const frame_info_ptr &` should the implementation change such that
it doesn't need to take `frame_info_ptr` anymore. It seems better to
have a simple rule and apply it everywhere.
Change-Id: I59d10addef687d157f82ccf4d54f5dde9a963fd0
Approved-By: Andrew Burgess <aburgess@redhat.com>
This commit is the result of the following actions:
- Running gdb/copyright.py to update all of the copyright headers to
include 2024,
- Manually updating a few files the copyright.py script told me to
update, these files had copyright headers embedded within the
file,
- Regenerating gdbsupport/Makefile.in to refresh it's copyright
date,
- Using grep to find other files that still mentioned 2023. If
these files were updated last year from 2022 to 2023 then I've
updated them this year to 2024.
I'm sure I've probably missed some dates. Feel free to fix them up as
you spot them.
The gdb.Objfile, gdb.Progspace, gdb.Type, and gdb.Inferior Python
types already have a __dict__ attribute, which allows users to create
user defined attributes within the objects. This is useful if the
user wants to cache information within an object.
This commit adds the same functionality to the gdb.InferiorThread
type.
After this commit there is a new gdb.InferiorThread.__dict__
attribute, which is a dictionary. A user can, for example, do this:
(gdb) pi
>>> t = gdb.selected_thread()
>>> t._user_attribute = 123
>>> t._user_attribute
123
>>>
There's a new test included.
Reviewed-By: Eli Zaretskii <eliz@gnu.org>
Approved-By: Tom Tromey <tom@tromey.com>
Many object types now have a __repr__() function implementation. A
common pattern is that, if an object is invalid, we print its
representation as: <TYPENAME (invalid)>.
I thought it might be a good idea to move the formatting of this
specific representation into a utility function, and then update all
of our existing code to call the new function.
The only place where I haven't made use of the new function is in
unwind_infopy_repr, where we currently return a different string.
This case is a little different as the UnwindInfo is invalid because
it references a frame, and it is the frame itself which is invalid.
That said, I think it would be fine to switch to using the standard
format; if the UnwindInfo references an invalid frame, then the
UnwindInfo is itself invalid. But changing this would be an actual
change in behaviour, while all the other changes in this commit are
just refactoring.
Approved-By: Tom Tromey <tom@tromey.com>
Move gdbpy_gil class into python-internal.h, the next
commit wants to make use of this class from a file other
than python.c.
Approved-By: Tom Tromey <tom@tromey.com>
Since GDB now requires C++17, we don't need the internally maintained
gdb::optional implementation. This patch does the following replacing:
- gdb::optional -> std::optional
- gdb::in_place -> std::in_place
- #include "gdbsupport/gdb_optional.h" -> #include <optional>
This change has mostly been done automatically. One exception is
gdbsupport/thread-pool.* which did not use the gdb:: prefix as it
already lives in the gdb namespace.
Change-Id: I19a92fa03e89637bab136c72e34fd351524f65e9
Approved-By: Tom Tromey <tom@tromey.com>
Approved-By: Pedro Alves <pedro@palves.net>
This commit adds a new Python function, gdb.notify_mi, that can be used
to emit custom async notification to MI channel. This can be used, among
other things, to implement notifications about events MI does not support,
such as remote connection closed or register change.
Reviewed-By: Eli Zaretskii <eliz@gnu.org>
Approved-By: Andrew Burgess <aburgess@redhat.com>
This commit generalizes serialize_mi_result() to make usable in
different contexts than printing result of custom MI command.
To do so, the check whether passed Python object is a dictionary has been
moved to the caller - at the very least, different uses require different
error messages. Also it has been renamed to serialize_mi_results() to better
match GDB/MI output syntax (see corresponding section in documentation,
in particular rules 'result-record' and 'async-output'.
Since it is now more generic function, it has been moved to py-mi.c.
This is a preparation for implementing Python support for sending custom
MI async events.
Approved-By: Andrew Burgess <aburgess@redhat.com>
This adds a new Python function, gdb.execute_mi, that can be used to
invoke an MI command but get the output as a Python object, rather
than a string. This is done by implementing a new ui_out subclass
that builds a Python object.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=11688
Reviewed-By: Eli Zaretskii <eliz@gnu.org>
If an MI command written in Python includes a number in its output,
currently that is simply emitted as a string. However, it's
convenient for a later patch if these are emitted using field_signed.
This does not make a difference to ordinary MI clients.
Currently, when we add a new python sub-system to GDB,
e.g. py-inferior.c, we end up having to create a new function like
gdbpy_initialize_inferior, which then has to be called from the
function do_start_initialization in python.c.
In some cases (py-micmd.c and py-tui.c), we have two functions
gdbpy_initialize_*, and gdbpy_finalize_*, with the second being called
from finalize_python which is also in python.c.
This commit proposes a mechanism to manage these initialization and
finalization calls, this means that adding a new Python subsystem will
no longer require changes to python.c or python-internal.h, instead,
the initialization and finalization functions will be registered
directly from the sub-system file, e.g. py-inferior.c, or py-micmd.c.
The initialization and finalization functions are managed through a
new class gdbpy_initialize_file in python-internal.h. This class
contains a single global vector of all the initialization and
finalization functions.
In each Python sub-system we create a new gdbpy_initialize_file
object, the object constructor takes care of registering the two
callback functions.
Now from python.c we can call static functions on the
gdbpy_initialize_file class which take care of walking the callback
list and invoking each callback in turn.
To slightly simplify the Python sub-system files I added a new macro
GDBPY_INITIALIZE_FILE, which hides the need to create an object. We
can now just do this:
GDBPY_INITIALIZE_FILE (gdbpy_initialize_registers);
One possible problem with this change is that there is now no
guaranteed ordering of how the various sub-systems are initialized (or
finalized). To try and avoid dependencies creeping in I have added a
use of the environment variable GDB_REVERSE_INIT_FUNCTIONS, this is
the same environment variable used in the generated init.c file.
Just like with init.c, when this environment variable is set we
reverse the list of Python initialization (and finalization)
functions. As there is already a test that starts GDB with the
environment variable set then this should offer some level of
protection against dependencies creeping in - though for full
protection I guess we'd need to run all gdb.python/*.exp tests with
the variable set.
I have tested this patch with the environment variable set, and saw no
regressions, so I think we are fine right now.
One other change of note was for gdbpy_initialize_gdb_readline, this
function previously returned void. In order to make this function
have the correct signature I've updated its return type to int, and we
now return 0 to indicate success.
All of the other initialize (and finalize) functions have been made
static within their respective sub-system files.
There should be no user visible changes after this commit.
Background
----------
When a thread-specific breakpoint is deleted as a result of the
specific thread exiting the function remove_threaded_breakpoints is
called which sets the disposition of the breakpoint to
disp_del_at_next_stop and sets the breakpoint number to 0. Setting
the breakpoint number to zero has the effect of hiding the breakpoint
from the user. We also print a message indicating that the breakpoint
has been deleted.
It was brought to my attention during a review of another patch[1]
that setting a breakpoints number to zero will suppress the MI
breakpoint-deleted notification for that breakpoint, and indeed, this
can be seen to be true, in delete_breakpoint, if the breakpoint number
is zero, then GDB will not notify the breakpoint_deleted observer.
It seems wrong that a user created, thread-specific breakpoint, will
have a =breakpoint-created notification, but will not have a
=breakpoint-deleted notification. I suspect that this is a bug.
[1] https://sourceware.org/pipermail/gdb-patches/2023-February/196560.html
The First Problem
-----------------
During my initial testing I wanted to see how GDB handled the
breakpoint after it's number was set to zero. To do this I created
the testcase gdb.threads/thread-bp-deleted.exp. This test creates a
worker thread, which immediately exits. After the worker thread has
exited the main thread spins in a loop.
In GDB I break once the worker thread has been created and place a
thread-specific breakpoint, then use 'continue&' to resume the
inferior in non-stop mode. The worker thread then exits, but the main
thread never stops - instead it sits in the spin. I then tried to use
'maint info breakpoints' to see what GDB thought of the
thread-specific breakpoint.
Unfortunately, GDB crashed like this:
(gdb) continue&
Continuing.
(gdb) [Thread 0x7ffff7c5d700 (LWP 1202458) exited]
Thread-specific breakpoint 3 deleted - thread 2 no longer in the thread list.
maint info breakpoints
... snip some output ...
Fatal signal: Segmentation fault
----- Backtrace -----
0x5ffb62 gdb_internal_backtrace_1
../../src/gdb/bt-utils.c:122
0x5ffc05 _Z22gdb_internal_backtracev
../../src/gdb/bt-utils.c:168
0x89965e handle_fatal_signal
../../src/gdb/event-top.c:964
0x8997ca handle_sigsegv
../../src/gdb/event-top.c:1037
0x7f96f5971b1f ???
/usr/src/debug/glibc-2.30-2-gd74461fa34/nptl/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
0xe602b0 _Z15print_thread_idP11thread_info
../../src/gdb/thread.c:1439
0x5b3d05 print_one_breakpoint_location
../../src/gdb/breakpoint.c:6542
0x5b462e print_one_breakpoint
../../src/gdb/breakpoint.c:6702
0x5b5354 breakpoint_1
../../src/gdb/breakpoint.c:6924
0x5b58b8 maintenance_info_breakpoints
../../src/gdb/breakpoint.c:7009
... etc ...
As the thread-specific breakpoint is set to disp_del_at_next_stop, and
GDB hasn't stopped yet, then the breakpoint still exists in the global
breakpoint list.
The breakpoint will not show in 'info breakpoints' as its number is
zero, but it will show in 'maint info breakpoints'.
As GDB prints the breakpoint, the thread-id for the breakpoint is
printed as part of the 'stop only in thread ...' line. Printing the
thread-id involves calling find_thread_global_id to convert the global
thread-id into a thread_info*. Then calling print_thread_id to
convert the thread_info* into a string.
The problem is that find_thread_global_id returns nullptr as the
thread for the thread-specific breakpoint has exited. The
print_thread_id assumes it will be passed a non-nullptr. As a result
GDB crashes.
In this commit I've added an assert to print_thread_id (gdb/thread.c)
to check that the pointed passed in is not nullptr. This assert would
have triggered in the above case before GDB crashed.
MI Notifications: The Dangers Of Changing A Breakpoint's Number
---------------------------------------------------------------
Currently the delete_breakpoint function doesn't trigger the
breakpoint_deleted observer for any breakpoint with the number zero.
There is a comment explaining why this is the case in the code; it's
something about watchpoints. But I did consider just removing the 'is
the number zero' guard and always triggering the breakpoint_deleted
observer, figuring that I'd then fix the watchpoint issue some other
way.
But I realised this wasn't going to be good enough. When the MI
notification was delivered the number would be zero, so any frontend
parsing the notifications would not be able to match
=breakpoint-deleted notification to the earlier =breakpoint-created
notification.
What this means is that, at the point the breakpoint_deleted observer
is called, the breakpoint's number must be correct.
MI Notifications: The Dangers Of Delaying Deletion
--------------------------------------------------
The test I used to expose the above crash also brought another problem
to my attention. In the above test we used 'continue&' to resume,
after which a thread exited, but the inferior didn't stop. Recreating
the same test in the MI looks like this:
-break-insert -p 2 main
^done,bkpt={number="2",type="breakpoint",disp="keep",...<snip>...}
(gdb)
-exec-continue
^running
*running,thread-id="all"
(gdb)
~"[Thread 0x7ffff7c5d700 (LWP 987038) exited]\n"
=thread-exited,id="2",group-id="i1"
~"Thread-specific breakpoint 2 deleted - thread 2 no longer in the thread list.\n"
At this point the we have a single thread left, which is still
running:
-thread-info
^done,threads=[{id="1",target-id="Thread 0x7ffff7c5eb80 (LWP 987035)",name="thread-bp-delet",state="running",core="4"}],current-thread-id="1"
(gdb)
Notice that we got the =thread-exited notification from GDB as soon as
the thread exited. We also saw the CLI line from GDB, the line
explaining that breakpoint 2 was deleted. But, as expected, we didn't
see the =breakpoint-deleted notification.
I say "as expected" because the number was set to zero. But, even if
the number was not set to zero we still wouldn't see the
notification. The MI notification is driven by the breakpoint_deleted
observer, which is only called when we actually delete the breakpoint,
which is only done the next time GDB stops.
Now, maybe this is fine. The notification is delivered a little
late. But remember, by setting the number to zero the breakpoint will
be hidden from the user, for example, the breakpoint is removed from
the MI's -break-info command output.
This means that GDB is in a position where the breakpoint doesn't show
up in the breakpoint table, but a =breakpoint-deleted notification has
not yet been sent out. This doesn't seem right to me.
What this means is that, when the thread exits, we should immediately
be sending out the =breakpoint-deleted notification. We should not
wait for GDB to next stop before sending the notification.
The Solution
------------
My proposed solution is this; in remove_threaded_breakpoints, instead
of setting the disposition to disp_del_at_next_stop and setting the
number to zero, we now just call delete_breakpoint directly.
The notification will now be sent out immediately; as soon as the
thread exits.
As the number has not changed when delete_breakpoint is called, the
notification will have the correct number.
And as the breakpoint is immediately removed from the breakpoint list,
we no longer need to worry about 'maint info breakpoints' trying to
print the thread-id for an exited thread.
My only concern is that calling delete_breakpoint directly seems so
obvious that I wonder why the original patch (that added
remove_threaded_breakpoints) didn't take this approach. This code was
added in commit 49fa26b041, but the commit message offers no clues
to why this approach was taken, and the original email thread offers
no insights either[2]. There are no test regressions after making
this change, so I'm hopeful that this is going to be fine.
[2] https://sourceware.org/pipermail/gdb-patches/2013-September/106493.html
The Complication
----------------
Of course, it couldn't be that simple.
The script gdb.python/py-finish-breakpoint.exp had some regressions
during testing.
The problem was with the FinishBreakpoint.out_of_scope callback
implementation. This callback is supposed to trigger whenever the
FinishBreakpoint goes out of scope; and this includes when the thread
for the breakpoint exits.
The problem I ran into is the Python FinishBreakpoint implementation.
Specifically, after this change I was loosing some of the out_of_scope
calls.
The problem is that the out_of_scope call (of which I'm interested) is
triggered from the inferior_exit observer. Before my change the
observers were called in this order:
thread_exit
inferior_exit
breakpoint_deleted
The inferior_exit would trigger the out_of_scope call.
After my change the breakpoint_deleted notification (for
thread-specific breakpoints) occurs earlier, as soon as the
thread-exits, so now the order is:
thread_exit
breakpoint_deleted
inferior_exit
Currently, after the breakpoint_deleted call the Python object
associated with the breakpoint is released, so, when we get to the
inferior_exit observer, there's no longer a Python object to call the
out_of_scope method on.
My solution is to follow the model for how bpfinishpy_pre_stop_hook
and bpfinishpy_post_stop_hook are called, this is done from
gdbpy_breakpoint_cond_says_stop in py-breakpoint.c.
I've now added a new bpfinishpy_pre_delete_hook
gdbpy_breakpoint_deleted in py-breakpoint.c, and from this new hook
function I check and where needed call the out_of_scope method.
With this fix in place I now see the
gdb.python/py-finish-breakpoint.exp test fully passing again.
Testing
-------
Tested on x86-64/Linux with unix, native-gdbserver, and
native-extended-gdbserver boards.
New tests added to covers all the cases I've discussed above.
Approved-By: Pedro Alves <pedro@palves.net>
Hannes filed a bug showing a crash, where a pretty-printer written in
Python could cause a use-after-free. He sent a patch, but I thought a
different approach was needed.
In a much earlier patch (see bug #12533), we changed the Python code
to release new values from the value chain when constructing a
gdb.Value. The rationale for this is that if you write a command that
does a lot of computations in a loop, all the values will be kept live
by the value chain, resulting in gdb using a large amount of memory.
However, suppose a value is passed to Python from some code in gdb
that needs to use the value after the call into Python. In this
scenario, value_to_value_object will still release the value -- and
because gdb code doesn't generally keep strong references to values (a
consequence of the ancient decision to use the value chain to avoid
memory management), this will result in a use-after-free.
This scenario can happen, as it turns out, when a value is passed to
Python for pretty-printing. Now, normally this route boxes the value
via value_to_value_object_no_release, avoiding the problematic release
from the value chain. However, if you then call Value.cast, the
underlying value API might return the same value, when is then
released from the chain.
This patch fixes the problem by changing how value boxing is done.
value_to_value_object no longer removes a value from the chain.
Instead, every spot in gdb that might construct new values uses a
scoped_value_mark to ensure that the requirements of bug #12533 are
met. And, because incoming values aren't ever released from the chain
(the Value.cast one comes earlier on the chain than the
scoped_value_mark), the bug can no longer occur. (Note that many
spots in the Python layer already take this approach, so not many
places needed to be touched.)
In the future I think we should replace the use of raw "value *" with
value_ref_ptr pretty much everywhere. This will ensure lifetime
safety throughout gdb.
The test case in this patch comes from Hannes' original patch. I only
made a trivial ("require") change to it. However, while this fails
for him, I can't make it fail on this machine; nevertheless, he tried
my patch and reported the bug as being fixed.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30044
