gdb: implement missing debug handler hook for Python

This commit builds on the previous commit, and implements the
extension_language_ops::handle_missing_debuginfo function for Python.
This hook will give user supplied Python code a chance to help find
missing debug information.

The implementation of the new hook is pretty minimal within GDB's C++
code; most of the work is out-sourced to a Python implementation which
is modelled heavily on how GDB's Python frame unwinders are
implemented.

The following new commands are added as commands implemented in
Python, this is similar to how the Python unwinder commands are
implemented:

  info missing-debug-handlers
  enable missing-debug-handler LOCUS HANDLER
  disable missing-debug-handler LOCUS HANDLER

To make use of this extension hook a user will create missing debug
information handler objects, and registers these handlers with GDB.
When GDB encounters an objfile that is missing debug information, each
handler is called in turn until one is able to help.  Here is a
minimal handler that does nothing useful:

  import gdb
  import gdb.missing_debug

  class MyFirstHandler(gdb.missing_debug.MissingDebugHandler):
      def __init__(self):
          super().__init__("my_first_handler")

      def __call__(self, objfile):
          # This handler does nothing useful.
          return None

  gdb.missing_debug.register_handler(None, MyFirstHandler())

Returning None from the __call__ method tells GDB that this handler
was unable to find the missing debug information, and GDB should ask
any other registered handlers.

By extending the __call__ method it is possible for the Python
extension to locate the debug information for objfile and return a
value that tells GDB how to use the information that has been located.

Possible return values from a handler:

  - None: This means the handler couldn't help.  GDB will call other
          registered handlers to see if they can help instead.

  - False: The handler has done all it can, but the debug information
           for the objfile still couldn't be found.  GDB will not call
	   any other handlers, and will continue without the debug
	   information for objfile.

  - True: The handler has installed the debug information into a
          location where GDB would normally expect to find it.  GDB
	  should look again for the debug information.

  - A string: The handler can return a filename, which is the file
              containing the missing debug information.  GDB will load
	      this file.

When a handler returns True, GDB will look again for the debug
information, but only using the standard built-in build-id and
.gnu_debuglink based lookup strategies.  It is not possible for an
extension to trigger another debuginfod lookup; the assumption is that
the debuginfod server is remote, and out of the control of extensions
running within GDB.

Handlers can be registered globally, or per program space.  GDB checks
the handlers for the current program space first, and then all of the
global handles.  The first handler that returns a value that is not
None, has "handled" the objfile, at which point GDB continues.

Reviewed-By: Eli Zaretskii <eliz@gnu.org>
Approved-By: Tom Tromey <tom@tromey.com>

5 files changed, 569 insertions, 1 deletions

diff --git a/gdb/python/lib/gdb/__init__.py b/gdb/python/lib/gdb/__init__.py
index b312436..93ed50e 100644
--- a/gdb/python/lib/gdb/__init__.py
+++ b/gdb/python/lib/gdb/__init__.py
@@ -84,6 +84,8 @@ xmethods = []
 frame_filters = {}
 # Initial frame unwinders.
 frame_unwinders = []
+# Initial missing debug handlers.
+missing_debug_handlers = []
 
 
 def _execute_unwinders(pending_frame):
@@ -291,3 +293,42 @@ class Thread(threading.Thread):
         # threads.
         with blocked_signals():
             super().start()
+
+
+def _handle_missing_debuginfo(objfile):
+    """Internal function called from GDB to execute missing debug
+    handlers.
+
+    Run each of the currently registered, and enabled missing debug
+    handler objects for the current program space and then from the
+    global list.  Stop after the first handler that returns a result
+    other than None.
+
+    Arguments:
+        objfile: A gdb.Objfile for which GDB could not find any debug
+                 information.
+
+    Returns:
+        None: No debug information could be found for objfile.
+        False: A handler has done all it can with objfile, but no
+               debug information could be found.
+        True: Debug information might have been installed by a
+              handler, GDB should check again.
+        A string: This is the filename of a file containing the
+                  required debug information.
+    """
+    pspace = objfile.progspace
+
+    for handler in pspace.missing_debug_handlers:
+        if handler.enabled:
+            result = handler(objfile)
+            if result is not None:
+                return result
+
+    for handler in missing_debug_handlers:
+        if handler.enabled:
+            result = handler(objfile)
+            if result is not None:
+                return result
+
+    return None
diff --git a/gdb/python/lib/gdb/command/missing_debug.py b/gdb/python/lib/gdb/command/missing_debug.py
new file mode 100644
index 0000000..6fe1cdb
--- /dev/null
+++ b/gdb/python/lib/gdb/command/missing_debug.py
@@ -0,0 +1,226 @@
+# Missing debug related commands.
+#
+# Copyright 2023 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import gdb
+import re
+
+
+def validate_regexp(exp, idstring):
+    """Compile exp into a compiler regular expression object.
+
+    Arguments:
+        exp: The string to compile into a re.Pattern object.
+        idstring: A string, what exp is a regexp for.
+
+    Returns:
+        A re.Pattern object representing exp.
+
+    Raises:
+        SyntaxError: If exp is an invalid regexp.
+    """
+    try:
+        return re.compile(exp)
+    except SyntaxError:
+        raise SyntaxError("Invalid %s regexp: %s." % (idstring, exp))
+
+
+def parse_missing_debug_command_args(arg):
+    """Internal utility to parse missing debug handler command argv.
+
+    Arguments:
+        arg: The arguments to the command. The format is:
+             [locus-regexp [name-regexp]]
+
+    Returns:
+        A 2-tuple of compiled regular expressions.
+
+    Raises:
+        SyntaxError: an error processing ARG
+    """
+    argv = gdb.string_to_argv(arg)
+    argc = len(argv)
+    if argc > 2:
+        raise SyntaxError("Too many arguments.")
+    locus_regexp = ""
+    name_regexp = ""
+    if argc >= 1:
+        locus_regexp = argv[0]
+        if argc >= 2:
+            name_regexp = argv[1]
+    return (
+        validate_regexp(locus_regexp, "locus"),
+        validate_regexp(name_regexp, "handler"),
+    )
+
+
+class InfoMissingDebugHanders(gdb.Command):
+    """GDB command to list missing debug handlers.
+
+    Usage: info missing-debug-handlers [LOCUS-REGEXP [NAME-REGEXP]]
+
+    LOCUS-REGEXP is a regular expression matching the location of the
+    handler.  If it is omitted, all registered handlers from all
+    loci are listed.  A locus can be 'global', 'progspace' to list
+    the handlers from the current progspace, or a regular expression
+    matching filenames of progspaces.
+
+    NAME-REGEXP is a regular expression to filter missing debug
+    handler names.  If this omitted for a specified locus, then all
+    registered handlers in the locus are listed.
+    """
+
+    def __init__(self):
+        super().__init__("info missing-debug-handlers", gdb.COMMAND_FILES)
+
+    def list_handlers(self, title, handlers, name_re):
+        """Lists the missing debug handlers whose name matches regexp.
+
+        Arguments:
+            title: The line to print before the list.
+            handlers: The list of the missing debug handlers.
+            name_re: handler name filter.
+        """
+        if not handlers:
+            return
+        print(title)
+        for handler in handlers:
+            if name_re.match(handler.name):
+                print(
+                    "  %s%s" % (handler.name, "" if handler.enabled else " [disabled]")
+                )
+
+    def invoke(self, arg, from_tty):
+        locus_re, name_re = parse_missing_debug_command_args(arg)
+
+        if locus_re.match("progspace") and locus_re.pattern != "":
+            cp = gdb.current_progspace()
+            self.list_handlers(
+                "Progspace %s:" % cp.filename, cp.missing_debug_handlers, name_re
+            )
+
+        for progspace in gdb.progspaces():
+            filename = progspace.filename or ""
+            if locus_re.match(filename):
+                if filename == "":
+                    if progspace == gdb.current_progspace():
+                        msg = "Current Progspace:"
+                    else:
+                        msg = "Progspace <no-file>:"
+                else:
+                    msg = "Progspace %s:" % filename
+                self.list_handlers(
+                    msg,
+                    progspace.missing_debug_handlers,
+                    name_re,
+                )
+
+        # Print global handlers last, as these are invoked last.
+        if locus_re.match("global"):
+            self.list_handlers("Global:", gdb.missing_debug_handlers, name_re)
+
+
+def do_enable_handler1(handlers, name_re, flag):
+    """Enable/disable missing debug handlers whose names match given regex.
+
+    Arguments:
+        handlers: The list of missing debug handlers.
+        name_re: Handler name filter.
+        flag: A boolean indicating if we should enable or disable.
+
+    Returns:
+        The number of handlers affected.
+    """
+    total = 0
+    for handler in handlers:
+        if name_re.match(handler.name) and handler.enabled != flag:
+            handler.enabled = flag
+            total += 1
+    return total
+
+
+def do_enable_handler(arg, flag):
+    """Enable or disable missing debug handlers."""
+    (locus_re, name_re) = parse_missing_debug_command_args(arg)
+    total = 0
+    if locus_re.match("global"):
+        total += do_enable_handler1(gdb.missing_debug_handlers, name_re, flag)
+    if locus_re.match("progspace") and locus_re.pattern != "":
+        total += do_enable_handler1(
+            gdb.current_progspace().missing_debug_handlers, name_re, flag
+        )
+    for progspace in gdb.progspaces():
+        filename = progspace.filename or ""
+        if locus_re.match(filename):
+            total += do_enable_handler1(progspace.missing_debug_handlers, name_re, flag)
+    print(
+        "%d missing debug handler%s %s"
+        % (total, "" if total == 1 else "s", "enabled" if flag else "disabled")
+    )
+
+
+class EnableMissingDebugHandler(gdb.Command):
+    """GDB command to enable missing debug handlers.
+
+    Usage: enable missing-debug-handler [LOCUS-REGEXP [NAME-REGEXP]]
+
+    LOCUS-REGEXP is a regular expression specifying the handlers to
+    enable.  It can be 'global', 'progspace' for the current
+    progspace, or the filename for a file associated with a progspace.
+
+    NAME_REGEXP is a regular expression to filter handler names.  If
+    this omitted for a specified locus, then all registered handlers
+    in the locus are affected.
+    """
+
+    def __init__(self):
+        super().__init__("enable missing-debug-handler", gdb.COMMAND_FILES)
+
+    def invoke(self, arg, from_tty):
+        """GDB calls this to perform the command."""
+        do_enable_handler(arg, True)
+
+
+class DisableMissingDebugHandler(gdb.Command):
+    """GDB command to disable missing debug handlers.
+
+    Usage: disable missing-debug-handler [LOCUS-REGEXP [NAME-REGEXP]]
+
+    LOCUS-REGEXP is a regular expression specifying the handlers to
+    enable.  It can be 'global', 'progspace' for the current
+    progspace, or the filename for a file associated with a progspace.
+
+    NAME_REGEXP is a regular expression to filter handler names.  If
+    this omitted for a specified locus, then all registered handlers
+    in the locus are affected.
+    """
+
+    def __init__(self):
+        super().__init__("disable missing-debug-handler", gdb.COMMAND_FILES)
+
+    def invoke(self, arg, from_tty):
+        """GDB calls this to perform the command."""
+        do_enable_handler(arg, False)
+
+
+def register_missing_debug_handler_commands():
+    """Installs the missing debug handler commands."""
+    InfoMissingDebugHanders()
+    EnableMissingDebugHandler()
+    DisableMissingDebugHandler()
+
+
+register_missing_debug_handler_commands()
diff --git a/gdb/python/lib/gdb/missing_debug.py b/gdb/python/lib/gdb/missing_debug.py
new file mode 100644
index 0000000..42d6985
--- /dev/null
+++ b/gdb/python/lib/gdb/missing_debug.py
@@ -0,0 +1,169 @@
+# Copyright (C) 2023 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+MissingDebugHandler base class, and register_handler function.
+"""
+
+import gdb
+
+
+def _validate_name(name):
+    """Validate a missing debug handler name string.
+
+    If name is valid as a missing debug handler name, then this
+    function does nothing.  If name is not valid then an exception is
+    raised.
+
+    Arguments:
+        name: A string, the name of a missing debug handler.
+
+    Returns:
+        Nothing.
+
+    Raises:
+        ValueError: If name is invalid as a missing debug handler
+                    name.
+    """
+    for ch in name:
+        if not ch.isascii() or not (ch.isalnum() or ch in "_-"):
+            raise ValueError("invalid character '%s' in handler name: %s" % (ch, name))
+
+
+class MissingDebugHandler(object):
+    """Base class for missing debug handlers written in Python.
+
+    A missing debug handler has a single method __call__ along with
+    the read/write attribute enabled, and a read-only attribute name.
+
+    Attributes:
+        name: Read-only attribute, the name of this handler.
+        enabled: When true this handler is enabled.
+    """
+
+    def __init__(self, name):
+        """Constructor.
+
+        Args:
+            name: An identifying name for this handler.
+
+        Raises:
+            TypeError: name is not a string.
+            ValueError: name contains invalid characters.
+        """
+
+        if not isinstance(name, str):
+            raise TypeError("incorrect type for name: %s" % type(name))
+
+        _validate_name(name)
+
+        self._name = name
+        self._enabled = True
+
+    @property
+    def name(self):
+        return self._name
+
+    @property
+    def enabled(self):
+        return self._enabled
+
+    @enabled.setter
+    def enabled(self, value):
+        if not isinstance(value, bool):
+            raise TypeError("incorrect type for enabled attribute: %s" % type(value))
+        self._enabled = value
+
+    def __call__(self, objfile):
+        """GDB handle missing debug information for an objfile.
+
+        Arguments:
+            objfile: A gdb.Objfile for which GDB could not find any
+                debug information.
+
+        Returns:
+            True: GDB should try again to locate the debug information
+                for objfile, the handler may have installed the
+                missing information.
+            False: GDB should move on without the debug information
+                for objfile.
+            A string: GDB should load the file at the given path; it
+                contains the debug information for objfile.
+            None: This handler can't help with objfile.  GDB should
+                try any other registered handlers.
+        """
+        raise NotImplementedError("MissingDebugHandler.__call__()")
+
+
+def register_handler(locus, handler, replace=False):
+    """Register handler in given locus.
+
+    The handler is prepended to the locus's missing debug handlers
+    list. The name of handler should be unique (or replace must be
+    True).
+
+    Arguments:
+        locus: Either a progspace, or None (in which case the unwinder
+               is registered globally).
+        handler: An object of a gdb.MissingDebugHandler subclass.
+
+        replace: If True, replaces existing handler with the same name
+                 within locus.  Otherwise, raises RuntimeException if
+                 unwinder with the same name already exists.
+
+    Returns:
+        Nothing.
+
+    Raises:
+        RuntimeError: The name of handler is not unique.
+        TypeError: Bad locus type.
+        AttributeError: Required attributes of handler are missing.
+    """
+
+    if locus is None:
+        if gdb.parameter("verbose"):
+            gdb.write("Registering global %s handler ...\n" % handler.name)
+        locus = gdb
+    elif isinstance(locus, gdb.Progspace):
+        if gdb.parameter("verbose"):
+            gdb.write(
+                "Registering %s handler for %s ...\n" % (handler.name, locus.filename)
+            )
+    else:
+        raise TypeError("locus should be gdb.Progspace or None")
+
+    # Some sanity checks on HANDLER.  Calling getattr will raise an
+    # exception if the attribute doesn't exist, which is what we want.
+    # These checks are not exhaustive; we don't check the attributes
+    # have the correct types, or the method has the correct signature,
+    # but this should catch some basic mistakes.
+    getattr(handler, "name")
+    getattr(handler, "enabled")
+    call_method = getattr(handler, "__call__")
+    if not callable(call_method):
+        raise AttributeError(
+            "'%s' object's '__call__' attribute is not callable"
+            % type(handler).__name__
+        )
+
+    i = 0
+    for needle in locus.missing_debug_handlers:
+        if needle.name == handler.name:
+            if replace:
+                del locus.missing_debug_handlers[i]
+            else:
+                raise RuntimeError("Handler %s already exists." % handler.name)
+        i += 1
+    locus.missing_debug_handlers.insert(0, handler)
diff --git a/gdb/python/py-progspace.c b/gdb/python/py-progspace.c
index f636ffd..0797ef1 100644
--- a/gdb/python/py-progspace.c
+++ b/gdb/python/py-progspace.c
@@ -55,6 +55,9 @@ struct pspace_object
 
   /* The debug method list.  */
   PyObject *xmethods;
+
+  /* The missing debug handler list.  */
+  PyObject *missing_debug_handlers;
 };
 
 extern PyTypeObject pspace_object_type
@@ -164,6 +167,7 @@ pspy_dealloc (PyObject *self)
   Py_XDECREF (ps_self->frame_unwinders);
   Py_XDECREF (ps_self->type_printers);
   Py_XDECREF (ps_self->xmethods);
+  Py_XDECREF (ps_self->missing_debug_handlers);
   Py_TYPE (self)->tp_free (self);
 }
 
@@ -199,6 +203,10 @@ pspy_initialize (pspace_object *self)
   if (self->xmethods == NULL)
     return 0;
 
+  self->missing_debug_handlers = PyList_New (0);
+  if (self->missing_debug_handlers == nullptr)
+    return 0;
+
   return 1;
 }
 
@@ -353,6 +361,47 @@ pspy_get_xmethods (PyObject *o, void *ignore)
   return self->xmethods;
 }
 
+/* Return the list of missing debug handlers for this program space.  */
+
+static PyObject *
+pspy_get_missing_debug_handlers (PyObject *o, void *ignore)
+{
+  pspace_object *self = (pspace_object *) o;
+
+  Py_INCREF (self->missing_debug_handlers);
+  return self->missing_debug_handlers;
+}
+
+/* Set this program space's list of missing debug handlers to HANDLERS.  */
+
+static int
+pspy_set_missing_debug_handlers (PyObject *o, PyObject *handlers,
+				 void *ignore)
+{
+  pspace_object *self = (pspace_object *) o;
+
+  if (handlers == nullptr)
+    {
+      PyErr_SetString (PyExc_TypeError,
+		       "cannot delete the missing debug handlers list");
+      return -1;
+    }
+
+  if (!PyList_Check (handlers))
+    {
+      PyErr_SetString (PyExc_TypeError,
+		       "the missing debug handlers attribute must be a list");
+      return -1;
+    }
+
+  /* Take care in case the LHS and RHS are related somehow.  */
+  gdbpy_ref<> tmp (self->missing_debug_handlers);
+  Py_INCREF (handlers);
+  self->missing_debug_handlers = handlers;
+
+  return 0;
+}
+
 /* Set the 'type_printers' attribute.  */
 
 static int
@@ -745,6 +794,8 @@ static gdb_PyGetSetDef pspace_getset[] =
     "Type printers.", NULL },
   { "xmethods", pspy_get_xmethods, NULL,
     "Debug methods.", NULL },
+  { "missing_debug_handlers", pspy_get_missing_debug_handlers,
+    pspy_set_missing_debug_handlers, "Missing debug handlers.", NULL },
   { NULL }
 };
 
diff --git a/gdb/python/python.c b/gdb/python/python.c
index d569fb5..4523e30 100644
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -127,7 +127,9 @@ static enum ext_lang_rc gdbpy_before_prompt_hook
 static gdb::optional<std::string> gdbpy_colorize
   (const std::string &filename, const std::string &contents);
 static gdb::optional<std::string> gdbpy_colorize_disasm
-  (const std::string &content, gdbarch *gdbarch);
+(const std::string &content, gdbarch *gdbarch);
+static ext_lang_missing_debuginfo_result gdbpy_handle_missing_debuginfo
+  (const struct extension_language_defn *extlang, struct objfile *objfile);
 
 /* The interface between gdb proper and loading of python scripts.  */
 
@@ -173,6 +175,8 @@ static const struct extension_language_ops python_extension_ops =
   gdbpy_colorize_disasm,
 
   gdbpy_print_insn,
+
+  gdbpy_handle_missing_debuginfo
 };
 
 #endif /* HAVE_PYTHON */
@@ -1688,6 +1692,83 @@ gdbpy_get_current_objfile (PyObject *unused1, PyObject *unused2)
   return objfile_to_objfile_object (gdbpy_current_objfile).release ();
 }
 
+/* Implement the 'handle_missing_debuginfo' hook for Python.  GDB has
+   failed to find any debug information for OBJFILE.  The extension has a
+   chance to record this, or even install the required debug information.
+   See the description of ext_lang_missing_debuginfo_result in
+   extension-priv.h for details of the return value.  */
+
+static ext_lang_missing_debuginfo_result
+gdbpy_handle_missing_debuginfo (const struct extension_language_defn *extlang,
+				struct objfile *objfile)
+{
+  /* Early exit if Python is not initialised.  */
+  if (!gdb_python_initialized)
+    return {};
+
+  struct gdbarch *gdbarch = objfile->arch ();
+
+  gdbpy_enter enter_py (gdbarch);
+
+  /* Convert OBJFILE into the corresponding Python object.  */
+  gdbpy_ref<> pyo_objfile = objfile_to_objfile_object (objfile);
+  if (pyo_objfile == nullptr)
+    {
+      gdbpy_print_stack ();
+      return {};
+    }
+
+  /* Lookup the helper function within the GDB module.  */
+  gdbpy_ref<> pyo_handler
+    (PyObject_GetAttrString (gdb_python_module, "_handle_missing_debuginfo"));
+  if (pyo_handler == nullptr)
+    {
+      gdbpy_print_stack ();
+      return {};
+    }
+
+  /* Call the function, passing in the Python objfile object.  */
+  gdbpy_ref<> pyo_execute_ret
+    (PyObject_CallFunctionObjArgs (pyo_handler.get (), pyo_objfile.get (),
+				   nullptr));
+  if (pyo_execute_ret == nullptr)
+    {
+      /* If the handler is cancelled due to a Ctrl-C, then propagate
+	 the Ctrl-C as a GDB exception instead of swallowing it.  */
+      gdbpy_print_stack_or_quit ();
+      return {};
+    }
+
+  /* Parse the result, and convert it back to the C++ object.  */
+  if (pyo_execute_ret == Py_None)
+    return {};
+
+  if (PyBool_Check (pyo_execute_ret.get ()))
+    {
+      bool try_again = PyObject_IsTrue (pyo_execute_ret.get ());
+      return ext_lang_missing_debuginfo_result (try_again);
+    }
+
+  if (!gdbpy_is_string (pyo_execute_ret.get ()))
+    {
+      PyErr_SetString (PyExc_ValueError,
+		       "return value from _handle_missing_debuginfo should "
+		       "be None, a Bool, or a String");
+      gdbpy_print_stack ();
+      return {};
+    }
+
+  gdb::unique_xmalloc_ptr<char> filename
+    = python_string_to_host_string (pyo_execute_ret.get ());
+  if (filename == nullptr)
+    {
+      gdbpy_print_stack ();
+      return {};
+    }
+
+  return ext_lang_missing_debuginfo_result (std::string (filename.get ()));
+}
+
 /* Compute the list of active python type printers and store them in
    EXT_PRINTERS->py_type_printers.  The product of this function is used by
    gdbpy_apply_type_printers, and freed by gdbpy_free_type_printers.