[PR gdb/22640] ptype: add option to use hexadecimal notation

This commit adds a flag to the ptype command in order to print the
offsets and sizes of struct members using the hexadecimal notation.  The
'x' flag ensures use of the hexadecimal notation while the 'd' flag
ensures use of the decimal notation.  The default is to use decimal
notation.

Before this patch, gdb only uses decimal notation, as pointed out in PR
gdb/22640.

Here is an example of this new behavior with hex output turned on:

    (gdb) ptype /ox struct type_print_options
    /* offset      |    size */  type = struct type_print_options {
    /* 0x0000: 0x0 |  0x0004 */    unsigned int raw : 1;
    /* 0x0000: 0x1 |  0x0004 */    unsigned int print_methods : 1;
    /* 0x0000: 0x2 |  0x0004 */    unsigned int print_typedefs : 1;
    /* 0x0000: 0x3 |  0x0004 */    unsigned int print_offsets : 1;
    /* 0x0000: 0x4 |  0x0004 */    unsigned int print_in_hex : 1;
    /* XXX  3-bit hole       */
    /* XXX  3-byte hole      */
    /* 0x0004      |  0x0004 */    int print_nested_type_limit;
    /* 0x0008      |  0x0008 */    typedef_hash_table *local_typedefs;
    /* 0x0010      |  0x0008 */    typedef_hash_table *global_typedefs;
    /* 0x0018      |  0x0008 */    ext_lang_type_printers *global_printers;

                                   /* total size (bytes):   32 */
                                 }

This patch also adds the 'set print type hex' and 'show print type hex'
commands in order to set and inspect the default behavior regarding the
use of decimal or hexadecimal notation when printing struct sizes and
offsets.

Tested using on x86_64.

gdb/ChangeLog:

	PR gdb/22640
	* typeprint.h (struct type_print_options): Add print_in_hex
	flag.
	(struct print_offset_data): Add print_in_hex flag, add a
	constructor accepting a type_print_options* argument.
	* typeprint.c (type_print_raw_options, default_ptype_flags): Set
	default value for print_in_hex.
	(print_offset_data::indentation): Allow more horizontal space.
	(print_offset_data::print_offset_data): Add ctor.
	(print_offset_data::maybe_print_hole, print_offset_data::update):
	Handle the print_in_hex flag.
	(whatis_exp): Handle 'x' and 'd' flags.
	(print_offsets_and_sizes_in_hex): Declare.
	(set_print_offsets_and_sizes_in_hex): Create.
	(show_print_offsets_and_sizes_in_hex): Create.
	(_initialize_typeprint): Update help message for the ptype
	command, register the 'set print type hex' and 'show print type
	hex' commands.
	* c-typeprint.c (c_print_type, c_type_print_base_struct_union)
	(c_type_print_base): Construct the print_offset_data
	object using the type_print_optons parameter.
	* rust-lang.c (rust_language::print_type): Construct the
	print_offset_data object using the type_print_optons parameter.
	* NEWS: Mention the new flags of the ptype command.

gdb/doc/ChangeLog:

	PR gdb/22640
	* gdb.texinfo (Symbols): Describe the 'x' and 'd' flags of the
	ptype command, describe 'set print type hex' and 'show print
	type hex' commands.  Update 'ptype/o' examples.

gdb/testsuite/ChangeLog:

	PR gdb/22640
	* gdb.base/ptype-offsets.exp: Add tests to verify the behavior
	of 'ptype/ox' and 'ptype/od'. Check that 'set print type hex'
	changes the default behavior of 'ptype/o'.  Update to take into
	account new horizontal layout.
	* gdb.rust/simple.exp: Update ptype test to check new horizontal
	layout.
	* gdb.rust/union.exp: Same.

2 files changed, 75 insertions, 45 deletions

diff --git a/gdb/doc/ChangeLog b/gdb/doc/ChangeLog
index 08e3182..c22d4a6 100644
--- a/gdb/doc/ChangeLog
+++ b/gdb/doc/ChangeLog
@@ -1,3 +1,10 @@
+2021-04-25  Lancelot Six  <lsix@lancelotsix.com>
+
+	PR gdb/22640
+	* gdb.texinfo (Symbols): Describe the 'x' and 'd' flags of the
+	ptype command, describe 'set print type hex' and 'show print
+	type hex' commands.  Update 'ptype/o' examples.
+
 2021-04-21  Tankut Baris Aktemur  <tankut.baris.aktemur@intel.com>
 
 	* gdb.texinfo (GDB/MI Breakpoint Information): Update the
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index fd04aee..a068de6 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -18885,6 +18885,21 @@ types.
 This command shows the current setting of typedef display when
 printing classes.
 
+@kindex set print type hex
+@item set print type hex
+@itemx set print type hex on
+@itemx set print type hex off
+
+When @value{GDBN} prints sizes and offsets of struct members, it can use
+either the decimal or hexadecimal notation.  You can select one or the
+other either by passing the appropriate flag to @code{ptype}, or by using
+the @command{set print type hex} command.
+
+@kindex show print type hex
+@item show print type hex
+This command shows whether the sizes and offsets of struct members are
+printed in decimal or hexadecimal notation.
+
 @kindex info address
 @cindex address of a symbol
 @item info address @var{symbol}
@@ -18997,6 +19012,14 @@ exists in case you change the default with @command{set print type typedefs}.
 Print the offsets and sizes of fields in a struct, similar to what the
 @command{pahole} tool does.  This option implies the @code{/tm} flags.
 
+@item x
+Use hexadecimal notation when printing offsets and sizes of fields in a
+struct.
+
+@item d
+Use decimal notation when printing offsets and sizes of fields in a
+struct.
+
 For example, given the following declarations:
 
 @smallexample
@@ -19037,14 +19060,14 @@ Issuing a @kbd{ptype /o struct tuv} command would print:
 
 @smallexample
 (@value{GDBP}) ptype /o struct tuv
-/* offset    |  size */  type = struct tuv @{
-/*    0      |     4 */    int a1;
-/* XXX  4-byte hole  */
-/*    8      |     8 */    char *a2;
-/*   16      |     4 */    int a3;
+/* offset      |    size */  type = struct tuv @{
+/*      0      |       4 */    int a1;
+/* XXX  4-byte hole      */
+/*      8      |       8 */    char *a2;
+/*     16      |       4 */    int a3;
 
-                           /* total size (bytes):   24 */
-                         @}
+                               /* total size (bytes):   24 */
+                             @}
 @end smallexample
 
 Notice the format of the first column of comments.  There, you can
@@ -19059,34 +19082,34 @@ It is also possible to print offsets inside an union:
 
 @smallexample
 (@value{GDBP}) ptype /o union qwe
-/* offset    |  size */  type = union qwe @{
-/*                24 */    struct tuv @{
-/*    0      |     4 */        int a1;
-/* XXX  4-byte hole  */
-/*    8      |     8 */        char *a2;
-/*   16      |     4 */        int a3;
-
-                               /* total size (bytes):   24 */
-                           @} fff1;
-/*                40 */    struct xyz @{
-/*    0      |     4 */        int f1;
-/*    4      |     1 */        char f2;
-/* XXX  3-byte hole  */
-/*    8      |     8 */        void *f3;
-/*   16      |    24 */        struct tuv @{
-/*   16      |     4 */            int a1;
-/* XXX  4-byte hole  */
-/*   24      |     8 */            char *a2;
-/*   32      |     4 */            int a3;
+/* offset      |    size */  type = union qwe @{
+/*                    24 */    struct tuv @{
+/*      0      |       4 */        int a1;
+/* XXX  4-byte hole      */
+/*      8      |       8 */        char *a2;
+/*     16      |       4 */        int a3;
 
                                    /* total size (bytes):   24 */
-                               @} f4;
+                               @} fff1;
+/*                    40 */    struct xyz @{
+/*      0      |       4 */        int f1;
+/*      4      |       1 */        char f2;
+/* XXX  3-byte hole      */
+/*      8      |       8 */        void *f3;
+/*     16      |      24 */        struct tuv @{
+/*     16      |       4 */            int a1;
+/* XXX  4-byte hole      */
+/*     24      |       8 */            char *a2;
+/*     32      |       4 */            int a3;
+
+                                       /* total size (bytes):   24 */
+                                   @} f4;
+
+                                   /* total size (bytes):   40 */
+                               @} fff2;
 
                                /* total size (bytes):   40 */
-                           @} fff2;
-
-                           /* total size (bytes):   40 */
-                         @}
+                             @}
 @end smallexample
 
 In this case, since @code{struct tuv} and @code{struct xyz} occupy the
@@ -19099,20 +19122,20 @@ bitfields:
 
 @smallexample
 (@value{GDBP}) ptype /o struct tyu
-/* offset    |  size */  type = struct tyu @{
-/*    0:31   |     4 */    int a1 : 1;
-/*    0:28   |     4 */    int a2 : 3;
-/*    0: 5   |     4 */    int a3 : 23;
-/*    3: 3   |     1 */    signed char a4 : 2;
-/* XXX  3-bit hole   */
-/* XXX  4-byte hole  */
-/*    8      |     8 */    int64_t a5;
-/*   16: 0   |     4 */    int a6 : 5;
-/*   16: 5   |     8 */    int64_t a7 : 3;
-"/* XXX  7-byte padding  */
-
-                           /* total size (bytes):   24 */
-                         @}
+/* offset      |    size */  type = struct tyu @{
+/*      0:31   |       4 */    int a1 : 1;
+/*      0:28   |       4 */    int a2 : 3;
+/*      0: 5   |       4 */    int a3 : 23;
+/*      3: 3   |       1 */    signed char a4 : 2;
+/* XXX  3-bit hole       */
+/* XXX  4-byte hole      */
+/*      8      |       8 */    int64_t a5;
+/*     16: 0   |       4 */    int a6 : 5;
+/*     16: 5   |       8 */    int64_t a7 : 3;
+/* XXX  7-byte padding   */
+
+                               /* total size (bytes):   24 */
+                             @}
 @end smallexample
 
 Note how the offset information is now extended to also include the