arch-utils: Make the last endianness actually chosen sticky

Use the last endianness explicitly selected, either by choosing a binary
file or with the `set endian' command, for future automatic selection.

As observed with the `gdb.base/step-over-no-symbols.exp' test case when
discarding the binary file even while connected to a live target the
endianness automatically selected is reset to the GDB target's default,
even if it does not match the endianness of the target being talked to.

For example with a little-endian MIPS target and the default endianness
being big we get this:

(gdb) file .../gdb/testsuite/outputs/gdb.base/step-over-no-symbols/step-over-no-symbols
Reading symbols from .../gdb/testsuite/outputs/gdb.base/step-over-no-symbols/step-over-no-symbols...done.
(gdb) delete breakpoints
(gdb) info breakpoints
No breakpoints or watchpoints.
(gdb) break main
Breakpoint 1 at 0x400840: file .../gdb/testsuite/gdb.base/start.c, line 34.
[...]
(gdb) continue
Continuing.

Breakpoint 1, main () at .../gdb/testsuite/gdb.base/start.c:34
34	  foo();
(gdb) delete breakpoints
Delete all breakpoints? (y or n) y
(gdb) info breakpoints
No breakpoints or watchpoints.
(gdb) file
A program is being debugged already.
Are you sure you want to change the file? (y or n) y
No executable file now.
Discard symbol table from `.../gdb/testsuite/outputs/gdb.base/step-over-no-symbols/step-over-no-symbols'? (y or n) y
No symbol file now.
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: purging symbols
p /x $pc
$1 = 0x40084000
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: get before PC
break *$pc
Breakpoint 2 at 0x40084000
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: break *$pc
set displaced-stepping off
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: set displaced-stepping off
stepi
Warning:
Cannot insert breakpoint 2.
Cannot access memory at address 0x40084000

Command aborted.
(gdb) FAIL: gdb.base/step-over-no-symbols.exp: displaced=off: stepi
p /x $pc
$2 = 0x40084000
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: get after PC
FAIL: gdb.base/step-over-no-symbols.exp: displaced=off: advanced
Remote debugging from host ...
monitor exit
(gdb) Killing process(es): ...
testcase .../gdb/testsuite/gdb.base/step-over-no-symbols.exp completed in 2 seconds

which shows that with the removal of the executable debugged the
endianness of $pc still at `main' gets swapped and the value in that
register is now incorrectly interpreted as 0x40084000 rather than
0x400840 as shown earlier on with the `break' command.  Consequently the
debug session no longer works as expected, until the endianness is
overridden with an explicit `set endian little' command.

This will happen while working with any target hardware whose endianness
does not match the default GDB target's endianness guessed and recorded
for a later use in `initialize_current_architecture'.

Given that within a single run of GDB it is more likely that consecutive
target connections will use the same endianness than that the endianness
will be swapped between connections, it makes sense to preserve the last
endianness explicitly selected as the automatic default.  It will make a
session like above, where an executable is removed, work correctly and
will retain the endianness for a further reconnection to the target.

And the new automatic default will still be overridden by subsequently
choosing a binary to debug, or with an explicit `set endian' command.

With the change in place the test case above completes successfully:

(gdb) continue
Continuing.

Breakpoint 1, main () at .../gdb/testsuite/gdb.base/start.c:34
34	  foo();
(gdb) delete breakpoints
Delete all breakpoints? (y or n) y
(gdb) info breakpoints
No breakpoints or watchpoints.
(gdb) file
A program is being debugged already.
Are you sure you want to change the file? (y or n) y
No executable file now.
Discard symbol table from `.../gdb/testsuite/outputs/gdb.base/step-over-no-symbols/step-over-no-symbols'? (y or n) y
No symbol file now.
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: purging symbols
p /x $pc
warning: GDB can't find the start of the function at 0x400840.

    GDB is unable to find the start of the function at 0x400840
and thus can't determine the size of that function's stack frame.
This means that GDB may be unable to access that stack frame, or
the frames below it.
    This problem is most likely caused by an invalid program counter or
stack pointer.
    However, if you think GDB should simply search farther back
from 0x400840 for code which looks like the beginning of a
function, you can increase the range of the search using the `set
heuristic-fence-post' command.
$1 = 0x400840
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: get before PC
break *$pc
Breakpoint 2 at 0x400840
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: break *$pc
set displaced-stepping off
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: set displaced-stepping off
stepi
warning: GDB can't find the start of the function at 0x4007f8.
0x004007f8 in ?? ()
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: stepi
p /x $pc
$2 = 0x4007f8
(gdb) PASS: gdb.base/step-over-no-symbols.exp: displaced=off: get after PC
PASS: gdb.base/step-over-no-symbols.exp: displaced=off: advanced
Remote debugging from host ...
monitor exit
(gdb) Killing process(es): ...
testcase .../gdb/testsuite/gdb.base/step-over-no-symbols.exp completed in 2 seconds

	gdb/
	* arch-utils.c (gdbarch_info_fill): Set `default_byte_order' to
	the endianness selected.
	* NEWS: Document `set endian auto' mode operation update.

	gdb/doc/
	* gdb.texinfo (Choosing Target Byte Order): Document endianness
	selection details with the `set endian auto' mode.

	gdb/testsuite
	* gdb.base/endian.exp: New test.
	* gdb.base/endian.c: New test source.

8 files changed, 149 insertions, 0 deletions

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index eaf0ce1..c6c862f 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,3 +1,9 @@
+2018-05-31  Maciej W. Rozycki  <macro@mips.com>
+
+	* arch-utils.c (gdbarch_info_fill): Set `default_byte_order' to
+	the endianness selected.
+	* NEWS: Document `set endian auto' mode operation update.
+
 2018-05-31  Alan Hayward  <alan.hayward@arm.com>
 
 	* Makefile.in: Add new header.
diff --git a/gdb/NEWS b/gdb/NEWS
index cef5580..392e299 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -3,6 +3,12 @@
 
 *** Changes since GDB 8.1
 
+* The endianness used with the 'set endian auto' mode in the absence of
+  an executable selected for debugging is now the last endianness chosen
+  either by one of the 'set endian big' and 'set endian little' commands
+  or by inferring from the last executable used, rather than the startup
+  default.
+
 * The commands 'info variables/functions/types' now show the source line
   numbers of symbol definitions when available.
 
diff --git a/gdb/arch-utils.c b/gdb/arch-utils.c
index b3169ca..76bc16f 100644
--- a/gdb/arch-utils.c
+++ b/gdb/arch-utils.c
@@ -768,6 +768,8 @@ gdbarch_info_fill (struct gdbarch_info *info)
   if (info->byte_order == BFD_ENDIAN_UNKNOWN)
     info->byte_order = default_byte_order;
   info->byte_order_for_code = info->byte_order;
+  /* Wire the default to the last selected byte order.  */
+  default_byte_order = info->byte_order;
 
   /* "(gdb) set osabi ...".  Handled by gdbarch_lookup_osabi.  */
   /* From the manual override, or from file.  */
diff --git a/gdb/doc/ChangeLog b/gdb/doc/ChangeLog
index 798ff37..1689e89 100644
--- a/gdb/doc/ChangeLog
+++ b/gdb/doc/ChangeLog
@@ -1,3 +1,8 @@
+2018-05-31  Maciej W. Rozycki  <macro@mips.com>
+
+	* gdb.texinfo (Choosing Target Byte Order): Document endianness
+	selection details with the `set endian auto' mode.
+
 2018-05-04  Tom Tromey  <tom@tromey.com>
 
 	PR python/22731:
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index 305f21a..2920022 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -20308,6 +20308,15 @@ Display @value{GDBN}'s current idea of the target byte order.
 
 @end table
 
+If the @code{set endian auto} mode is in effect and no executable has
+been selected, then the endianness used is the last one chosen either
+by one of the @code{set endian big} and @code{set endian little}
+commands or by inferring from the last executable used.  If no
+endianness has been previously chosen, then the default for this mode
+is inferred from the target @value{GDBN} has been built for, and is
+@code{little} if the name of the target CPU has an @code{el} suffix
+and @code{big} otherwise.
+
 Note that these commands merely adjust interpretation of symbolic
 data on the host, and that they have absolutely no effect on the
 target system.
diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index b2938b1..c303d80 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,3 +1,8 @@
+2018-05-31  Maciej W. Rozycki  <macro@mips.com>
+
+	* gdb.base/endian.exp: New test.
+	* gdb.base/endian.c: New test source.
+
 2018-05-24  Andrew Burgess  <andrew.burgess@embecosm.com>
 
 	PR gdb/23203
diff --git a/gdb/testsuite/gdb.base/endian.c b/gdb/testsuite/gdb.base/endian.c
new file mode 100644
index 0000000..867440c
--- /dev/null
+++ b/gdb/testsuite/gdb.base/endian.c
@@ -0,0 +1,22 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright (C) 2018 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+int
+main (void)
+{
+  return 0;
+}
diff --git a/gdb/testsuite/gdb.base/endian.exp b/gdb/testsuite/gdb.base/endian.exp
new file mode 100644
index 0000000..f6682cd
--- /dev/null
+++ b/gdb/testsuite/gdb.base/endian.exp
@@ -0,0 +1,94 @@
+# Copyright (C) 2018 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test automatic endianness selection.
+
+standard_testfile
+
+set en_auto "The target endianness is set automatically"
+set en_set "The target is assumed to be"
+
+clean_restart
+
+# First check that the automatic endianness is updated
+# with the `set endian' command.
+gdb_test "set endian auto" \
+    "$en_auto \\\(currently \(\?:big\|little\) endian\\\)" \
+    "default target endianness"
+gdb_test "set endian big" "$en_set big endian" \
+    "set target endianness"
+gdb_test "set endian auto" "$en_auto \\\(currently big endian\\\)" \
+    "auto target endianness"
+gdb_test "set endian little" "$en_set little endian" \
+    "set target endianness little"
+gdb_test "set endian auto" "$en_auto \\\(currently little endian\\\)" \
+    "auto target endianness little"
+gdb_test "set endian big" "$en_set big endian" \
+    "set target endianness big"
+gdb_test "set endian auto" "$en_auto \\\(currently big endian\\\)" \
+    "auto target endianness big"
+
+if { [build_executable ${testfile}.exp $testfile] } {
+    gdb_suppress_entire_file "$pf_prefix cannot build executable"
+}
+
+if { [gdb_file_cmd $binfile] } {
+    gdb_suppress_entire_file "$pf_prefix cannot select executable"
+}
+set test "get target endianness"
+if { [gdb_test_multiple "show endian" "$test" {
+    -re "$en_auto \\\(currently \(big\|little\) endian\\\).*$gdb_prompt" {
+	set endian $expect_out(1,string)
+	pass "$test"
+    }
+}] } {
+    gdb_suppress_entire_file \
+	"$pf_prefix cannot determine executable endianness"
+    set endian ""
+}
+
+# Now check that the automatic endianness is updated
+# according to the executable selected.
+if { [gdb_unload] } {
+    gdb_suppress_entire_file "$pf_prefix cannot unselect executable"
+}
+gdb_test "set endian big" "$en_set big endian" \
+    "override target endianness big"
+gdb_test "set endian auto" "$en_auto \\\(currently big endian\\\)" \
+    "override auto target endianness big"
+if { [gdb_file_cmd $binfile] } {
+    gdb_suppress_entire_file "$pf_prefix cannot select executable"
+}
+gdb_test "show endian" "$en_auto \\\(currently $endian endian\\\)" \
+    "previously big default executable endianness"
+if { [gdb_unload] } {
+    gdb_suppress_entire_file "$pf_prefix cannot unselect executable"
+}
+gdb_test "show endian" "$en_auto \\\(currently $endian endian\\\)" \
+    "previously big default no executable endianness"
+gdb_test "set endian little" "$en_set little endian" \
+    "override target endianness little"
+gdb_test "set endian auto" "$en_auto \\\(currently little endian\\\)" \
+    "override auto target endianness little"
+if { [gdb_file_cmd $binfile] } {
+    gdb_suppress_entire_file "$pf_prefix cannot select executable"
+}
+gdb_test "show endian" "$en_auto \\\(currently $endian endian\\\)" \
+    "previously little default executable endianness"
+if { [gdb_unload] } {
+    gdb_suppress_entire_file "$pf_prefix cannot unselect executable"
+}
+gdb_test "show endian" "$en_auto \\\(currently $endian endian\\\)" \
+    "previously little default no executable endianness"