[GDBserver] Make Zx/zx packet handling idempotent.

This patch fixes hardware breakpoint regressions exposed by my fix for
"PR breakpoints/7143 - Watchpoint does not trigger when first set", at
https://sourceware.org/ml/gdb-patches/2014-03/msg00167.html

The testsuite caught them on Linux/x86_64, at least.  gdb.sum:

gdb.sum:

 FAIL: gdb.base/hbreak2.exp: next over recursive call
 FAIL: gdb.base/hbreak2.exp: backtrace from factorial(5.1)
 FAIL: gdb.base/hbreak2.exp: continue until exit at recursive next test

gdb.log:

 (gdb) next

 Program received signal SIGTRAP, Trace/breakpoint trap.
 factorial (value=4) at ../../../src/gdb/testsuite/gdb.base/break.c:113
 113       if (value > 1) {  /* set breakpoint 7 here */
 (gdb) FAIL: gdb.base/hbreak2.exp: next over recursive call

Actually, that patch just exposed a latent issue to "breakpoints
always-inserted off" mode, not really caused it.  After that patch,
GDB no longer removes breakpoints at each internal event, thus making
some scenarios behave like breakpoint always-inserted on.  The bug is
easy to trigger with always-inserted on.

The issue is that since the target-side breakpoint conditions support,
if the stub/server supports evaluating breakpoint conditions on the
target side, then GDB is sending duplicate Zx packets to the target
without removing them before, and GDBserver is not really expecting
that for Z packets other than Z0/z0.  E.g., with "set breakpoint
always-inserted on" and "set debug remote 1":

 (gdb) b main
 Sending packet: $m410943,1#ff...Packet received: 48
 Breakpoint 4 at 0x410943: file ../../../src/gdb/gdbserver/server.c, line 3028.
 Sending packet: $Z0,410943,1#48...Packet received: OK
                 ^^^^^^^^^^^^
 (gdb) b main
 Note: breakpoint 4 also set at pc 0x410943.
 Sending packet: $m410943,1#ff...Packet received: 48
 Breakpoint 5 at 0x410943: file ../../../src/gdb/gdbserver/server.c, line 3028.
 Sending packet: $Z0,410943,1#48...Packet received: OK
                 ^^^^^^^^^^^^
 (gdb) b main
 Note: breakpoints 4 and 5 also set at pc 0x410943.
 Sending packet: $m410943,1#ff...Packet received: 48
 Breakpoint 6 at 0x410943: file ../../../src/gdb/gdbserver/server.c, line 3028.
 Sending packet: $Z0,410943,1#48...Packet received: OK
                 ^^^^^^^^^^^^
 (gdb) del
 Delete all breakpoints? (y or n) y
 Sending packet: $Z0,410943,1#48...Packet received: OK
 Sending packet: $Z0,410943,1#48...Packet received: OK
 Sending packet: $z0,410943,1#68...Packet received: OK

And for Z1, similarly:

 (gdb) hbreak main
 Sending packet: $m410943,1#ff...Packet received: 48
 Hardware assisted breakpoint 4 at 0x410943: file ../../../src/gdb/gdbserver/server.c, line 3028.
 Sending packet: $Z1,410943,1#49...Packet received: OK
                 ^^^^^^^^^^^^
 Packet Z1 (hardware-breakpoint) is supported
 (gdb) hbreak main
 Note: breakpoint 4 also set at pc 0x410943.
 Sending packet: $m410943,1#ff...Packet received: 48
 Hardware assisted breakpoint 5 at 0x410943: file ../../../src/gdb/gdbserver/server.c, line 3028.
 Sending packet: $Z1,410943,1#49...Packet received: OK
                 ^^^^^^^^^^^^
 (gdb) hbreak main
 Note: breakpoints 4 and 5 also set at pc 0x410943.
 Sending packet: $m410943,1#ff...Packet received: 48
 Hardware assisted breakpoint 6 at 0x410943: file ../../../src/gdb/gdbserver/server.c, line 3028.
 Sending packet: $Z1,410943,1#49...Packet received: OK
                 ^^^^^^^^^^^^
 (gdb) del
 Delete all breakpoints? (y or n) y
 Sending packet: $Z1,410943,1#49...Packet received: OK
                 ^^^^^^^^^^^^
 Sending packet: $Z1,410943,1#49...Packet received: OK
                 ^^^^^^^^^^^^
 Sending packet: $z1,410943,1#69...Packet received: OK
                 ^^^^^^^^^^^^

So GDB sent a bunch of Z1 packets, and then when finally removing the
breakpoint, only one z1 packet was sent.  On the GDBserver side (with
monitor set debug-hw-points 1), in the Z1 case, we see:

 $ ./gdbserver :9999 ./gdbserver
 Process ./gdbserver created; pid = 8629
 Listening on port 9999
 Remote debugging from host 127.0.0.1
 insert_watchpoint (addr=410943, len=1, type=instruction-execute):
	 CONTROL (DR7): 00000101          STATUS (DR6): 00000000
	 DR0: addr=0x410943, ref.count=1  DR1: addr=0x0, ref.count=0
	 DR2: addr=0x0, ref.count=0  DR3: addr=0x0, ref.count=0
 insert_watchpoint (addr=410943, len=1, type=instruction-execute):
	 CONTROL (DR7): 00000101          STATUS (DR6): 00000000
	 DR0: addr=0x410943, ref.count=2  DR1: addr=0x0, ref.count=0
	 DR2: addr=0x0, ref.count=0  DR3: addr=0x0, ref.count=0
 insert_watchpoint (addr=410943, len=1, type=instruction-execute):
	 CONTROL (DR7): 00000101          STATUS (DR6): 00000000
	 DR0: addr=0x410943, ref.count=3  DR1: addr=0x0, ref.count=0
	 DR2: addr=0x0, ref.count=0  DR3: addr=0x0, ref.count=0
 insert_watchpoint (addr=410943, len=1, type=instruction-execute):
	 CONTROL (DR7): 00000101          STATUS (DR6): 00000000
	 DR0: addr=0x410943, ref.count=4  DR1: addr=0x0, ref.count=0
	 DR2: addr=0x0, ref.count=0  DR3: addr=0x0, ref.count=0
 insert_watchpoint (addr=410943, len=1, type=instruction-execute):
	 CONTROL (DR7): 00000101          STATUS (DR6): 00000000
	 DR0: addr=0x410943, ref.count=5  DR1: addr=0x0, ref.count=0
	 DR2: addr=0x0, ref.count=0  DR3: addr=0x0, ref.count=0
 remove_watchpoint (addr=410943, len=1, type=instruction-execute):
	 CONTROL (DR7): 00000101          STATUS (DR6): 00000000
	 DR0: addr=0x410943, ref.count=4  DR1: addr=0x0, ref.count=0
	 DR2: addr=0x0, ref.count=0  DR3: addr=0x0, ref.count=0

That's one insert_watchpoint call for each Z1 packet, and then one
remove_watchpoint call for the z1 packet.  Notice how ref.count
increased for each insert_watchpoint call, and then in the end, after
GDB told GDBserver to forget about the hardware breakpoint, GDBserver
ends with the the first debug register still with ref.count=4!  IOW,
the hardware breakpoint is left armed on the target, while on the GDB
end it's gone.  If the program happens to execute 0x410943 afterwards,
then the CPU traps, GDBserver reports the trap to GDB, and GDB not
having a breakpoint set at that address anymore, reports to the user a
spurious SIGTRAP.

This is exactly what is happening in the hbreak2.exp test, though in
that case, it's a shared library event that triggers a
breakpoint_re_set, when breakpoints are still inserted (because
nowadays GDB doesn't remove breakpoints while handling internal
events), and that recreates breakpoint locations, which likewise
forces breakpoint reinsertion and Zx packet resends...

That is a lot of bogus Zx duplication that should possibly be
addressed on the GDB side.  GDB resends Zx packets because the way to
change the target-side condition, is to resend the breakpoint to the
server with the new condition.  (That's an option in the packet: e.g.,
"Z1,410943,1;X3,220027" for "hbreak main if 0".  The packets in the
examples above are shorter because the breakpoints don't have
conditions attached).  GDB doesn't remove the breakpoint first before
reinserting it because that'd be bad for non-stop, as it'd open a
window where the inferior could miss the breakpoint.  The conditions
actually haven't changed between the resends, but GDB isn't smart
enough to realize that.

(TBC, if the target doesn't support target-side conditions, then GDB
doesn't trigger these resends (init_bp_location calls
mark_breakpoint_location_modified, and that does nothing if condition
evaluation is on the host side.  The resends are caused by the
'loc->condition_changed = condition_modified.'  line.)

But, even if GDB was made smarter, GDBserver should really still
handle the resends anyway.  So target-side conditions also aren't
really to blame.  The documentation of the Z/z packets says:

 "To avoid potential problems with duplicate packets, the operations
 should be implemented in an idempotent way."

As such, we may want to fix GDB, but we should definitely fix
GDBserver.  The fix is a prerequisite for target-side conditions on
hardware breakpoints anyway (and while at it, on watchpoints too).

GDBserver indeed already treats duplicate Z0 packets in an idempotent
way.  mem-break.c has the concept of high-level and low-level
breakpoints, somewhat similar to GDB's split of breakpoints vs
breakpoint locations, and keeps track of multiple breakpoints
referencing the same address/location, for the case of an internal
GDBserver breakpoint or a tracepoint being set at the same address as
a GDB breakpoint.  But, it only allows GDB to ever contribute one
reference to a software breakpoint location.  IOW, if gdbserver sees a
Z0 packet for the same address where it already had a GDB breakpoint
set, then GDBserver won't create another high-level GDB breakpoint.

However, mem-break.c only tracks GDB Z0 breakpoints.  The same logic
should apply to all kinds of Zx packets.  Currently, gdbserver passes
down each duplicate Zx (other than Z0) request directly to the
target->insert_point routine.  The x86 watchpoint support itself
refcounts watchpoint / hw breakpoint requests, to handle overlapping
watchpoints, and save debug registers.  But that code doesn't (and
really shouldn't) handle the duplicate requests, assuming that for
each insert there will be a corresponding remove.

So the fix is to generalize mem-break.c to track all kinds of Zx
breakpoints, and filter out duplicates.  As mentioned, this ends up
adding support for target-side conditions on hardware breakpoints and
watchpoints too (though GDB itself doesn't support the latter yet).

Probably the least obvious change in the patch is that it kind of
turns the breakpoint insert/remove APIs inside out.  Before, the
target methods were only called for GDB breakpoints.  The internal
breakpoint set/delete methods inserted memory breakpoints directly
bypassing the insert/remove target methods.  That's not good when the
target should use a debug API to set software breakpoints, instead of
relying on GDBserver patching memory with breakpoint instructions, as
is the case of NTO.

Now removal/insertion of all kinds of breakpoints/watchpoints, either
internal, or from GDB, always go through the target methods.  The
insert_point/remove_point methods no longer get passed a Z packet
type, but an internal/raw breakpoint type.  They're also passed a
pointer to the raw breakpoint itself (note that's still opaque outside
mem-break.c), so that insert_memory_breakpoint /
remove_memory_breakpoint have access to the breakpoint's shadow
buffer.  I first tried passing down a new structure based on GDB's
"struct bp_target_info" (actually with that name exactly), but then
decided against it as unnecessary complication.

As software/memory breakpoints work by poking at memory, when setting
a GDB Z0 breakpoint (but not internal breakpoints, as those can assume
the conditions are already right), we need to tell the target to
prepare to access memory (which on Linux means stop threads).  If that
operation fails, we need to return error to GDB.  Seeing an error, if
this is the first breakpoint of that type that GDB tries to insert,
GDB would then assume the breakpoint type is supported, but it may
actually not be.  So we need to check whether the type is supported at
all before preparing to access memory.  And to solve that, the patch
adds a new target->supports_z_point_type method that is called before
actually trying to insert the breakpoint.

Other than that, hopefully the change is more or less obvious.

New test added that exercises the hbreak2.exp regression in a more
direct way, without relying on a breakpoint re-set happening before
main is reached.

Tested by building GDBserver for:

 aarch64-linux-gnu
 arm-linux-gnueabihf
 i686-pc-linux-gnu
 i686-w64-mingw32
 m68k-linux-gnu
 mips-linux-gnu
 mips-uclinux
 nios2-linux-gnu
 powerpc-linux-gnu
 sh-linux-gnu
 tilegx-unknown-linux-gnu
 x86_64-redhat-linux
 x86_64-w64-mingw32

And also regression tested on x86_64 Fedora 20.

gdb/gdbserver/
2014-05-20  Pedro Alves  <palves@redhat.com>

	* linux-aarch64-low.c (aarch64_insert_point)
	(aarch64_remove_point): No longer check whether the type is
	supported here.  Adjust to new interface.
	(the_low_target): Install aarch64_supports_z_point_type as
	supports_z_point_type method.
	* linux-arm-low.c (raw_bkpt_type_to_arm_hwbp_type): New function.
	(arm_linux_hw_point_initialize): Take an enum raw_bkpt_type
	instead of a Z packet char.  Adjust.
	(arm_supports_z_point_type): New function.
	(arm_insert_point, arm_remove_point): Adjust to new interface.
	(the_low_target): Install arm_supports_z_point_type.
	* linux-crisv32-low.c (cris_supports_z_point_type): New function.
	(cris_insert_point, cris_remove_point): Adjust to new interface.
	Don't check whether the type is supported here.
	(the_low_target): Install cris_supports_z_point_type.
	* linux-low.c (linux_supports_z_point_type): New function.
	(linux_insert_point, linux_remove_point): Adjust to new interface.
	* linux-low.h (struct linux_target_ops) <insert_point,
	remove_point>: Take an enum raw_bkpt_type instead of a char.  Add
	raw_breakpoint pointer parameter.
	<supports_z_point_type>: New method.
	* linux-mips-low.c (mips_supports_z_point_type): New function.
	(mips_insert_point, mips_remove_point): Adjust to new interface.
	Use mips_supports_z_point_type.
	(the_low_target): Install mips_supports_z_point_type.
	* linux-ppc-low.c (the_low_target): Install NULL as
	supports_z_point_type method.
	* linux-s390-low.c (the_low_target): Install NULL as
	supports_z_point_type method.
	* linux-sparc-low.c (the_low_target): Install NULL as
	supports_z_point_type method.
	* linux-x86-low.c (x86_supports_z_point_type): New function.
	(x86_insert_point): Adjust to new insert_point interface.  Use
	insert_memory_breakpoint.  Adjust to new
	i386_low_insert_watchpoint interface.
	(x86_remove_point): Adjust to remove_point interface.  Use
	remove_memory_breakpoint.  Adjust to new
	i386_low_remove_watchpoint interface.
	(the_low_target): Install x86_supports_z_point_type.
	* lynx-low.c (lynx_target_ops): Install NULL as
	supports_z_point_type callback.
	* nto-low.c (nto_supports_z_point_type): New.
	(nto_insert_point, nto_remove_point): Adjust to new interface.
	(nto_target_ops): Install nto_supports_z_point_type.
	* mem-break.c: Adjust intro comment.
	(struct raw_breakpoint) <raw_type, size>: New fields.
	<inserted>: Update comment.
	<shlib_disabled>: Delete field.
	(enum bkpt_type) <gdb_breakpoint>: Delete value.
	<gdb_breakpoint_Z0, gdb_breakpoint_Z1, gdb_breakpoint_Z2,
	gdb_breakpoint_Z3, gdb_breakpoint_Z4>: New values.
	(raw_bkpt_type_to_target_hw_bp_type): New function.
	(find_enabled_raw_code_breakpoint_at): New function.
	(find_raw_breakpoint_at): New type and size parameters.  Use them.
	(insert_memory_breakpoint): New function, based off
	set_raw_breakpoint_at.
	(remove_memory_breakpoint): New function.
	(set_raw_breakpoint_at): Reimplement.
	(set_breakpoint): New, based on set_breakpoint_at.
	(set_breakpoint_at): Reimplement.
	(delete_raw_breakpoint): Go through the_target->remove_point
	instead of assuming memory breakpoints.
	(find_gdb_breakpoint_at): Delete.
	(Z_packet_to_bkpt_type, Z_packet_to_raw_bkpt_type): New functions.
	(find_gdb_breakpoint): New function.
	(set_gdb_breakpoint_at): Delete.
	(z_type_supported): New function.
	(set_gdb_breakpoint_1): New function, loosely based off
	set_gdb_breakpoint_at.
	(check_gdb_bp_preconditions, set_gdb_breakpoint): New functions.
	(delete_gdb_breakpoint_at): Delete.
	(delete_gdb_breakpoint_1): New function, loosely based off
	delete_gdb_breakpoint_at.
	(delete_gdb_breakpoint): New function.
	(clear_gdb_breakpoint_conditions): Rename to ...
	(clear_breakpoint_conditions): ... this.  Don't handle a NULL
	breakpoint.
	(add_condition_to_breakpoint): Make static.
	(add_breakpoint_condition): Take a struct breakpoint pointer
	instead of an address.  Adjust.
	(gdb_condition_true_at_breakpoint): Rename to ...
	(gdb_condition_true_at_breakpoint_z_type): ... this, and add
	z_type parameter.
	(gdb_condition_true_at_breakpoint): Reimplement.
	(add_breakpoint_commands): Take a struct breakpoint pointer
	instead of an address.  Adjust.
	(gdb_no_commands_at_breakpoint): Rename to ...
	(gdb_no_commands_at_breakpoint_z_type): ... this.  Add z_type
	parameter.  Return true if no breakpoint was found.  Change debug
	output.
	(gdb_no_commands_at_breakpoint): Reimplement.
	(run_breakpoint_commands): Rename to ...
	(run_breakpoint_commands_z_type): ... this.  Add z_type parameter,
	and change return type to boolean.
	(run_breakpoint_commands): New function.
	(gdb_breakpoint_here): Also check for Z1 breakpoints.
	(uninsert_raw_breakpoint): Don't try to reinsert a disabled
	breakpoint.  Go through the_target->remove_point instead of
	assuming memory breakpoint.
	(uninsert_breakpoints_at, uninsert_all_breakpoints): Uninsert
	software and hardware breakpoints.
	(reinsert_raw_breakpoint): Go through the_target->insert_point
	instead of assuming memory breakpoint.
	(reinsert_breakpoints_at, reinsert_all_breakpoints): Reinsert
	software and hardware breakpoints.
	(check_breakpoints, breakpoint_here, breakpoint_inserted_here):
	Check both software and hardware breakpoints.
	(validate_inserted_breakpoint): Assert the breakpoint is a
	software breakpoint.  Set the inserted flag to -1 instead of
	setting shlib_disabled.
	(delete_disabled_breakpoints): Adjust.
	(validate_breakpoints): Only validate software breakpoints.
	Adjust to inserted flag change.
	(check_mem_read, check_mem_write): Skip breakpoint types other
	than software breakpoints.  Adjust to inserted flag change.
	* mem-break.h (enum raw_bkpt_type): New enum.
	(raw_breakpoint, struct process_info): Forward declare.
	(Z_packet_to_target_hw_bp_type): Delete declaration.
	(raw_bkpt_type_to_target_hw_bp_type, Z_packet_to_raw_bkpt_type)
	(set_gdb_breakpoint, delete_gdb_breakpoint)
	(clear_breakpoint_conditions): New declarations.
	(set_gdb_breakpoint_at, clear_gdb_breakpoint_conditions): Delete.
	(breakpoint_inserted_here): Update comment.
	(add_breakpoint_condition, add_breakpoint_commands): Replace
	address parameter with a breakpoint pointer parameter.
	(gdb_breakpoint_here): Update comment.
	(delete_gdb_breakpoint_at): Delete.
	(insert_memory_breakpoint, remove_memory_breakpoint): Declare.
	* server.c (process_point_options): Take a struct breakpoint
	pointer instead of an address.  Adjust.
	(process_serial_event) <Z/z packets>: Use set_gdb_breakpoint and
	delete_gdb_breakpoint.
	* spu-low.c (spu_target_ops): Install NULL as
	supports_z_point_type method.
	* target.h: Include mem-break.h.
	(struct target_ops) <prepare_to_access_memory>: Update comment.
	<supports_z_point_type>: New field.
	<insert_point, remove_point>: Take an enum raw_bkpt_type argument
	instead of a char.  Also take a raw breakpoint pointer.
	* win32-arm-low.c (the_low_target): Install NULL as
	supports_z_point_type.
	* win32-i386-low.c (i386_supports_z_point_type): New function.
	(i386_insert_point, i386_remove_point): Adjust to new interface.
	(the_low_target): Install i386_supports_z_point_type.
	* win32-low.c (win32_supports_z_point_type): New function.
	(win32_insert_point, win32_remove_point): Adjust to new interface.
	(win32_target_ops): Install win32_supports_z_point_type.
	* win32-low.h (struct win32_target_ops):
	<supports_z_point_type>: New method.
	<insert_point, remove_point>: Take an enum raw_bkpt_type argument
	instead of a char.  Also take a raw breakpoint pointer.

gdb/testsuite/
2014-05-20  Pedro Alves  <palves@redhat.com>

	* gdb.base/break-idempotent.c: New file.
	* gdb.base/break-idempotent.exp: New file.

3 files changed, 238 insertions, 0 deletions

diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index 6767997..97fb49f 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,3 +1,8 @@
+2014-05-20  Pedro Alves  <palves@redhat.com>
+
+	* gdb.base/break-idempotent.c: New file.
+	* gdb.base/break-idempotent.exp: New file.
+
 2014-05-20  Markus Metzger  <markus.t.metzger@intel.com>
 
 	* gdb.btrace/nohist.exp: New.
diff --git a/gdb/testsuite/gdb.base/break-idempotent.c b/gdb/testsuite/gdb.base/break-idempotent.c
new file mode 100644
index 0000000..c451116
--- /dev/null
+++ b/gdb/testsuite/gdb.base/break-idempotent.c
@@ -0,0 +1,52 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2014 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+int global;
+
+int cond_global;
+
+int
+foo (int i)
+{
+  global++;
+  return i;
+}
+
+int
+bar (int i)
+{
+  return i;
+}
+
+void
+test ()
+{
+  foo (1);
+  foo (2);
+  bar (3);
+}
+
+int
+main ()
+{
+  int i;
+
+  for (i = 0; i < 1000; i++)
+    test ();
+
+  return 0;
+}
diff --git a/gdb/testsuite/gdb.base/break-idempotent.exp b/gdb/testsuite/gdb.base/break-idempotent.exp
new file mode 100644
index 0000000..6019960
--- /dev/null
+++ b/gdb/testsuite/gdb.base/break-idempotent.exp
@@ -0,0 +1,181 @@
+# Copyright 2014 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+# Test that the Zx breakpoint/watchpoint packets are idempotent.
+
+# GDBserver used to not treat Zx breakpoints other than Z0 as
+# idempotent, although it must, to avoid problems with
+# retransmissions.  Even without spurious transport problems, if the
+# target supports target conditions or commands, GDB re-inserts Zx
+# breakpoints even if they are already inserted, to update the
+# target-side condition/commands.  E.g., simply when a duplicate
+# breakpoint is created, or when a shared library load causes a
+# re-set, which creates duplicate locations while breakpoints are
+# inserted, or when the condition is really changed while breakpoints
+# are inserted.  To make the test not depend on shared library support
+# or on details of the breakpoint re-set implementation, or on GDB
+# optimizing out re-sends if the condition hasn't actually changed, we
+# force always-inserted on, and really change the breakpoint's
+# condition.  For good measure, test with both always-inserted "on"
+# and "off" modes.
+
+# The test is written in black-box style, and doesn't actually use
+# anything target remote specific, so let it run on all targets.
+
+standard_testfile
+
+if {[prepare_for_testing "failed to prepare" $testfile $srcfile debug]} {
+    return -1
+}
+
+if ![runto_main] then {
+    fail "Can't run to main"
+    return 0
+}
+
+if [is_remote host] {
+    set arg [remote_download host $binfile]
+    if { $arg == "" } {
+	perror "download failed"
+	return -1
+    }
+}
+
+# Force a breakpoint re-set in GDB.  Currently this is done by
+# reloading symbols with the "file" command.
+
+proc force_breakpoint_re_set {} {
+    global binfile gdb_prompt
+
+    set test "file \$binfile"
+    gdb_test_multiple "file $binfile" $test {
+	-re "Are you sure you want to change the file. .*y or n. $" {
+	    send_gdb "y\n"
+	    exp_continue
+	}
+	-re "Load new symbol table from \".*\".*y or n. $" {
+	    send_gdb "y\n"
+	    exp_continue
+	}
+	-re "Reading symbols from.*done.*$gdb_prompt $" {
+	    pass $test
+	}
+    }
+}
+
+# Set a break/hbreak/watch/rwatch/awatch.
+
+proc set_breakpoint { break_command } {
+    global gdb_prompt srcfile
+
+    if { $break_command == "break" } {
+	gdb_test "$break_command foo" "Breakpoint.*at.* file .*$srcfile, line.*"
+    } elseif { $break_command == "hbreak" } {
+	set test "$break_command foo"
+	gdb_test_multiple $test $test {
+	    -re "No hardware breakpoint support in the target.*$gdb_prompt $" {
+		unsupported $test
+	    }
+	    -re "Hardware breakpoints used exceeds limit.*$gdb_prompt $" {
+		unsupported $test
+	    }
+	    -re "Cannot insert hardware breakpoint.*$gdb_prompt $" {
+		unsupported $test
+	    }
+	    -re "Hardware assisted breakpoint.*at.* file .*$srcfile, line.*$gdb_prompt $" {
+		pass $test
+	    }
+	}
+    } elseif { [string first "watch" $break_command] != -1 } {
+	set test "$break_command global"
+	gdb_test_multiple $test $test {
+	    -re "Target does not support this type of hardware watchpoint\\.\r\n$gdb_prompt $" {
+		unsupported $test
+	    }
+	    -re "Could not insert hardware watchpoint.*$gdb_prompt $" {
+		unsupported $test
+	    }
+	    -re "atchpoint \[0-9\]+: global\r\n$gdb_prompt $" {
+		pass $test
+	    }
+	}
+    } else {
+	error "unhandled command: $break_command"
+    }
+}
+
+# Run the test proper.  ALWAYS_INSERT determines whether
+# always-inserted mode is on/off, and BREAK_COMMAND is the
+# break/watch/etc. command being tested.
+#
+proc test_break { always_inserted break_command } {
+    set cmd [lindex [split "$break_command"] 0]
+
+    with_test_prefix "always-inserted $always_inserted: $cmd" {
+	delete_breakpoints
+
+	if ![runto_main] then {
+	    fail "Can't run to main"
+	    return
+	}
+
+	gdb_test_no_output "set breakpoint always-inserted $always_inserted"
+
+	# Set breakpoints/watchpoints twice.  With always-inserted on,
+	# GDB reinserts the exact same Z breakpoint twice...  Do this
+	# to make sure the stub pays attention to idempotency even
+	# when the condition doesn't change.  If GDB end up optimizing
+	# out exact duplicate packets, we should come up with a way to
+	# keep testing this case.
+	foreach iter { "once" "twice" } {
+	    with_test_prefix $iter {
+		set_breakpoint $break_command
+	    }
+	}
+
+	# Force a breakpoint re-set.  In always-inserted mode, this
+	# makes GDB re-send Z packets too...
+	force_breakpoint_re_set
+
+	# Now really change the condition, which forces a reinsert by
+	# design.
+	gdb_test "condition \$bpnum cond_global == 0" ".*"
+
+	# Now delete breakpoints, and let the program execute the
+	# address where the breakpoint used to be set.  If the target
+	# doesn't treat insertions an idempotent way, we'll get a
+	# spurious SIGTRAP.
+	delete_breakpoints
+	gdb_test "b bar" "Breakpoint .* at .*"
+	gdb_test "continue" "Breakpoint .*, bar .*"
+    }
+}
+
+foreach always_inserted { "off" "on" } {
+    test_break $always_inserted "break"
+
+    if {![skip_hw_breakpoint_tests]} {
+	test_break $always_inserted "hbreak"
+    }
+
+    if {![skip_hw_watchpoint_tests]} {
+	test_break $always_inserted "watch"
+    }
+
+    if {![skip_hw_watchpoint_access_tests]} {
+	test_break $always_inserted "rwatch"
+	test_break $always_inserted "awatch"
+    }
+}