ubsan: default to trap on unreachable at -O0 and -Og [PR104642]

When not optimizing, we can't do anything useful with unreachability in
terms of code performance, so we might as well improve debugging by turning
__builtin_unreachable into a trap.  I think it also makes sense to do this
when we're explicitly optimizing for the debugging experience.

In the PR richi suggested introducing an -funreachable-traps flag for this.
This functionality is already implemented as -fsanitize=unreachable
-fsanitize-trap=unreachable, and we want to share the implementation, but it
does seem useful to have a separate flag that isn't affected by the various
sanitization controls.  -fsanitize=unreachable takes priority over
-funreachable-traps if both are enabled.

Jakub observed that this would slow down -O0 by default from running the
sanopt pass, so this revision avoids the need for sanopt by rewriting calls
introduced by the compiler immediately, and calls written by the user at
fold time.  Many of the calls introduced by the compiler are also rewritten
immediately to ubsan calls when not trapping, which fixes ubsan-8b.C;
previously the call to f() was optimized away before sanopt.  But this early
rewriting isn't practical for uses of __builtin_unreachable in
devirtualization and such, so sanopt rewriting is still done for
non-trapping sanitize.

	PR c++/104642

gcc/ChangeLog:

	* common.opt: Add -funreachable-traps.
	* doc/invoke.texi (-funreachable-traps): Document it.
	* opts.cc (finish_options): Enable at -O0 or -Og.
	* tree.cc (build_common_builtin_nodes): Add __builtin_trap.
	(builtin_decl_unreachable, build_builtin_unreachable): New.
	* tree.h: Declare them.
	* ubsan.cc (sanitize_unreachable_fn): Factor out.
	(ubsan_instrument_unreachable): Use
	gimple_build_builtin_unreachable.
	* ubsan.h (sanitize_unreachable_fn): Declare.
	* gimple.cc (gimple_build_builtin_unreachable): New.
	* gimple.h: Declare it.
	* builtins.cc (expand_builtin_unreachable): Add assert.
	(fold_builtin_0): Call build_builtin_unreachable.
	* sanopt.cc: Don't run for just SANITIZE_RETURN
	or SANITIZE_UNREACHABLE when trapping.
	* cgraphunit.cc (walk_polymorphic_call_targets): Use new
	unreachable functions.
	* gimple-fold.cc (gimple_fold_call)
	(gimple_get_virt_method_for_vtable)
	* ipa-fnsummary.cc (redirect_to_unreachable)
	* ipa-prop.cc (ipa_make_edge_direct_to_target)
	(ipa_impossible_devirt_target)
	* ipa.cc (walk_polymorphic_call_targets)
	* tree-cfg.cc (pass_warn_function_return::execute)
	(execute_fixup_cfg)
	* tree-ssa-loop-ivcanon.cc (remove_exits_and_undefined_stmts)
	(unloop_loops)
	* tree-ssa-sccvn.cc (eliminate_dom_walker::eliminate_stmt):
	Likewise.

gcc/cp/ChangeLog:

	* constexpr.cc (cxx_eval_builtin_function_call): Handle
	unreachable/trap earlier.
	* cp-gimplify.cc (cp_maybe_instrument_return): Use
	build_builtin_unreachable.

gcc/testsuite/ChangeLog:

	* g++.dg/ubsan/return-8a.C: New test.
	* g++.dg/ubsan/return-8b.C: New test.
	* g++.dg/ubsan/return-8d.C: New test.
	* g++.dg/ubsan/return-8e.C: New test.

1 files changed, 14 insertions, 0 deletions

diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index fb50636..e7526c5 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -12939,6 +12939,20 @@ also at @option{-O0} if @option{-fsection-anchors} is explicitly requested.
 Additionally @option{-fno-toplevel-reorder} implies
 @option{-fno-section-anchors}.
 
+@item -funreachable-traps
+@opindex funreachable-traps
+With this option, the compiler turns calls to
+@code{__builtin_unreachable} into traps, instead of using them for
+optimization.  This also affects any such calls implicitly generated
+by the compiler.
+
+This option has the same effect as @option{-fsanitize=unreachable
+-fsanitize-trap=unreachable}, but does not affect the values of those
+options.  If @option{-fsanitize=unreachable} is enabled, that option
+takes priority over this one.
+
+This option is enabled by default at @option{-O0} and @option{-Og}.
+
 @item -fweb
 @opindex fweb
 Constructs webs as commonly used for register allocation purposes and assign