ubsan: default to trap on unreachable at -O0 and -Og [PR104642]

When not optimizing, we can't do anything useful with unreachability in
terms of code performance, so we might as well improve debugging by turning
__builtin_unreachable into a trap.  I think it also makes sense to do this
when we're explicitly optimizing for the debugging experience.

In the PR richi suggested introducing an -funreachable-traps flag for this.
This functionality is already implemented as -fsanitize=unreachable
-fsanitize-trap=unreachable, and we want to share the implementation, but it
does seem useful to have a separate flag that isn't affected by the various
sanitization controls.  -fsanitize=unreachable takes priority over
-funreachable-traps if both are enabled.

Jakub observed that this would slow down -O0 by default from running the
sanopt pass, so this revision avoids the need for sanopt by rewriting calls
introduced by the compiler immediately, and calls written by the user at
fold time.  Many of the calls introduced by the compiler are also rewritten
immediately to ubsan calls when not trapping, which fixes ubsan-8b.C;
previously the call to f() was optimized away before sanopt.  But this early
rewriting isn't practical for uses of __builtin_unreachable in
devirtualization and such, so sanopt rewriting is still done for
non-trapping sanitize.

	PR c++/104642

gcc/ChangeLog:

	* common.opt: Add -funreachable-traps.
	* doc/invoke.texi (-funreachable-traps): Document it.
	* opts.cc (finish_options): Enable at -O0 or -Og.
	* tree.cc (build_common_builtin_nodes): Add __builtin_trap.
	(builtin_decl_unreachable, build_builtin_unreachable): New.
	* tree.h: Declare them.
	* ubsan.cc (sanitize_unreachable_fn): Factor out.
	(ubsan_instrument_unreachable): Use
	gimple_build_builtin_unreachable.
	* ubsan.h (sanitize_unreachable_fn): Declare.
	* gimple.cc (gimple_build_builtin_unreachable): New.
	* gimple.h: Declare it.
	* builtins.cc (expand_builtin_unreachable): Add assert.
	(fold_builtin_0): Call build_builtin_unreachable.
	* sanopt.cc: Don't run for just SANITIZE_RETURN
	or SANITIZE_UNREACHABLE when trapping.
	* cgraphunit.cc (walk_polymorphic_call_targets): Use new
	unreachable functions.
	* gimple-fold.cc (gimple_fold_call)
	(gimple_get_virt_method_for_vtable)
	* ipa-fnsummary.cc (redirect_to_unreachable)
	* ipa-prop.cc (ipa_make_edge_direct_to_target)
	(ipa_impossible_devirt_target)
	* ipa.cc (walk_polymorphic_call_targets)
	* tree-cfg.cc (pass_warn_function_return::execute)
	(execute_fixup_cfg)
	* tree-ssa-loop-ivcanon.cc (remove_exits_and_undefined_stmts)
	(unloop_loops)
	* tree-ssa-sccvn.cc (eliminate_dom_walker::eliminate_stmt):
	Likewise.

gcc/cp/ChangeLog:

	* constexpr.cc (cxx_eval_builtin_function_call): Handle
	unreachable/trap earlier.
	* cp-gimplify.cc (cp_maybe_instrument_return): Use
	build_builtin_unreachable.

gcc/testsuite/ChangeLog:

	* g++.dg/ubsan/return-8a.C: New test.
	* g++.dg/ubsan/return-8b.C: New test.
	* g++.dg/ubsan/return-8d.C: New test.
	* g++.dg/ubsan/return-8e.C: New test.

1 files changed, 17 insertions, 12 deletions

diff --git a/gcc/cp/constexpr.cc b/gcc/cp/constexpr.cc
index fd7f8c0..0dc94d9 100644
--- a/gcc/cp/constexpr.cc
+++ b/gcc/cp/constexpr.cc
@@ -1438,6 +1438,20 @@ cxx_eval_builtin_function_call (const constexpr_ctx *ctx, tree t, tree fun,
 	/* These builtins shall be ignored during constant expression
 	   evaluation.  */
 	return void_node;
+      case BUILT_IN_UNREACHABLE:
+      case BUILT_IN_TRAP:
+	if (!*non_constant_p && !ctx->quiet)
+	  {
+	    /* Do not allow__builtin_unreachable in constexpr function.
+	       The __builtin_unreachable call with BUILTINS_LOCATION
+	       comes from cp_maybe_instrument_return.  */
+	    if (EXPR_LOCATION (t) == BUILTINS_LOCATION)
+	      error ("%<constexpr%> call flows off the end of the function");
+	    else
+	      error ("%q+E is not a constant expression", t);
+	  }
+	*non_constant_p = true;
+	return t;
       default:
 	break;
       }
@@ -1531,18 +1545,9 @@ cxx_eval_builtin_function_call (const constexpr_ctx *ctx, tree t, tree fun,
     {
       if (!*non_constant_p && !ctx->quiet)
 	{
-	  /* Do not allow__builtin_unreachable in constexpr function.
-	     The __builtin_unreachable call with BUILTINS_LOCATION
-	     comes from cp_maybe_instrument_return.  */
-	  if (fndecl_built_in_p (fun, BUILT_IN_UNREACHABLE)
-	      && EXPR_LOCATION (t) == BUILTINS_LOCATION)
-	    error ("%<constexpr%> call flows off the end of the function");
-	  else
-	    {
-	      new_call = build_call_array_loc (EXPR_LOCATION (t), TREE_TYPE (t),
-					       CALL_EXPR_FN (t), nargs, args);
-	      error ("%q+E is not a constant expression", new_call);
-	    }
+	  new_call = build_call_array_loc (EXPR_LOCATION (t), TREE_TYPE (t),
+					   CALL_EXPR_FN (t), nargs, args);
+	  error ("%q+E is not a constant expression", new_call);
 	}
       *non_constant_p = true;
       return t;