libsanitizer: Add hwasan pass and associated gimple changes

There are four main features to this change:

1) Check pointer tags match address tags.

When sanitizing for hwasan we now put HWASAN_CHECK internal functions before
memory accesses in the `asan` pass.  This checks that a tag in the pointer
being used match the tag stored in shadow memory for the memory region being
used.

These internal functions are expanded into actual checks in the sanopt
pass that happens just before expansion into RTL.

We use the same mechanism that currently inserts ASAN_CHECK internal
functions to insert the new HWASAN_CHECK functions.

2) Instrument known builtin function calls.

Handle all builtin functions that we know use memory accesses.
This commit uses the machinery added for ASAN to identify builtin
functions that access memory.

The main differences between the approaches for HWASAN and ASAN are:
 - libhwasan intercepts much less builtin functions.
 - Alloca needs to be transformed differently (instead of adding
   redzones it needs to tag shadow memory and return a tagged pointer).
 - stack_restore needs to untag the shadow stack between the current
   position and where it's going.
 - `noreturn` functions can not be handled by simply unpoisoning the
   entire shadow stack -- there is no "always valid" tag.
   (exceptions and things such as longjmp need to be handled in a
   different way, usually in the runtime).

For hardware implemented checking (such as AArch64's memory tagging
extension) alloca and stack_restore will need to be handled by hooks in
the backend rather than transformation at the gimple level.  This will
allow architecture specific handling of such stack modifications.

3) Introduce HWASAN block-scope poisoning

Here we use exactly the same mechanism as ASAN_MARK to poison/unpoison
variables on entry/exit of a block.

In order to simply use the exact same machinery we're using the same
internal functions until the SANOPT pass.  This means that all handling
of ASAN_MARK is the same.
This has the negative that the naming may be a little confusing, but a
positive that handling of the internal function doesn't have to be
duplicated for a function that behaves exactly the same but has a
different name.

gcc/ChangeLog:

	* asan.c (asan_instrument_reads): New.
	(asan_instrument_writes): New.
	(asan_memintrin): New.
	(handle_builtin_stack_restore): Account for HWASAN.
	(handle_builtin_alloca): Account for HWASAN.
	(get_mem_refs_of_builtin_call): Special case strlen for HWASAN.
	(hwasan_instrument_reads): New.
	(hwasan_instrument_writes): New.
	(hwasan_memintrin): New.
	(report_error_func): Assert not HWASAN.
	(build_check_stmt): Make HWASAN_CHECK instead of ASAN_CHECK.
	(instrument_derefs): HWASAN does not tag globals.
	(instrument_builtin_call): Use new helper functions.
	(maybe_instrument_call): Don't instrument `noreturn` functions.
	(initialize_sanitizer_builtins): Add new type.
	(asan_expand_mark_ifn): Account for HWASAN.
	(asan_expand_check_ifn): Assert never called by HWASAN.
	(asan_expand_poison_ifn): Account for HWASAN.
	(asan_instrument): Branch based on whether using HWASAN or ASAN.
	(pass_asan::gate): Return true if sanitizing HWASAN.
	(pass_asan_O0::gate): Return true if sanitizing HWASAN.
	(hwasan_check_func): New.
	(hwasan_expand_check_ifn): New.
	(hwasan_expand_mark_ifn): New.
	(gate_hwasan): New.
	* asan.h (hwasan_expand_check_ifn): New decl.
	(hwasan_expand_mark_ifn): New decl.
	(gate_hwasan): New decl.
	(asan_intercepted_p): Always false for hwasan.
	(asan_sanitize_use_after_scope): Account for HWASAN.
	* builtin-types.def (BT_FN_PTR_CONST_PTR_UINT8): New.
	* gimple-fold.c (gimple_build): New overload for building function
	calls without arguments.
	(gimple_build_round_up): New.
	* gimple-fold.h (gimple_build): New decl.
	(gimple_build): New inline function.
	(gimple_build_round_up): New decl.
	(gimple_build_round_up): New inline function.
	* gimple-pretty-print.c (dump_gimple_call_args): Account for
	HWASAN.
	* gimplify.c (asan_poison_variable): Account for HWASAN.
	(gimplify_function_tree): Remove requirement of
	SANITIZE_ADDRESS, requiring asan or hwasan is accounted for in
	`asan_sanitize_use_after_scope`.
	* internal-fn.c (expand_HWASAN_CHECK): New.
	(expand_HWASAN_ALLOCA_UNPOISON): New.
	(expand_HWASAN_CHOOSE_TAG): New.
	(expand_HWASAN_MARK): New.
	(expand_HWASAN_SET_TAG): New.
	* internal-fn.def (HWASAN_ALLOCA_UNPOISON): New.
	(HWASAN_CHOOSE_TAG): New.
	(HWASAN_CHECK): New.
	(HWASAN_MARK): New.
	(HWASAN_SET_TAG): New.
	* sanitizer.def (BUILT_IN_HWASAN_LOAD1): New.
	(BUILT_IN_HWASAN_LOAD2): New.
	(BUILT_IN_HWASAN_LOAD4): New.
	(BUILT_IN_HWASAN_LOAD8): New.
	(BUILT_IN_HWASAN_LOAD16): New.
	(BUILT_IN_HWASAN_LOADN): New.
	(BUILT_IN_HWASAN_STORE1): New.
	(BUILT_IN_HWASAN_STORE2): New.
	(BUILT_IN_HWASAN_STORE4): New.
	(BUILT_IN_HWASAN_STORE8): New.
	(BUILT_IN_HWASAN_STORE16): New.
	(BUILT_IN_HWASAN_STOREN): New.
	(BUILT_IN_HWASAN_LOAD1_NOABORT): New.
	(BUILT_IN_HWASAN_LOAD2_NOABORT): New.
	(BUILT_IN_HWASAN_LOAD4_NOABORT): New.
	(BUILT_IN_HWASAN_LOAD8_NOABORT): New.
	(BUILT_IN_HWASAN_LOAD16_NOABORT): New.
	(BUILT_IN_HWASAN_LOADN_NOABORT): New.
	(BUILT_IN_HWASAN_STORE1_NOABORT): New.
	(BUILT_IN_HWASAN_STORE2_NOABORT): New.
	(BUILT_IN_HWASAN_STORE4_NOABORT): New.
	(BUILT_IN_HWASAN_STORE8_NOABORT): New.
	(BUILT_IN_HWASAN_STORE16_NOABORT): New.
	(BUILT_IN_HWASAN_STOREN_NOABORT): New.
	(BUILT_IN_HWASAN_TAG_MISMATCH4): New.
	(BUILT_IN_HWASAN_HANDLE_LONGJMP): New.
	(BUILT_IN_HWASAN_TAG_PTR): New.
	* sanopt.c (sanopt_optimize_walker): Act for hwasan.
	(pass_sanopt::execute): Act for hwasan.
	* toplev.c (compile_file): Use `gate_hwasan` function.

1 files changed, 17 insertions, 4 deletions

diff --git a/gcc/sanopt.c b/gcc/sanopt.c
index 6c3bce9..965ab36 100644
--- a/gcc/sanopt.c
+++ b/gcc/sanopt.c
@@ -776,7 +776,8 @@ sanopt_optimize_walker (basic_block bb, class sanopt_ctx *ctx)
   basic_block son;
   gimple_stmt_iterator gsi;
   sanopt_info *info = (sanopt_info *) bb->aux;
-  bool asan_check_optimize = (flag_sanitize & SANITIZE_ADDRESS) != 0;
+  bool asan_check_optimize
+    = ((flag_sanitize & (SANITIZE_ADDRESS | SANITIZE_HWADDRESS)) != 0);
 
   for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi);)
     {
@@ -806,6 +807,7 @@ sanopt_optimize_walker (basic_block bb, class sanopt_ctx *ctx)
       if (asan_check_optimize
 	  && gimple_call_builtin_p (stmt, BUILT_IN_ASAN_BEFORE_DYNAMIC_INIT))
 	{
+	  gcc_assert (!hwasan_sanitize_p ());
 	  use_operand_p use;
 	  gimple *use_stmt;
 	  if (single_imm_use (gimple_vdef (stmt), &use, &use_stmt))
@@ -834,6 +836,7 @@ sanopt_optimize_walker (basic_block bb, class sanopt_ctx *ctx)
 	  case IFN_UBSAN_PTR:
 	    remove = maybe_optimize_ubsan_ptr_ifn (ctx, stmt);
 	    break;
+	  case IFN_HWASAN_CHECK:
 	  case IFN_ASAN_CHECK:
 	    if (asan_check_optimize)
 	      remove = maybe_optimize_asan_check_ifn (ctx, stmt);
@@ -1262,6 +1265,10 @@ sanitize_rewrite_addressable_params (function *fun)
 unsigned int
 pass_sanopt::execute (function *fun)
 {
+  /* n.b. ASAN_MARK is used for both HWASAN and ASAN.
+     asan_num_accesses is hence used to count either HWASAN_CHECK or ASAN_CHECK
+     stuff.  This is fine because you can only have one of these active at a
+     time.  */
   basic_block bb;
   int asan_num_accesses = 0;
   bool contains_asan_mark = false;
@@ -1269,10 +1276,10 @@ pass_sanopt::execute (function *fun)
   /* Try to remove redundant checks.  */
   if (optimize
       && (flag_sanitize
-	  & (SANITIZE_NULL | SANITIZE_ALIGNMENT
+	  & (SANITIZE_NULL | SANITIZE_ALIGNMENT | SANITIZE_HWADDRESS
 	     | SANITIZE_ADDRESS | SANITIZE_VPTR | SANITIZE_POINTER_OVERFLOW)))
     asan_num_accesses = sanopt_optimize (fun, &contains_asan_mark);
-  else if (flag_sanitize & SANITIZE_ADDRESS)
+  else if (flag_sanitize & (SANITIZE_ADDRESS | SANITIZE_HWADDRESS))
     {
       gimple_stmt_iterator gsi;
       FOR_EACH_BB_FN (bb, fun)
@@ -1292,7 +1299,7 @@ pass_sanopt::execute (function *fun)
       sanitize_asan_mark_poison ();
     }
 
-  if (asan_sanitize_stack_p ())
+  if (asan_sanitize_stack_p () || hwasan_sanitize_stack_p ())
     sanitize_rewrite_addressable_params (fun);
 
   bool use_calls = param_asan_instrumentation_with_call_threshold < INT_MAX
@@ -1334,6 +1341,9 @@ pass_sanopt::execute (function *fun)
 		case IFN_UBSAN_VPTR:
 		  no_next = ubsan_expand_vptr_ifn (&gsi);
 		  break;
+		case IFN_HWASAN_CHECK:
+		  no_next = hwasan_expand_check_ifn (&gsi, use_calls);
+		  break;
 		case IFN_ASAN_CHECK:
 		  no_next = asan_expand_check_ifn (&gsi, use_calls);
 		  break;
@@ -1345,6 +1355,9 @@ pass_sanopt::execute (function *fun)
 						    &need_commit_edge_insert,
 						    shadow_vars_mapping);
 		  break;
+		case IFN_HWASAN_MARK:
+		  no_next = hwasan_expand_mark_ifn (&gsi);
+		  break;
 		default:
 		  break;
 		}