Introduce strub: machine-independent stack scrubbing

This patch adds the strub attribute for function and variable types,
command-line options, passes and adjustments to implement it,
documentation, and tests.

Stack scrubbing is implemented in a machine-independent way: functions
with strub enabled are modified so that they take an extra stack
watermark argument, that they update with their stack use, and the
caller can then zero it out once it regains control, whether by return
or exception.  There are two ways to go about it: at-calls, that
modifies the visible interface (signature) of the function, and
internal, in which the body is moved to a clone, the clone undergoes
the interface change, and the function becomes a wrapper, preserving
its original interface, that calls the clone and then clears the stack
used by it.

Variables can also be annotated with the strub attribute, so that
functions that read from them get stack scrubbing enabled implicitly,
whether at-calls, for functions only usable within a translation unit,
or internal, for functions whose interfaces must not be modified.

There is a strict mode, in which functions that have their stack
scrubbed can only call other functions with stack-scrubbing
interfaces, or those explicitly marked as callable from strub
contexts, so that an entire call chain gets scrubbing, at once or
piecemeal depending on optimization levels.  In the default mode,
relaxed, this requirement is not enforced by the compiler.

The implementation adds two IPA passes, one that assigns strub modes
early on, another that modifies interfaces and adds calls to the
builtins that jointly implement stack scrubbing.  Another builtin,
that obtains the stack pointer, is added for use in the implementation
of the builtins, whether expanded inline or called in libgcc.

There are new command-line options to change operation modes and to
force the feature disabled; it is enabled by default, but it has no
effect and is implicitly disabled if the strub attribute is never
used.  There are also options meant to use for testing the feature,
enabling different strubbing modes for all (viable) functions.


for  gcc/ChangeLog

	* Makefile.in (OBJS): Add ipa-strub.o.
	(GTFILES): Add ipa-strub.cc.
	* builtins.def (BUILT_IN_STACK_ADDRESS): New.
	(BUILT_IN___STRUB_ENTER): New.
	(BUILT_IN___STRUB_UPDATE): New.
	(BUILT_IN___STRUB_LEAVE): New.
	* builtins.cc: Include ipa-strub.h.
	(STACK_STOPS, STACK_UNSIGNED): Define.
	(expand_builtin_stack_address): New.
	(expand_builtin_strub_enter): New.
	(expand_builtin_strub_update): New.
	(expand_builtin_strub_leave): New.
	(expand_builtin): Call them.
	* common.opt (fstrub=*): New options.
	* doc/extend.texi (strub): New type attribute.
	(__builtin_stack_address): New function.
	(Stack Scrubbing): New section.
	* doc/invoke.texi (-fstrub=*): New options.
	(-fdump-ipa-*): New passes.
	* gengtype-lex.l: Ignore multi-line pp-directives.
	* ipa-inline.cc: Include ipa-strub.h.
	(can_inline_edge_p): Test strub_inlinable_to_p.
	* ipa-split.cc: Include ipa-strub.h.
	(execute_split_functions): Test strub_splittable_p.
	* ipa-strub.cc, ipa-strub.h: New.
	* passes.def: Add strub_mode and strub passes.
	* tree-cfg.cc (gimple_verify_flow_info): Note on debug stmts.
	* tree-pass.h (make_pass_ipa_strub_mode): Declare.
	(make_pass_ipa_strub): Declare.
	(make_pass_ipa_function_and_variable_visibility): Fix
	formatting.
	* tree-ssa-ccp.cc (optimize_stack_restore): Keep restores
	before strub leave.
	* attribs.cc: Include ipa-strub.h.
	(decl_attributes): Support applying attributes to function
	type, rather than pointer type, at handler's request.
	(comp_type_attributes): Combine strub_comptypes and target
	comp_type results.
	* doc/tm.texi.in (TARGET_STRUB_USE_DYNAMIC_ARRAY): New.
	(TARGET_STRUB_MAY_USE_MEMSET): New.
	* doc/tm.texi: Rebuilt.
	* cgraph.h (symtab_node::reset): Add preserve_comdat_group
	param, with a default.
	* cgraphunit.cc (symtab_node::reset): Use it.

for  gcc/c-family/ChangeLog

	* c-attribs.cc: Include ipa-strub.h.
	(handle_strub_attribute): New.
	(c_common_attribute_table): Add strub.

for  gcc/ada/ChangeLog

	* gcc-interface/trans.cc: Include ipa-strub.h.
	(gigi): Make internal decls for targets of compiler-generated
	calls strub-callable too.
	(build_raise_check): Likewise.
	* gcc-interface/utils.cc: Include ipa-strub.h.
	(handle_strub_attribute): New.
	(gnat_internal_attribute_table): Add strub.

for  gcc/testsuite/ChangeLog

	* c-c++-common/strub-O0.c: New.
	* c-c++-common/strub-O1.c: New.
	* c-c++-common/strub-O2.c: New.
	* c-c++-common/strub-O2fni.c: New.
	* c-c++-common/strub-O3.c: New.
	* c-c++-common/strub-O3fni.c: New.
	* c-c++-common/strub-Og.c: New.
	* c-c++-common/strub-Os.c: New.
	* c-c++-common/strub-all1.c: New.
	* c-c++-common/strub-all2.c: New.
	* c-c++-common/strub-apply1.c: New.
	* c-c++-common/strub-apply2.c: New.
	* c-c++-common/strub-apply3.c: New.
	* c-c++-common/strub-apply4.c: New.
	* c-c++-common/strub-at-calls1.c: New.
	* c-c++-common/strub-at-calls2.c: New.
	* c-c++-common/strub-defer-O1.c: New.
	* c-c++-common/strub-defer-O2.c: New.
	* c-c++-common/strub-defer-O3.c: New.
	* c-c++-common/strub-defer-Os.c: New.
	* c-c++-common/strub-internal1.c: New.
	* c-c++-common/strub-internal2.c: New.
	* c-c++-common/strub-parms1.c: New.
	* c-c++-common/strub-parms2.c: New.
	* c-c++-common/strub-parms3.c: New.
	* c-c++-common/strub-relaxed1.c: New.
	* c-c++-common/strub-relaxed2.c: New.
	* c-c++-common/strub-short-O0-exc.c: New.
	* c-c++-common/strub-short-O0.c: New.
	* c-c++-common/strub-short-O1.c: New.
	* c-c++-common/strub-short-O2.c: New.
	* c-c++-common/strub-short-O3.c: New.
	* c-c++-common/strub-short-Os.c: New.
	* c-c++-common/strub-strict1.c: New.
	* c-c++-common/strub-strict2.c: New.
	* c-c++-common/strub-tail-O1.c: New.
	* c-c++-common/strub-tail-O2.c: New.
	* c-c++-common/torture/strub-callable1.c: New.
	* c-c++-common/torture/strub-callable2.c: New.
	* c-c++-common/torture/strub-const1.c: New.
	* c-c++-common/torture/strub-const2.c: New.
	* c-c++-common/torture/strub-const3.c: New.
	* c-c++-common/torture/strub-const4.c: New.
	* c-c++-common/torture/strub-data1.c: New.
	* c-c++-common/torture/strub-data2.c: New.
	* c-c++-common/torture/strub-data3.c: New.
	* c-c++-common/torture/strub-data4.c: New.
	* c-c++-common/torture/strub-data5.c: New.
	* c-c++-common/torture/strub-indcall1.c: New.
	* c-c++-common/torture/strub-indcall2.c: New.
	* c-c++-common/torture/strub-indcall3.c: New.
	* c-c++-common/torture/strub-inlinable1.c: New.
	* c-c++-common/torture/strub-inlinable2.c: New.
	* c-c++-common/torture/strub-ptrfn1.c: New.
	* c-c++-common/torture/strub-ptrfn2.c: New.
	* c-c++-common/torture/strub-ptrfn3.c: New.
	* c-c++-common/torture/strub-ptrfn4.c: New.
	* c-c++-common/torture/strub-pure1.c: New.
	* c-c++-common/torture/strub-pure2.c: New.
	* c-c++-common/torture/strub-pure3.c: New.
	* c-c++-common/torture/strub-pure4.c: New.
	* c-c++-common/torture/strub-run1.c: New.
	* c-c++-common/torture/strub-run2.c: New.
	* c-c++-common/torture/strub-run3.c: New.
	* c-c++-common/torture/strub-run4.c: New.
	* c-c++-common/torture/strub-run4c.c: New.
	* c-c++-common/torture/strub-run4d.c: New.
	* c-c++-common/torture/strub-run4i.c: New.
	* g++.dg/strub-run1.C: New.
	* g++.dg/torture/strub-init1.C: New.
	* g++.dg/torture/strub-init2.C: New.
	* g++.dg/torture/strub-init3.C: New.
	* gnat.dg/strub_attr.adb, gnat.dg/strub_attr.ads: New.
	* gnat.dg/strub_ind.adb, gnat.dg/strub_ind.ads: New.

for  libgcc/ChangeLog

	* Makefile.in (LIB2ADD): Add strub.c.
	* libgcc2.h (__strub_enter, __strub_update, __strub_leave):
	Declare.
	* strub.c: New.
	* libgcc-std.ver.in (__strub_enter): Add to GCC_14.0.0.
	(__strub_update, __strub_leave): Likewise.

4 files changed, 159 insertions, 0 deletions

diff --git a/libgcc/Makefile.in b/libgcc/Makefile.in
index 8dedd10..d8163c5 100644
--- a/libgcc/Makefile.in
+++ b/libgcc/Makefile.in
@@ -433,6 +433,9 @@ LIB2ADD += enable-execute-stack.c
 # Control Flow Redundancy hardening out-of-line checker.
 LIB2ADD += $(srcdir)/hardcfr.c
 
+# Stack scrubbing infrastructure.
+LIB2ADD += $(srcdir)/strub.c
+
 # While emutls.c has nothing to do with EH, it is in LIB2ADDEH*
 # instead of LIB2ADD because that's the way to be sure on some targets
 # (e.g. *-*-darwin*) only one copy of it is linked.
diff --git a/libgcc/libgcc-std.ver.in b/libgcc/libgcc-std.ver.in
index de00db6..a6b8963 100644
--- a/libgcc/libgcc-std.ver.in
+++ b/libgcc/libgcc-std.ver.in
@@ -1957,4 +1957,7 @@ GCC_14.0.0 {
   __PFX__floatbitintsf
   __PFX__floatbitintdf
   __PFX__hardcfr_check
+  __PFX__strub_enter
+  __PFX__strub_update
+  __PFX__strub_leave
 }
diff --git a/libgcc/libgcc2.h b/libgcc/libgcc2.h
index 7e6696d..750670e 100644
--- a/libgcc/libgcc2.h
+++ b/libgcc/libgcc2.h
@@ -554,6 +554,10 @@ extern int __parityDI2 (UDWtype);
 
 extern void __enable_execute_stack (void *);
 
+extern void __strub_enter (void **);
+extern void __strub_update (void**);
+extern void __strub_leave (void **);
+
 #ifndef HIDE_EXPORTS
 #pragma GCC visibility pop
 #endif
diff --git a/libgcc/strub.c b/libgcc/strub.c
new file mode 100644
index 0000000..b0f990d
--- /dev/null
+++ b/libgcc/strub.c
@@ -0,0 +1,149 @@
+/* Stack scrubbing infrastructure
+   Copyright (C) 2021-2023 Free Software Foundation, Inc.
+   Contributed by Alexandre Oliva <oliva@adacore.com>
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+Under Section 7 of GPL version 3, you are granted additional
+permissions described in the GCC Runtime Library Exception, version
+3.1, as published by the Free Software Foundation.
+
+You should have received a copy of the GNU General Public License and
+a copy of the GCC Runtime Library Exception along with this program;
+see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
+<http://www.gnu.org/licenses/>.  */
+
+#include "tconfig.h"
+#include "tsystem.h"
+#include "coretypes.h"
+#include "tm.h"
+#include "libgcc_tm.h"
+#include "libgcc2.h"
+
+#if ! STACK_GROWS_DOWNWARD
+# define TOPS >
+#else
+# define TOPS <
+#endif
+
+#define ATTRIBUTE_STRUB_CALLABLE __attribute__ ((__strub__ ("callable")))
+
+/* Enter a stack scrubbing context, initializing the watermark to the caller's
+   stack address.  */
+void ATTRIBUTE_STRUB_CALLABLE
+__strub_enter (void **watermark)
+{
+  *watermark = __builtin_frame_address (0);
+}
+
+/* Update the watermark within a stack scrubbing context with the current stack
+   pointer.  */
+void ATTRIBUTE_STRUB_CALLABLE
+__strub_update (void **watermark)
+{
+  void *sp = __builtin_frame_address (0);
+
+  if (sp TOPS *watermark)
+    *watermark = sp;
+}
+
+#if TARGET_STRUB_USE_DYNAMIC_ARRAY && ! defined TARGET_STRUB_MAY_USE_MEMSET
+# define TARGET_STRUB_MAY_USE_MEMSET 1
+#endif
+
+#if defined __x86_64__ && __OPTIMIZE__
+# define TARGET_STRUB_DISABLE_RED_ZONE \
+  /* __attribute__ ((__target__ ("no-red-zone"))) // not needed when optimizing */
+#elif !defined RED_ZONE_SIZE || defined __i386__
+# define TARGET_STRUB_DISABLE_RED_ZONE
+#endif
+
+#ifndef TARGET_STRUB_DISABLE_RED_ZONE
+/* Dummy function, called to force the caller to not be a leaf function, so
+   that it can't use the red zone.  */
+static void ATTRIBUTE_STRUB_CALLABLE
+__attribute__ ((__noinline__, __noipa__))
+__strub_dummy_force_no_leaf (void)
+{
+}
+#endif
+
+/* Leave a stack scrubbing context, clearing the stack between its top and
+   *MARK.  */
+void ATTRIBUTE_STRUB_CALLABLE
+#if ! TARGET_STRUB_MAY_USE_MEMSET
+__attribute__ ((__optimize__ ("-fno-tree-loop-distribute-patterns")))
+#endif
+#ifdef TARGET_STRUB_DISABLE_RED_ZONE
+TARGET_STRUB_DISABLE_RED_ZONE
+#endif
+__strub_leave (void **mark)
+{
+  void *sp = __builtin_stack_address ();
+
+  void **base, **end;
+#if ! STACK_GROWS_DOWNWARD
+  base = sp; /* ??? Do we need an offset here?  */
+  end = *mark;
+#else
+  base = *mark;
+  end = sp; /* ??? Does any platform require an offset here?  */
+#endif
+
+  if (! (base < end))
+    return;
+
+#if TARGET_STRUB_USE_DYNAMIC_ARRAY
+  /* Compute the length without assuming the pointers are both sufficiently
+     aligned.  They should be, but pointer differences expected to be exact may
+     yield unexpected results when the assumption doesn't hold.  Given the
+     potential security implications, compute the length without that
+     expectation.  If the pointers are misaligned, we may leave a partial
+     unscrubbed word behind.  */
+  ptrdiff_t len = ((char *)end - (char *)base) / sizeof (void *);
+  /* Allocate a dynamically-sized array covering the desired range, so that we
+     can safely call memset on it.  */
+  void *ptr[len];
+  base = &ptr[0];
+  end = &ptr[len];
+#elifndef TARGET_STRUB_DISABLE_RED_ZONE
+  /* Prevent the use of the red zone, by making this function non-leaf through
+     an unreachable call that, because of the asm stmt, the compiler will
+     consider reachable.  */
+  asm goto ("" : : : : no_leaf);
+  if (0)
+    {
+    no_leaf:
+      __strub_dummy_force_no_leaf ();
+      return;
+    }
+#endif
+
+  /* ldist may turn these loops into a memset (thus the conditional
+     -fno-tree-loop-distribute-patterns above).  Without the dynamic array
+     above, that call would likely be unsafe: possibly tail-called, and likely
+     scribbling over its own stack frame.  */
+#if ! STACK_GROWS_DOWNWARD
+  do
+    *base++ = 0;
+  while (base < end);
+  /* Make sure the stack overwrites are not optimized away.  */
+  asm ("" : : "m" (end[0]));
+#else
+  do
+    *--end = 0;
+  while (base < end);
+  /* Make sure the stack overwrites are not optimized away.  */
+  asm ("" : : "m" (base[0]));
+#endif
+}