tailc: Handle musttail in case of non-cleaned-up cleanups, especially ASan related [PR120608]

The following testcases FAIL at -O0 -fsanitize=address.  The problem is
we end up with something like
  _26 = foo (x_24(D)); [must tail call]
  // predicted unlikely by early return (on trees) predictor.
  finally_tmp.3_27 = 0;
  goto <bb 5>; [INV]
...
  <bb 5> :
  # _6 = PHI <_26(3), _23(D)(4)>
  # finally_tmp.3_8 = PHI <finally_tmp.3_27(3), finally_tmp.3_22(4)>
  .ASAN_MARK (POISON, &c, 4);
  if (finally_tmp.3_8 == 1)
    goto <bb 7>; [INV]
  else
    goto <bb 6>; [INV]

  <bb 6> :
<L4>:
  finally_tmp.4_31 = 0;
  goto <bb 8>; [INV]
...
  <bb 8> :
  # finally_tmp.4_9 = PHI <finally_tmp.4_31(6), finally_tmp.4_30(7)>
  .ASAN_MARK (POISON, &b, 4);
  if (finally_tmp.4_9 == 1)
    goto <bb 9>; [INV]
  else
    goto <bb 10>; [INV]
...
  <bb 10> :
  # _7 = PHI <_6(8), _34(9)>
  .ASAN_MARK (POISON, &a, 4);

  <bb 11> :
<L11>:
  return _7;
before the sanopt pass.  This is -O0, we don't try to do forward
propagation, jump threading etc.  And what is worse, the sanopt
pass lowers the .ASAN_MARK calls that the tailc/musttail passes
already handle into somewthing that they can't easily pattern match.

The following patch fixes that by
1) moving the musttail pass 2 passes earlier (this is mostly just
   for -O0/-Og, for normal optimization levels musttail calls are
   handled in the tailc pass), i.e. across the sanopt and cleanup_eh
   passes
2) recognizes these finally_tmp SSA_NAME assignments, PHIs using those
   and GIMPLE_CONDs deciding based on those both on the backwards
   walk (when we start from the edges to EXIT) and forwards walk
   (when we find a candidate tail call and process assignments
   after those up to the return statement).  For backwards walk,
   ESUCC argument has been added which is either NULL for the
   noreturn musttail case, or the succ edge through which we've
   reached bb and if it sees GIMPLE_COND with such comparison,
   based on the ESUCC and comparison it will remember which later
   edges to ignore later on and which bb must be walked up to the
   start during tail call discovery (the one with the PHI).
3) the move of musttail pass across cleanup_eh pass resulted in
   g++.dg/opt/pr119613.C regressions but moving cleanup_eh before
   sanopt doesn't work too well, so I've extended
   empty_eh_cleanup to also handle resx which doesn't throw
   externally

I know moving a pass on release branches feels risky, though the
musttail pass is only relevant to functions with musttail calls,
so something quite rare and only at -O0/-Og (unless one e.g.
disables the tailc pass).

2025-07-01  Jakub Jelinek  <jakub@redhat.com>

	PR middle-end/120608
	* passes.def (pass_musttail): Move before pass_sanopt.
	* tree-tailcall.cc (empty_eh_cleanup): Handle GIMPLE_RESX
	which doesn't throw externally through recursion on single
	eh edge (if any and cnt still allows that).
	(find_tail_calls): Add ESUCC, IGNORED_EDGES and MUST_SEE_BBS
	arguments.  Handle GIMPLE_CONDs for non-simplified cleanups with
	finally_tmp temporaries both on backward and forward walks, adjust
	recursive call.
	(tree_optimize_tail_calls_1): Adjust find_tail_calls callers.

	* c-c++-common/asan/pr120608-3.c: New test.
	* c-c++-common/asan/pr120608-4.c: New test.
	* g++.dg/asan/pr120608-3.C: New test.
	* g++.dg/asan/pr120608-4.C: New test.

0 files changed, 0 insertions, 0 deletions