libcpp: Fix up #__VA_OPT__ handling [PR103415]

stringify_arg uses pfile->u_buff to create the string literal.
Unfortunately, paste_tokens -> _cpp_lex_direct -> lex_number -> _cpp_unaligned_alloc
can in some cases use pfile->u_buff too, which results in losing everything
prepared for the string literal until the token pasting.

The following patch fixes that by not calling paste_token during the
construction of the string literal, but doing that before.  All the tokens
we are processing have been pushed into a token buffer using
tokens_buff_add_token so it is fine if we paste some of them in that buffer
(successful pasting creates a new token in that buffer), move following
tokens if any to make it contiguous, pop (throw away) the extra tokens at
the end and then do stringify_arg.

Also, paste_tokens now copies over PREV_WHITE and PREV_FALLTHROUGH flags
from the original lhs token to the replacement token.  Copying that way
the PREV_WHITE flag is needed for the #__VA_OPT__ handling and copying
over PREV_FALLTHROUGH fixes the new Wimplicit-fallthrough-38.c test.

2021-12-01  Jakub Jelinek  <jakub@redhat.com>

	PR preprocessor/103415
libcpp/
	* macro.c (stringify_arg): Remove va_opt argument and va_opt handling.
	(paste_tokens): On successful paste or in PREV_WHITE and
	PREV_FALLTHROUGH flags from the *plhs token to the new token.
	(replace_args): Adjust stringify_arg callers.  For #__VA_OPT__,
	perform token pasting in a separate loop before stringify_arg call.
gcc/testsuite/
	* c-c++-common/cpp/va-opt-8.c: New test.
	* c-c++-common/Wimplicit-fallthrough-38.c: New test.

2 files changed, 42 insertions, 0 deletions

diff --git a/gcc/testsuite/c-c++-common/Wimplicit-fallthrough-38.c b/gcc/testsuite/c-c++-common/Wimplicit-fallthrough-38.c
new file mode 100644
index 0000000..80f7d45
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/Wimplicit-fallthrough-38.c
@@ -0,0 +1,24 @@
+/* { dg-do compile } */
+/* { dg-options "-Wimplicit-fallthrough=3" } */
+
+#define FOO \
+int				\
+foo (int a)			\
+{				\
+  switch (a)			\
+    {				\
+    case 1:			\
+      ++a;			\
+      /* FALLTHRU */		\
+    case 2:			\
+      ++a;			\
+      /* FALLTHRU */		\
+    ca##se 3:			\
+      ++a;			\
+    default:			\
+      break;			\
+    }				\
+  return a;			\
+}
+
+FOO
diff --git a/gcc/testsuite/c-c++-common/cpp/va-opt-8.c b/gcc/testsuite/c-c++-common/cpp/va-opt-8.c
new file mode 100644
index 0000000..583ebf0
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cpp/va-opt-8.c
@@ -0,0 +1,18 @@
+/* PR preprocessor/103415 */
+/* { dg-do run } */
+/* { dg-options "-std=gnu99" { target c } } */
+/* { dg-options "-std=c++20" { target c++ } } */
+
+#define n(x, ...) = #__VA_OPT__(x##3)
+#define o(x, ...) #__VA_OPT__(x##__VA_ARGS__##9)
+const char *c n(1 2, 4);
+const char *d = o(5  6, 7	8);
+
+int
+main ()
+{
+  if (__builtin_strcmp (c, "1 23")
+      || __builtin_strcmp (d, "5 67 89"))
+    __builtin_abort ();
+  return 0;
+}