preprocessor: C2x identifier rules

C2x has, like C++, adopted rules for identifiers based directly on an
unversioned normative reference to Unicode.  Make libcpp follow those
rules for c2x / gnu2x standards (this involves bringing back a flag
separate from the C++ one for whether to use these identifier rules,
but this time enabled for all C++ language versions since that was the
conclusion adopted for C++ identifier handling).

There is one change here that affects C++.  I believe the new
normative requirement for NFC only applies to identifiers, not to the
use of identifier-continue characters in pp-numbers, where there is no
such requirement and so the diagnostic ought to be a warning not a
pedwarn in pp-numbers, and that this is the case for both C and C++.

Bootstrapped with no regressions for x86_64-pc-linux-gnu.

libcpp/
	* charset.cc (ucn_valid_in_identifier): Check xid_identifiers not
	cplusplus to determine whether to use CXX23 and NXX23 flags.
	* include/cpplib.h (struct cpp_options): Add xid_identifiers.
	* init.cc (struct lang_flags, lang_defaults): Add xid_identifiers.
	(cpp_set_lang): Set xid_identifiers.
	* lex.cc (warn_about_normalization): Add parameter identifier.
	Only pedwarn about non-NFC for identifiers, not pp-numbers.
	(_cpp_lex_direct): Update calls to warn_about_normalization.

gcc/testsuite/
	* gcc.dg/cpp/c2x-ucnid-1-utf8.c, gcc.dg/cpp/c2x-ucnid-1.c: New
	tests.

1 files changed, 2 insertions, 2 deletions

diff --git a/libcpp/charset.cc b/libcpp/charset.cc
index 6834969..12a398e 100644
--- a/libcpp/charset.cc
+++ b/libcpp/charset.cc
@@ -1291,7 +1291,7 @@ ucn_valid_in_identifier (cpp_reader *pfile, cppchar_t c,
   valid_flags = C99 | CXX | C11 | CXX23;
   if (CPP_PEDANTIC (pfile))
     {
-      if (CPP_OPTION (pfile, cplusplus))
+      if (CPP_OPTION (pfile, xid_identifiers))
 	valid_flags = CXX23;
       else if (CPP_OPTION (pfile, c11_identifiers))
 	valid_flags = C11;
@@ -1355,7 +1355,7 @@ ucn_valid_in_identifier (cpp_reader *pfile, cppchar_t c,
       return 2;
     }
 
-  if (CPP_OPTION (pfile, cplusplus))
+  if (CPP_OPTION (pfile, xid_identifiers))
     invalid_start_flags = NXX23;
   else if (CPP_OPTION (pfile, c11_identifiers))
     invalid_start_flags = N11;