gccrs: Normalize all identifier tokens

gcc/rust/ChangeLog:

	* lex/rust-lex.cc (assert_source_content): Fix namespace specifier
	(test_buffer_input_source): Likewise.
	(test_file_input_source): Likewise.
	* lex/rust-lex.h: Move InputSource ...
	* lex/rust-input-source.h: ... to here. (New file)
	* lex/rust-token.cc (nfc_normalize_token_string): New function
	* lex/rust-token.h (nfc_normalize_token_string): New function
	* rust-lang.cc (run_rust_tests): Modify order of selftests.
	* rust-session-manager.cc (validate_crate_name): Modify interface of Utf8String.
	* util/rust-unicode.cc (lookup_cc): Modify codepoint_t typedef.
	(lookup_recomp): Likewise.
	(recursive_decomp_cano): Likewise.
	(decomp_cano): Likewise.
	(sort_cano): Likewise.
	(compose_hangul): Likewise.
	(assert_normalize): Likewise.
	(Utf8String::nfc_normalize): New function.
	* util/rust-unicode.h: Modify interface of Utf8String.

gcc/testsuite/ChangeLog:

	* rust/compile/unicode_norm1.rs: New test.

Signed-off-by: Raiki Tamura <tamaron1203@gmail.com>

1 files changed, 4 insertions, 5 deletions

diff --git a/gcc/rust/rust-session-manager.cc b/gcc/rust/rust-session-manager.cc
index 401133c..55f6ec5 100644
--- a/gcc/rust/rust-session-manager.cc
+++ b/gcc/rust/rust-session-manager.cc
@@ -115,16 +115,15 @@ infer_crate_name (const std::string &filename)
 static bool
 validate_crate_name (const std::string &crate_name, Error &error)
 {
-  Utf8String utf8_name = {crate_name};
-  tl::optional<std::vector<Codepoint>> uchars_opt = utf8_name.get_chars ();
-
-  if (!uchars_opt.has_value ())
+  tl::optional<Utf8String> utf8_name_opt
+    = Utf8String::make_utf8_string (crate_name);
+  if (!utf8_name_opt.has_value ())
     {
       error = Error (UNDEF_LOCATION, "crate name is not a valid UTF-8 string");
       return false;
     }
 
-  std::vector<Codepoint> uchars = uchars_opt.value ();
+  std::vector<Codepoint> uchars = utf8_name_opt->get_chars ();
   if (uchars.empty ())
     {
       error = Error (UNDEF_LOCATION, "crate name cannot be empty");