preprocessor: C++ module-directives

C++20 modules introduces a new kind of preprocessor directive -- a
module directive.  These are directives but without the leading '#'.
We have to detect them by sniffing the start of a logical line.  When
detected we replace the initial identifiers with unspellable tokens
and pass them through to the language parser the same way deferred
pragmas are.  There's a PRAGMA_EOL at the logical end of line too.

One additional complication is that we have to do header-name lexing
after the initial tokens, and that requires changes in the macro-aware
piece of the preprocessor.  The above sniffer sets a counter in the
lexer state, and that triggers at the appropriate point.  We then do
the same header-name lexing that occurs on a #include directive or
has_include pseudo-macro.  Except that the header name ends up in the
token stream.

A couple of token emitters need to deal with the new token possibility.

	gcc/c-family/
	* c-lex.c (c_lex_with_flags): CPP_HEADER_NAMEs can now be seen.
	libcpp/
	* include/cpplib.h (struct cpp_options): Add module_directives
	option.
	(NODE_MODULE): New node flag.
	(struct cpp_hashnode): Make rid-code a bitfield, increase bits in
	flags and swap with type field.
	* init.c (post_options): Create module-directive identifier nodes.
	* internal.h (struct lexer_state): Add directive_file_token &
	n_modules fields.  Add module node enumerator.
	* lex.c (cpp_maybe_module_directive): New.
	(_cpp_lex_token): Call it.
	(cpp_output_token): Add '"' around CPP_HEADER_NAME token.
	(do_peek_ident, do_peek_module): New.
	(cpp_directives_only): Detect module-directive lines.
	* macro.c (cpp_get_token_1): Deal with directive_file_token
	triggering.

1 files changed, 23 insertions, 0 deletions

diff --git a/libcpp/init.c b/libcpp/init.c
index 76882bc..fc82658 100644
--- a/libcpp/init.c
+++ b/libcpp/init.c
@@ -843,4 +843,27 @@ post_options (cpp_reader *pfile)
       CPP_OPTION (pfile, trigraphs) = 0;
       CPP_OPTION (pfile, warn_trigraphs) = 0;
     }
+
+  if (CPP_OPTION (pfile, module_directives))
+    {
+      /* These unspellable tokens have a leading space.  */
+      const char *const inits[spec_nodes::M_HWM]
+	= {"export ", "module ", "import ", "__import"};
+
+      for (int ix = 0; ix != spec_nodes::M_HWM; ix++)
+	{
+	  cpp_hashnode *node = cpp_lookup (pfile, UC (inits[ix]),
+					   strlen (inits[ix]));
+
+	  /* Token we pass to the compiler.  */
+	  pfile->spec_nodes.n_modules[ix][1] = node;
+
+	  if (ix != spec_nodes::M__IMPORT)
+	    /* Token we recognize when lexing, drop the trailing ' '.  */
+	    node = cpp_lookup (pfile, NODE_NAME (node), NODE_LEN (node) - 1);
+
+	  node->flags |= NODE_MODULE;
+	  pfile->spec_nodes.n_modules[ix][0] = node;
+	}
+    }
 }