From 0223119f1a6351543c6e96a9735e05cbd4583889 Mon Sep 17 00:00:00 2001 From: Jakub Jelinek Date: Fri, 6 Dec 2024 09:09:12 +0100 Subject: libcpp, c++: Optimize initializers using #embed in C++ MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This patch adds similar optimizations to the C++ FE as have been implemented earlier in the C FE. The libcpp hunk enables use of CPP_EMBED token even for C++, not just C; the preprocessor guarantees there is always a CPP_NUMBER CPP_COMMA before CPP_EMBED and CPP_COMMA CPP_NUMBER after it which simplifies parsing (unless #embed is more than 2GB, in that case it could be CPP_NUMBER CPP_COMMA CPP_EMBED CPP_COMMA CPP_EMBED CPP_COMMA CPP_EMBED CPP_COMMA CPP_NUMBER etc. with each CPP_EMBED covering at most INT_MAX bytes). Similarly to the C patch, this patch parses it into RAW_DATA_CST tree in the braced initializers (and from there peels into INTEGER_CSTs unless it is an initializer of an std::byte array or integral array with CHAR_BIT element precision), parses CPP_EMBED in cp_parser_expression into just the last INTEGER_CST in it because I think users don't need millions of -Wunused-value warnings because they did useless int a = ( #embed "megabyte.dat" ); and so most of the inner INTEGER_CSTs would be there just for the warning, and in the rest of contexts like template argument list, function argument list, attribute argument list, ...) parse it into a sequence of INTEGER_CSTs (I wrote a range/iterator classes to simplify that). My dumb cat embed-11.c constexpr unsigned char a[] = { #embed "cc1plus" }; const unsigned char *b = a; testcase where cc1plus is 492329008 bytes long when configured --enable-checking=yes,rtl,extra against recent binutils with .base64 gas support results in: time ./xg++ -B ./ -S -O2 embed-11.c real 0m4.350s user 0m2.427s sys 0m0.830s time ./xg++ -B ./ -c -O2 embed-11.c real 0m6.932s user 0m6.034s sys 0m0.888s (compared to running out of memory or very long compilation). On a shorter inclusion, cat embed-12.c constexpr unsigned char a[] = { #embed "xg++" }; const unsigned char *b = a; where xg++ is 15225904 bytes long, this takes using GCC with the #embed patchset except for this patch: time ~/src/gcc/obj36/gcc/xg++ -B ~/src/gcc/obj36/gcc/ -S -O2 embed-12.c real 0m33.190s user 0m32.327s sys 0m0.790s and with this patch: time ./xg++ -B ./ -S -O2 embed-12.c real 0m0.118s user 0m0.090s sys 0m0.028s The patch doesn't change anything on what the first patch in the series introduces even for C++, namely that #embed is expanded (actually or as if) into a sequence of literals like 127,69,76,70,2,1,1,3,0,0,0,0,0,0,0,0,2,0,62,0,1,0,0,0,80,211,64,0,0,0,0,0,64,0,0,0,0,0,0,0,8,253 and so each element has int type. That is how I believe it is in C23, and the different versions of the C++ P1967 paper specified there some casts, P1967R12 in particular "Otherwise, the integral constant expression is the value of std::fgetc’s return is cast to unsigned char." but please see https://github.com/llvm/llvm-project/pull/97274#issuecomment-2230929277 comment and whether we really want the preprocessor to preprocess it for C++ as (or as-if) static_cast(127),static_cast(69),static_cast(76),static_cast(70),static_cast(2),... i.e. 9 tokens per byte rather than 2, or (unsigned char)127,(unsigned char)69,... or ((unsigned char)127),((unsigned char)69),... etc. Without a literal suffix for unsigned char constant literals it is horrible, plus the incompatibility between C and C++. Sure, we could use the magic form more often for C++ to save the size and do the 9 or how many tokens form only for the boundary constants and use #embed "." __gnu__::__base64__("...") for what is in between if there are at least 2 tokens inside of it. E.g. (unsigned char)127 vs. static_cast(127) behaves differently if there is constexpr long long p[] = { ... }; ... #embed __FILE__ [p] 2024-12-06 Jakub Jelinek libcpp/ * files.cc (finish_embed): Use CPP_EMBED even for C++. gcc/ * tree.h (RAW_DATA_UCHAR_ELT, RAW_DATA_SCHAR_ELT): Define. gcc/cp/ChangeLog: * cp-tree.h (class raw_data_iterator): New type. (class raw_data_range): New type. * parser.cc (cp_parser_postfix_open_square_expression): Handle parsing of CPP_EMBED. (cp_parser_parenthesized_expression_list): Likewise. Use cp_lexer_next_token_is. (cp_parser_expression): Handle parsing of CPP_EMBED. (cp_parser_template_argument_list): Likewise. (cp_parser_initializer_list): Likewise. (cp_parser_oacc_clause_tile): Likewise. (cp_parser_omp_tile_sizes): Likewise. * pt.cc (tsubst_expr): Handle RAW_DATA_CST. * constexpr.cc (reduced_constant_expression_p): Likewise. (raw_data_cst_elt): New function. (find_array_ctor_elt): Handle RAW_DATA_CST. (cxx_eval_array_reference): Likewise. * typeck2.cc (digest_init_r): Emit -Wnarrowing and/or -Wconversion diagnostics. (process_init_constructor_array): Handle RAW_DATA_CST. * decl.cc (maybe_deduce_size_from_array_init): Likewise. (is_direct_enum_init): Fail for RAW_DATA_CST. (cp_maybe_split_raw_data): New function. (consume_init): New function. (reshape_init_array_1): Add VECTOR_P argument. Handle RAW_DATA_CST. (reshape_init_array): Adjust reshape_init_array_1 caller. (reshape_init_vector): Likewise. (reshape_init_class): Handle RAW_DATA_CST. (reshape_init_r): Likewise. gcc/testsuite/ * c-c++-common/cpp/embed-22.c: New test. * c-c++-common/cpp/embed-23.c: New test. * g++.dg/cpp/embed-4.C: New test. * g++.dg/cpp/embed-5.C: New test. * g++.dg/cpp/embed-6.C: New test. * g++.dg/cpp/embed-7.C: New test. * g++.dg/cpp/embed-8.C: New test. * g++.dg/cpp/embed-9.C: New test. * g++.dg/cpp/embed-10.C: New test. * g++.dg/cpp/embed-11.C: New test. * g++.dg/cpp/embed-12.C: New test. * g++.dg/cpp/embed-13.C: New test. * g++.dg/cpp/embed-14.C: New test. --- libcpp/files.cc | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'libcpp/files.cc') diff --git a/libcpp/files.cc b/libcpp/files.cc index a60fe1f..6a16503 100644 --- a/libcpp/files.cc +++ b/libcpp/files.cc @@ -1243,8 +1243,7 @@ finish_embed (cpp_reader *pfile, _cpp_file *file, limit = params->limit; size_t embed_tokens = 0; - if (!CPP_OPTION (pfile, cplusplus) - && CPP_OPTION (pfile, lang) != CLK_ASM + if (CPP_OPTION (pfile, lang) != CLK_ASM && limit >= 64) embed_tokens = ((limit - 2) / INT_MAX) + (((limit - 2) % INT_MAX) != 0); -- cgit v1.1