aboutsummaryrefslogtreecommitdiff
path: root/flang/lib/Parser/token-sequence.cpp
AgeCommit message (Collapse)AuthorFilesLines
2025-07-03[flang][prescanner] fix invalid check (#146613)Andre Kuhlenschmidt1-1/+2
`TokenSequence::pop_back()` had a check assumed that tokens are never empty. Loosen this check since isn't true. towards #146362
2025-05-12[flang] Further refinement of OpenMP !$ lines in -E mode (#138956)Peter Klausler1-2/+4
Address failing Fujitsu test suite cases that were broken by the patch to defer the handling of !$ lines in -fopenmp vs. normal compilation to actual compilation rather than processing them immediately in -E mode. Tested on the samples in the bug report as well as all of the Fujitsu tests that I could find that use !$ lines. Fixes https://github.com/llvm/llvm-project/issues/136845.
2025-04-18[flang] Don't perform macro replacement on exponents (#136176)Peter Klausler1-6/+7
See new test. I inadvertently broke this behavior with a recent fix for another problem, because the effects of the overloaded TokenSequence::Put() member function on token merging were confusing. Rename and document the various overloads.
2025-04-09[flang] Fix preprocessor regression (#134405)Peter Klausler1-1/+1
For numeric kind suffixes like 1_foo, the preprocessor should be able to perform macro replacement for macros named either "_foo" or "foo". Fixes https://github.com/llvm/llvm-project/issues/133399.
2025-04-04[flang] Preserve compiler directives in -E output (#133959)Peter Klausler1-0/+1
No longer require -fopenmp or -fopenacc with -E, unless specific version number options are also required for predefined macros. This means that most source can be preprocessed with -E and then later compiled with -fopenmp, -fopenacc, or neither. This means that OpenMP conditional compilation lines (!$) are also passed through to -E output. The tricky part of this patch was dealing with the fact that those conditional lines can also contain regular Fortran line continuation, and that now has to be deferred when !$ lines are interspersed.
2024-11-05[flang] Handle "defined" in macro expansions (#114844)Peter Klausler1-0/+10
The preprocessor implements "defined(X)" and "defined X" in if/elif directive expressions in such a way that they only work at the top level, not when they appear in macro expansions. Fix that, which is a little tricky due to the need to detect the "defined" keyword before applying any macro expansion to its argument, and add a bunch of tests. Fixes https://github.com/llvm/llvm-project/issues/114064.
2024-06-28[flang] Accept a compiler directive sentinel after a semicolon (#96966)Peter Klausler1-7/+4
Don't treat !DIR$ or an active !$ACC, !$OMP, &c. as a comment when they appear after a semicolon, but instead treat them as a compiler directive sentinel.
2024-06-13[flang][preprocessor] Fixed-form continuation across preprocessing di… ↵Peter Klausler1-4/+8
(#95332) …rective Implement fixed-form line continuation when the continuation line is the result of text produced by an #include or other preprocessing directive. This accommodates the somewhat common practice of putting dummy or actual arguments into a header file and #including it into several code sites. Fixes https://github.com/llvm/llvm-project/issues/78928.
2024-06-12[flang] Disable Fortran free form line continuation in non-source lin… ↵Peter Klausler1-1/+4
(#94663) …e produced by keyword macro replacement When later initial keyword macro replacement will yield a line that is not Fortran source, don't interpret "&" as a Fortran source line continuation marker during tokenization of the line. Fixes https://github.com/llvm/llvm-project/issues/82579.
2024-04-10[flang][Frontend] Implement printing defined macros via -dM (#87627)Krzysztof Parzyszek1-1/+2
This should work the same way as in clang.
2024-03-11[flang] Avoid forming a reference from null pointer (#84787)Krzysztof Parzyszek1-1/+4
Doing so is an undefined behavior. This was detected by the undefined behavior sanitizer.
2023-10-31[flang] Accept directive sentinels in macro-replaced source better (#70699)Peter Klausler1-10/+12
At present, the prescanner emits an error if a source line or compiler directive, after macro replacement or not, contains a token with a non-Fortran character. In the particular case of the '!' character, the code that checks for bad character will accept the '!' if it appears after a ';', since the '!' might begin a compiler directive. This current implementation fails when a compiler directive appears after some other character that might (by means of further source processing not visible to the prescanner) be replaced with a ';' or newline. Extend the bad character check for '!' to actually check for a compiler directive sentinel instead.
2023-07-31[flang] Stricter "implicit continuation" in preprocessingPeter Klausler1-3/+5
The prescanner performs implicit line continuation when it looks like the parenthesized arguments of a call to a function-like macro may span multiple lines. In an attempt to work more like a Fortran-oblivious C preprocessor, the prescanner will act as if the following lines had been continuations so that the function-like macro could be invoked. This still seems like a good idea, but a recent bug report on LLVM's GitHub issue tracker shows one way in which it could trigger inadvertently and mess up a program. So this patch makes the conditions for implicit line continuation much more strict. First, the leading parenthesis has to have been preceded by an identifier that's known to be a macro name. (It doesn't have to be a function-like macro, since it's possible for a keyword-like macro to expand to the name of a function-like macro.) Second, no macro definition can ever have had unbalanced parentheses in its replacement text. Also cleans up some parenthesis recognition code to fix some issues found in testing, so that a token with leading or trailing spaces can still be recognized as a parenthesis or comma. Fixes https://github.com/llvm/llvm-project/issues/63844. Differential Revision: https://reviews.llvm.org/D155499
2023-06-01[flang] Fix character initialization after continuationLeandro Lupori1-4/+13
The insertion of a space on a line continuation right before a character literal was confusing TokenSequence::ToLowerCase(), that was unable to identify the character literal as such, causing it to be converted to lower case. Fix this by skipping spaces in the beginning and end of each token, before testing for token type. Fixes https://github.com/llvm/llvm-project/issues/62039 Reviewed By: klausler Differential Revision: https://reviews.llvm.org/D151885
2023-05-18[flang] Allow compiler directives in macrosPeter Klausler1-4/+31
Modify the prescanner to allow compiler directives to appear in macro expansions, and adjust the parser to accept a semicolon as a directive terminator. Differential Revision: https://reviews.llvm.org/D150780
2023-02-13[flang] Fix new assertion failurePeter Klausler1-2/+4
Don't compute the address of element [0] of a vector when the vector is empty, even if the address is not used. Differential Revision: https://reviews.llvm.org/D143824
2021-10-06[flang] Catch mismatched parentheses in prescannerpeter klausler1-10/+39
Source lines with mismatched parentheses are hard cases for error recovery in parsing, and the best error message (viz., "here's an unmatched parenthesis") can be emitted from the prescanner. Differential Revision: https://reviews.llvm.org/D111254#3046173
2021-09-17[flang] Enforce fixed form rules about END continuationpeter klausler1-2/+6
From subclause 6.3.3.5: a program unit END statement cannot be continued in fixed form, and other statements cannot have initial lines that look like program unit END statements. I think this is to avoid violating assumptions that are important to legacy compilers' statement classification routines. Differential Revision: https://reviews.llvm.org/D109933
2021-07-30[flang] Produce proper "preprocessor output" for -E optionpeter klausler1-1/+2
Rename the current -E option to "-E -Xflang -fno-reformat". Add a new Parsing::EmitPreprocessedSource() routine to convert the cooked character stream output of the prescanner back to something more closely resembling output from a traditional preprocessor; call this new routine when -E appears. The new -E output is suitable for use as fixed form Fortran source to compilation by (one hopes) any Fortran compiler. If the original top-level source file had been free form source, the output will be suitable for use as free form source as well; otherwise there may be diagnostics about missing spaces if they were indeed absent in the original fixed form source. Unless the -P option appears, #line directives are interspersed with the output (but be advised, f18 will ignore these if presented with them in a later compilation). An effort has been made to preserve original alphabetic character case and source indentation. Add -P and -fno-reformat to the new drivers. Tweak test options to avoid confusion with prior -E output; use -fno-reformat where needed, but prefer to keep -E, sometimes in concert with -P, on most, updating expected results accordingly. Differential Revision: https://reviews.llvm.org/D106727
2020-09-21Do not dereference an array out of bound just to take its addressserge-sans-paille1-1/+2
This is UB by the standard, and caught by the libstdc++ asserts Differential Revision: https://reviews.llvm.org/D87892
2020-08-25[flang] Improve error handling for bad characters in sourcepeter klausler1-0/+22
When an illegal character appears in Fortran source (after preprocessing), catch and report it in the prescanning phase rather than leaving it for the parser to cope with. Differential Revision: https://reviews.llvm.org/D86553
2020-07-17[flang] Allow ! and // comments after some preprocessing directivespeter klausler1-0/+25
Old-style C /*comments*/ are omitted from preprocessor directive token sequences by the prescanner, but line-ending C++ and Fortran free-form comments are not since their handling might depend on the directive. Add code to skip these line-ending comments as appropriate in place of existing code that just skipped blanks. Reviewed By: sscalpone Differential Revision: https://reviews.llvm.org/D84061
2020-03-28[flang] Reformat with latest clang-format and .clang-formatTim Keith1-2/+2
Original-commit: flang-compiler/f18@9fe84f45d7fd685051004678d6b5775dcc4c6f8f Reviewed-on: https://github.com/flang-compiler/f18/pull/1094
2020-03-19[flang] [LLVMify F18] Replace the use std::ostream with LLVM streams ↵Caroline Concatto1-2/+3
llvm::ostream This patch replaces the occurrence of std::ostream by llvm::raw_ostream. In LLVM Coding Standards[1] "All new code should use raw_ostream instead of ostream".[1] As a consequence, this patch also replaces the use of: std::stringstream by llvm::raw_string_ostream or llvm::raw_ostream* std::ofstream by llvm::raw_fd_ostream std::endl by '\n' and flush()[2] std::cout by llvm::outs() and std::cerr by llvm::errs() It also replaces std::strerro by llvm::sys::StrError** , but NOT in Fortran runtime libraries *std::stringstream were replaced by llvm::raw_ostream in all methods that used std::stringstream as a parameter. Moreover, it removes the pointers to these streams. [1]https://llvm.org/docs/CodingStandards.html [2]https://releases.llvm.org/2.5/docs/CodingStandards.html#ll_avoidendl Signed-off-by: Caroline Concatto <caroline.concatto@arm.com> Running clang-format-7 Signed-off-by: Caroline Concatto <caroline.concatto@arm.com> Removing residue of ostream library Signed-off-by: Caroline Concatto <caroline.concatto@arm.com> Original-commit: flang-compiler/f18@a3507d44b8911e6024033aa583c1dc54e0eb89fd Reviewed-on: https://github.com/flang-compiler/f18/pull/1047
2020-02-25[flang] [LLVMify F18] Compiler module folders should have capitalised names ↵CarolineConcatto1-0/+287
(flang-compiler/f18#980) This patch renames the modules in f18 to use a capital letter in the module name Signed-off-by: Caroline Concatto <caroline.concatto@arm.com> Original-commit: flang-compiler/f18@d2eb7a1c443d1539ef12b6f027074a0eb15b1ea0 Reviewed-on: https://github.com/flang-compiler/f18/pull/980