aboutsummaryrefslogtreecommitdiff
path: root/clang/lib/AST/CommentLexer.cpp
AgeCommit message (Collapse)AuthorFilesLines
2025-08-14[clang] fix comment lexing of command names with underscore (#152943)mdenson1-1/+1
Comment lexer fails to parse non-alphanumeric names. fixes #33296 --------- Co-authored-by: Brock Denson <brock.denson@virscient.com>
2025-06-26[clang] NFC: Add alias for std::pair<FileID, unsigned> used in ↵Haojian Wu1-1/+1
SourceLocation (#145711) Introduce a type alias for the commonly used `std::pair<FileID, unsigned>` to improve code readability, and make it easier for future updates (64-bit source locations).
2025-02-05[Clang][Comments] Allow HTML tags across multiple lines (#120843)nerix1-6/+63
HTML starting tags that span multiple lines were previously not allowed (or rather, only the starting line was lexed as HTML). Doxygen allows those tags. This PR allows the starting tags to span multiple lines. They can't span multiple (C-)Comments, though (it's likely a user-error). Multiple BCPL comments are fine as those are single lines (shown below). Example: ```c /// <a /// href="foo" /// >Aaa</a>b int Test; ``` Fixes #28321.
2024-12-06[AST] Include clang/Basic/DiagnosticComment.h instead of ↵Kazu Hirata1-1/+1
clang/AST/CommentDiagnostic.h (#117499) Since: commit d076608d58d1ec55016eb747a995511e3a3f72aa Author: Richard Trieu <rtrieu@google.com> Date: Sat Dec 8 05:05:03 2018 +0000 clang/AST/CommentDiagnostic.h has been forwarding to clang/Basic/DiagnosticComment.h. This patch includes clang/Basic/DiagnosticComment.h instead of clang/AST/CommentDiagnostic.h.
2022-09-01[Clang][Comments] Parse `<img src=""/>` in doc comments correctlyEgor Zhdan1-1/+1
This is a valid HTML5 tag. Previously it triggered a Clang error (`HTML start tag prematurely ended, expected attribute name or '>'`) since Clang was treating `/>` as a text token. This was happening because after lexing the closing quote (`"`) the lexer state was reset to "Normal" while the tag was not actually closed yet: `>` was not yet parsed at that point. rdar://91464292 Differential Revision: https://reviews.llvm.org/D132932
2022-01-14Comment parsing: Don't recognize commands in single-line double quotationAaron Puchert1-11/+25
This is consistent with the behavior of Doxygen, and allows users to write strings with C escapes or document input/output formats containing special characters (@ or \) without escaping them, which might be confusing. For example, if a function wants to document its expected input format as "user@host" it doesn't have to write user\@host instead, which would look right in the documentation but confusing in the code. Now users can just use double quotes (which they might do anyway). This fixes a lot of false positives of -Wdocumentation-unknown-command, but it could also fix issues with -Wdocumentation if the text triggers an actual command. Reviewed By: gribozavr2 Differential Revision: https://reviews.llvm.org/D116190
2022-01-14Comment parsing: Simplify Lexer::skipLineStartingDecorations (NFC)Aaron Puchert1-24/+5
Inspection of the first character can just be handled by the loop as well, it does exactly the same thing. Dereferencing the pointer a second time shouldn't be an issue: the middle end can eliminate that second read as it's separated from the first only by a pure function call. Reviewed By: gribozavr2 Differential Revision: https://reviews.llvm.org/D116186
2021-11-09Comment parsing: Complete list of Doxygen commandsAaron Puchert1-2/+3
These should be all the commands from [1] except those that are marked obsolete, and "link" / "endlink", as that conflicts with the existing HeaderDoc pair "link / "/link". For some commands we don't have the ideal category, but it should work good enough for most cases. There seems to be no existing test for most commands (except the ones interpreted by -Wdocumentation), and to some extent such a test wouldn't look very interesting. But I added a test for the correct parsing of formulas, as they're a bit special. And I had to adapt comment-lots-of-unknown-commands.c because typo correction was kicking in and recognizing some of the commands. This should fix a couple of reported bugs: PR17437, PR19581, PR24062 (partially, no diagnostic for matching cond/endcond), PR32909, PR37813, PR44243 (partially, email@domain.com must be addressed separately). [1] https://www.doxygen.nl/manual/commands.html Reviewed By: gribozavr2 Differential Revision: https://reviews.llvm.org/D111190
2020-10-27[clang][NFC] Rearrange Comment Token and Lexer fields to reduce paddingNathan James1-5/+4
Rearrange the fields to reduce the size of the classes Reviewed By: gribozavr2 Differential Revision: https://reviews.llvm.org/D90127
2019-09-18[AST] CommentLexer - Remove (optional) Invalid parameter from getSpelling.Simon Pilgrim1-5/+2
The static analyzer noticed that we were dereferencing it even when the default null value was being used. Further investigation showed that we never explicitly set the parameter so I've just removed it entirely. llvm-svn: 372217
2019-01-19Update the file headers across all of the LLVM projects in the monorepoChandler Carruth1-4/+3
to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2018-08-15Add a newline to SourceLocation dump outputStephen Kelly1-1/+1
Summary: Migrate callers to print(). dump() should be useful to downstreams and third parties as a debugging aid. Everyone trips up on this and creates confusing output. Subscribers: cfe-commits Differential Revision: https://reviews.llvm.org/D50661 llvm-svn: 339810
2018-07-30Remove trailing spaceFangrui Song1-1/+1
sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h} llvm-svn: 338291
2018-05-16[AST] Added a helper to extract a user-friendly text of a comment.Ilya Biryukov1-117/+129
Summary: The helper is used in clangd for documentation shown in code completion and storing the docs in the symbols. See D45999. This patch reuses the code of the Doxygen comment lexer, disabling the bits that do command and html tag parsing. The new helper works on all comments, including non-doxygen comments. However, it does not understand or transform any doxygen directives, i.e. cannot extract brief text, etc. Reviewers: sammccall, hokein, ioeric Reviewed By: ioeric Subscribers: mgorny, cfe-commits Differential Revision: https://reviews.llvm.org/D46000 llvm-svn: 332458
2016-10-25Fix 'unknown documentation command' warning rangesErik Verbruggen1-4/+6
Warnings generated by -Wdocumentation-unknown-command did only have a start location, not a full source range. This resulted in only the "carret" being show in messages, and IDEs highlighting only the single initial character. llvm-svn: 285056
2016-02-10Fix some Clang-tidy readability-redundant-control-flow warnings; other minor ↵Eugene Zelenko1-5/+11
fixes. Differential revision: http://reviews.llvm.org/D17060 llvm-svn: 260414
2015-04-15Comment parsing: fix an assertion failure on a verbatim block terminated ↵Dmitri Gribenko1-0/+6
with "**/" llvm-svn: 235057
2014-08-30Fix some cases where StringRef was being passed by const reference. Remove ↵Craig Topper1-2/+2
const from some other StringRefs since its implicitly const already. llvm-svn: 216825
2013-12-07CommentLexer: eliminate an NDEBUG from the headersAlp Toker1-0/+13
Code in headers shouldn't be conditional on the build configuration. llvm-svn: 196656
2013-12-01CommentLexer: When proceeding with a typo corrected name don't clobber the ↵Benjamin Kramer1-5/+6
token. This would crash if the token is used in another diagnostic. PR18051. llvm-svn: 196048
2013-08-23Fix indentationDmitri Gribenko1-1/+1
llvm-svn: 189119
2013-05-09[doc parsing]: make single character command imposturesFariborz Jahanian1-4/+0
warn in pedantic mode. llvm-svn: 181523
2013-05-09[doc parsing]: So, in this patch, single characterFariborz Jahanian1-0/+4
'commands' will not go through typo fixit logic, preserving the old behavior (no typo, no diagnostics). // rdar://12381408 llvm-svn: 181521
2013-05-08Turn off a warning caused by my last patch.Fariborz Jahanian1-1/+1
llvm-svn: 181464
2013-05-08documentation parsing. Patch to do typo correction for Fariborz Jahanian1-2/+12
documentation commands. Patch was reviewed, along with great suggestions for improvement, by Doug. // rdar://12381408 llvm-svn: 181458
2013-05-04[doc parsing]: Make warning about unknown commandFariborz Jahanian1-3/+2
tags off by default for now. Move diagnostic code to DiagnosticCommentKinds.td. // rdar://12381408 llvm-svn: 181081
2013-05-03[Doc parsing] Provide diagnostics for unknown documentation Fariborz Jahanian1-2/+6
commands. // rdar://12381408 llvm-svn: 181071
2013-03-04Comment parsing: refactor handling of command markers in ASTDmitri Gribenko1-4/+6
* Use the term 'command marker', because the semantics of 'backslash' and 'at' commands are the same. (Talking about 'at commands' makes them look like a special entity.) * Sink the flag down into bitfields, reducing the size of AST nodes. * Change the flag into an enum for clarity. Boolean function parameters are not very clear. * Add unittests for new tok::at_command tokens. llvm-svn: 176461
2013-03-02Some refactoring in my patch on documentFariborz Jahanian1-2/+4
command source fidelity. // rdar://13066276 llvm-svn: 176401
2013-03-01comment parsing. Keep the original command format Fariborz Jahanian1-0/+1
in AST for source fidelity and use it in diagnostics to refer to the original format. // rdar://13066276 llvm-svn: 176387
2013-02-10Use static functions instead of an unnamed namespaceDmitri Gribenko1-6/+8
llvm-svn: 174835
2013-02-09Comment parsing: use CharInfo.hDmitri Gribenko1-38/+16
This also gives us 0.2% speedup on '-fsyntax-only -Wdocumentation' time for a testcase that consists of all Clang headers. llvm-svn: 174810
2013-01-30Comment parsing: resolve more named character referencesDmitri Gribenko1-182/+22
This reimplements r173850 with a better approach: (1) use a TableGen-generated matcher instead of doing a linear search; (2) avoid allocations for new strings by converting code points to string iterals with TableGen. llvm-svn: 173931
2013-01-30Move UTF conversion routines from clang/lib/Basic to llvm/lib/SupportDmitri Gribenko1-3/+3
This is required to use them in TableGen. llvm-svn: 173924
2013-01-30Removed couple of html named character references inFariborz Jahanian1-2/+0
my last patch. llvm-svn: 173856
2013-01-29[Doc parsing] Patch to parse Doxygen-supported HTML character Fariborz Jahanian1-18/+181
references to their UTIF-8 encoding. Reviewed offline by Doug. // rdar://12392215 llvm-svn: 173850
2013-01-19Use llvm::hexDigitValue in comment lexerDmitri Gribenko1-6/+2
llvm-svn: 172924
2012-12-30Comment lexing: replace manual comparison with StringRef::find_first_ofDmitri Gribenko1-9/+6
This gives an about 1.8% improvement on Clang bootstrap with -Wdocumentation llvm-svn: 171262
2012-09-14Comment parsing: don't parse comment marker followed by a digit as a commandDmitri Gribenko1-1/+6
since no Doxygen command starts with a digit. llvm-svn: 163909
2012-09-10Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp.Dmitri Gribenko1-14/+21
Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-08-31Remove the useless CommentOptions class.Dmitri Gribenko1-2/+2
llvm-svn: 162986
2012-08-31Comment HTML tag name machers: move from StringSwitch to an efficientDmitri Gribenko1-26/+2
TableGen-generated string matcher. llvm-svn: 162969
2012-08-22Comment parsing: parse "<blah" as an HTML tag only if "blah" is a known tagDmitri Gribenko1-2/+38
name. This should reduce the amount of warning false positives about bad HTML in comments when the comment author intended to put a reference to a template. This change will also enable us parse the comment as intended in these cases. Fixes part 1 of PR13374. llvm-svn: 162407
2012-08-09Comment parsing: extract TableGen'able pieces into new CommandTraits class.Dmitri Gribenko1-93/+5
llvm-svn: 161548
2012-07-27Implement resolving of HTML character references (named: &amp;, decimal: &#42;,Dmitri Gribenko1-24/+174
hex: &#x1a;) during comment parsing. Now internal representation of plain text in comment AST does not contain character references, but the characters themselves. llvm-svn: 160891
2012-07-18Comment parsing: don't parse whitespace before \endverbatim as a separate ↵Dmitri Gribenko1-0/+10
line of whitespace. llvm-svn: 160464
2012-07-13Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML ↵Dmitri Gribenko1-17/+17
start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-11Enable comment parsing and semantic analysis to emit diagnostics. A fewDmitri Gribenko1-1/+13
diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-09Comment lexing: fix lexing to actually work in non-error cases.Dmitri Gribenko1-13/+18
llvm-svn: 159963
2012-07-06Implement AST classes for comments, a real parser for Doxygen comments and aDmitri Gribenko1-8/+36
very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
generated by cgit v1.1 at 2025-10-12 04:37:18 +0000