diff options
| author | Raiki Tamura <tamaron1203@gmail.com> | 2023-06-19 18:06:11 +0900 |
|---|---|---|
| committer | Philip Herron <philip.herron@embecosm.com> | 2023-06-20 08:10:20 +0000 |
| commit | 1931d49c42ad0dfcfbd30845dc0dcc89a805643b (patch) | |
| tree | 243f2f10ba4375b3be40e85727d06bef0fb84934 /gcc/rust/resolve | |
| parent | c05b7f22389964e45ef0e59bc87a9eb3494a33fa (diff) | |
| download | gcc-1931d49c42ad0dfcfbd30845dc0dcc89a805643b.zip gcc-1931d49c42ad0dfcfbd30845dc0dcc89a805643b.tar.gz gcc-1931d49c42ad0dfcfbd30845dc0dcc89a805643b.tar.bz2 | |
gccrs: tokenize Unicode identifiers
gcc/rust/ChangeLog:
* lex/rust-lex.cc (is_whitespace):add all lacked codepoints valid as whitespaces
(is_identifier_start):new function to check XID_Start and underscore
(is_identifier_continue):new function to check XID_Continue
(Lexer::build_token):tokenize Unicode identifiers
(Lexer::parse_partial_string_continue):add comments
(Lexer::parse_partial_unicode_escape):add comments
(Lexer::parse_raw_identifier):change to use `is_identifier_scontinue`
(Lexer::parse_identifier_or_keyword):change to use `is_identifier_continue`
(Lexer::parse_char_or_lifetime):change to use `is_identifier_start/continue`
(Lexer::skip_codepoint_input):do not attempt to skip input when bumping EOF
* lex/rust-lex.h:add `current_char32` field
Signed-off-by: Raiki Tamura <tamaron1203@gmail.com>
Diffstat (limited to 'gcc/rust/resolve')
0 files changed, 0 insertions, 0 deletions