diff options
| author | Raiki Tamura <tamaron1203@gmail.com> | 2023-06-18 23:25:31 +0900 |
|---|---|---|
| committer | Arthur Cohen <arthur.cohen@embecosm.com> | 2024-01-16 18:46:30 +0100 |
| commit | 0ee16a42074486d6514379d7faa6aee54100f0a4 (patch) | |
| tree | 71b24c76f06458621ba468704b004ee1220d8828 /gcc/rust/ast/rust-ast.cc | |
| parent | 0770c239f475ed2ae25d96175019865156db516e (diff) | |
| download | gcc-0ee16a42074486d6514379d7faa6aee54100f0a4.zip gcc-0ee16a42074486d6514379d7faa6aee54100f0a4.tar.gz gcc-0ee16a42074486d6514379d7faa6aee54100f0a4.tar.bz2 | |
gccrs: Refactor lexer to handle UTF-8
gcc/rust/ChangeLog:
* lex/rust-lex.cc (is_float_digit):Change types of param to `uint32_t`
(is_x_digit):Likewise
(is_octal_digit):Likewise
(is_bin_digit):Likewise
(check_valid_float_dot_end):Likewise
(is_whitespace):Likewise
(is_non_decimal_int_literal_separator):Likewise
(is_identifier_start):Likewise
(is_identifier_continue):Likewise
(Lexer::skip_broken_string_input):Likewise
(Lexer::build_token):Remove handling BOM
(Lexer::parse_in_type_suffix):Modify use of `current_char`
(Lexer::parse_in_decimal):Likewise
(Lexer::parse_escape):Likewise
(Lexer::parse_utf8_escape):Likewise
(Lexer::parse_partial_string_continue):Likewise
(Lexer::parse_partial_hex_escape):Likewise
(Lexer::parse_partial_unicode_escape):Likewise
(Lexer::parse_byte_char):Likewise
(Lexer::parse_byte_string):Likewise
(Lexer::parse_raw_byte_string):Likewise
(Lexer::parse_raw_identifier):Likewise
(Lexer::parse_non_decimal_int_literal):Likewise
(Lexer::parse_decimal_int_or_float):Likewise
(Lexer::peek_input):Change return type to `Codepoint`
(Lexer::get_input_codepoint_length):Change to return 1
(Lexer::peek_codepoint_input):Change to be wrapper of `peek_input`
(Lexer::skip_codepoint_input):Change to be wrapper of `skip_input`
(Lexer::test_get_input_codepoint_n_length):Deleted
(Lexer::split_current_token):Deleted
(Lexer::test_peek_codepoint_input):Deleted
(Lexer::start_line):Move backwards
(assert_source_content):New helper function for selftest
(test_buffer_input_source):New helper function for selftest
(test_file_input_source):Likewise
(rust_input_source_test):New test
* lex/rust-lex.h (rust_input_source_test):New test
* rust-lang.cc (run_rust_tests):Add selftest
Signed-off-by: Raiki Tamura <tamaron1203@gmail.com>
Diffstat (limited to 'gcc/rust/ast/rust-ast.cc')
0 files changed, 0 insertions, 0 deletions