lex: accept zero codepoints in strings

Zero characters (codepoints) are acceptable in strings. The current
Lexer::parse_string skipped such zero codepoints by accidents. The
zero codepoint was also used as error/skip indicator, but that is only
true if the third argument of utf8_escape_pair is true (yes, it is
called pair, but is a triple).

Add a testcase that checks the (sub)strings are separated by zero
chars. Since we cannot slice strings yet this uses extern "C"
functions, printf and memchr.

0 files changed, 0 insertions, 0 deletions