Fix bugs in EOL marking in command line tokenizers

Add unit tests for this behavior, since the integration test for
clang-cl did not catch these bugs.

Fixes PR47604

Differential Revision: https://reviews.llvm.org/D90866