[clang-format] Break long string literals in C#, etc.

Now strings that are too long for one line in C#, Java, JavaScript, and
Verilog get broken into several lines.  C# and JavaScript interpolated
strings are not broken.

A new subclass BreakableStringLiteralUsingOperators is used to handle
the logic for adding plus signs and commas.  The updateAfterBroken
method was added because now parentheses or braces may be required after
the parentheses or commas are added.  In order to decide whether the
added plus sign should be unindented in the BreakableToken object, the
logic for it is taken out into a separate function
shouldUnindentNextOperator.

The logic for finding the continuation indentation when the option
AlignAfterOpenBracket is set to DontAlign is not implemented yet.  So in
that case the new line may have the wrong indentation, and the parts may
have the wrong length if the string needs to be broken more than once
because finding where to break the string depends on where the string
starts.

The preambles for the C# and Java unit tests are changed to the newer
style in order to allow the 3-argument verifyFormat macro.  Some cases
are changed from verifyFormat to verifyImcompleteFormat because those
use incomplete code and the new verifyFormat function checks that the
code is complete.

The line in the doc was changed to being indented by 4 spaces, that is,
the default continuation indentation.  It has always been the case.  It
was probably a mistake that the doc showed 2 spaces previously.

This commit was fist committed as 16ccba51072b.  The tests caused
assertion failures.  Then it was reverted in 547bce36132a.

Reviewed By: MyDeveloperDay

Differential Revision: https://reviews.llvm.org/D154093

1 files changed, 91 insertions, 0 deletions

diff --git a/clang/unittests/Format/FormatTestJS.cpp b/clang/unittests/Format/FormatTestJS.cpp
index 1df7859..bc4c7ff 100644
--- a/clang/unittests/Format/FormatTestJS.cpp
+++ b/clang/unittests/Format/FormatTestJS.cpp
@@ -1505,6 +1505,97 @@ TEST_F(FormatTestJS, TryCatch) {
 TEST_F(FormatTestJS, StringLiteralConcatenation) {
   verifyFormat("var literal = 'hello ' +\n"
                "    'world';");
+
+  // Long strings should be broken.
+  verifyFormat("var literal =\n"
+               "    'xxxxxxxx ' +\n"
+               "    'xxxxxxxx';",
+               "var literal = 'xxxxxxxx xxxxxxxx';",
+               getGoogleJSStyleWithColumns(17));
+  verifyFormat("var literal =\n"
+               "    'xxxxxxxx ' +\n"
+               "    'xxxxxxxx';",
+               "var literal = 'xxxxxxxx xxxxxxxx';",
+               getGoogleJSStyleWithColumns(18));
+  verifyFormat("var literal =\n"
+               "    'xxxxxxxx' +\n"
+               "    ' xxxxxxxx';",
+               "var literal = 'xxxxxxxx xxxxxxxx';",
+               getGoogleJSStyleWithColumns(16));
+  // The quotes should be correct.
+  for (char OriginalQuote : {'\'', '"'}) {
+    auto VerifyQuotes = [=](FormatStyle::JavaScriptQuoteStyle StyleQuote,
+                            char TargetQuote) {
+      auto Style = getGoogleJSStyleWithColumns(17);
+      Style.JavaScriptQuotes = StyleQuote;
+      std::string Target{"var literal =\n"
+                         "    \"xxxxxxxx \" +\n"
+                         "    \"xxxxxxxx\";"};
+      std::string Original{"var literal = \"xxxxxxxx xxxxxxxx\";"};
+      std::replace(Target.begin(), Target.end(), '"', TargetQuote);
+      std::replace(Original.begin(), Original.end(), '"', OriginalQuote);
+      verifyFormat(Target, Original, Style);
+    };
+    VerifyQuotes(FormatStyle::JSQS_Leave, OriginalQuote);
+    VerifyQuotes(FormatStyle::JSQS_Single, '\'');
+    VerifyQuotes(FormatStyle::JSQS_Double, '"');
+  }
+  // Parentheses should be added when necessary.
+  verifyFormat("var literal =\n"
+               "    ('xxxxxxxx ' +\n"
+               "     'xxxxxx')[0];",
+               "var literal = 'xxxxxxxx xxxxxx'[0];",
+               getGoogleJSStyleWithColumns(18));
+  auto Style = getGoogleJSStyleWithColumns(20);
+  Style.SpacesInParens = FormatStyle::SIPO_Custom;
+  Style.SpacesInParensOptions.Other = true;
+  verifyFormat("var literal =\n"
+               "    ( 'xxxxxxxx ' +\n"
+               "      'xxxxxx' )[0];",
+               "var literal = 'xxxxxxxx xxxxxx'[0];", Style);
+  // FIXME: When the part before the string literal is shorter than the
+  // continuation indentation, and the option AlignAfterOpenBracket is set to
+  // AlwaysBreak which is the default for the Google style, the unbroken string
+  // does not get to a new line while the broken string does due to the added
+  // parentheses. The formatter does not do it in one pass.
+  EXPECT_EQ(
+      "x = ('xxxxxxxx ' +\n"
+      "     'xxxxxx')[0];",
+      format("x = 'xxxxxxxx xxxxxx'[0];", getGoogleJSStyleWithColumns(18)));
+  verifyFormat("x =\n"
+               "    ('xxxxxxxx ' +\n"
+               "     'xxxxxx')[0];",
+               getGoogleJSStyleWithColumns(18));
+  // Breaking of template strings and regular expressions is not implemented.
+  verifyFormat("var literal =\n"
+               "    `xxxxxxxx xxxxxxxx`;",
+               getGoogleJSStyleWithColumns(18));
+  verifyFormat("var literal =\n"
+               "    /xxxxxxxx xxxxxxxx/;",
+               getGoogleJSStyleWithColumns(18));
+  // There can be breaks in the code inside a template string.
+  verifyFormat("var literal = `xxxxxx ${\n"
+               "    xxxxxxxxxx} xxxxxx`;",
+               "var literal = `xxxxxx ${xxxxxxxxxx} xxxxxx`;",
+               getGoogleJSStyleWithColumns(14));
+  verifyFormat("var literal = `xxxxxx ${\n"
+               "    xxxxxxxxxx} xxxxxx`;",
+               "var literal = `xxxxxx ${xxxxxxxxxx} xxxxxx`;",
+               getGoogleJSStyleWithColumns(15));
+  // Identifiers inside the code inside a template string should not be broken
+  // even if the column limit is exceeded.  This following behavior is not
+  // optimal.  The part after the closing brace which exceeds the column limit
+  // can be put on a new line.  Change this test when it is implemented.
+  verifyFormat("var literal = `xxxxxx ${\n"
+               "    xxxxxxxxxxxxxxxxxxxxxx} xxxxxx`;",
+               "var literal = `xxxxxx ${xxxxxxxxxxxxxxxxxxxxxx} xxxxxx`;",
+               getGoogleJSStyleWithColumns(14));
+  verifyFormat("var literal = `xxxxxx ${\n"
+               "    xxxxxxxxxxxxxxxxxxxxxx +\n"
+               "    xxxxxxxxxxxxxxxxxxxxxx} xxxxxx`;",
+               "var literal = `xxxxxx ${xxxxxxxxxxxxxxxxxxxxxx + "
+               "xxxxxxxxxxxxxxxxxxxxxx} xxxxxx`;",
+               getGoogleJSStyleWithColumns(14));
 }
 
 TEST_F(FormatTestJS, RegexLiteralClassification) {