diagnostics: Support conversion of tabs to spaces [PR49973] [PR86904]

Supports conversion of tabs to spaces when outputting diagnostics. Also
adds -fdiagnostics-column-unit and -fdiagnostics-column-origin options to
control how the column number is output, thereby resolving the two PRs.

gcc/c-family/ChangeLog:

	PR other/86904
	* c-indentation.c (should_warn_for_misleading_indentation): Get
	global tabstop from the new source.
	* c-opts.c (c_common_handle_option): Remove handling of -ftabstop, which
	is now a common option.
	* c.opt: Likewise.

gcc/ChangeLog:

	PR preprocessor/49973
	PR other/86904
	* common.opt: Handle -ftabstop here instead of in c-family
	options.  Add -fdiagnostics-column-unit= and
	-fdiagnostics-column-origin= options.
	* opts.c (common_handle_option): Handle the new options.
	* diagnostic-format-json.cc (json_from_expanded_location): Add
	diagnostic_context argument.  Use it to convert column numbers as per
	the new options.
	(json_from_location_range): Likewise.
	(json_from_fixit_hint): Likewise.
	(json_end_diagnostic): Pass the new context argument to helper
	functions above.  Add "column-origin" field to the output.
	(test_unknown_location): Add the new context argument to calls to
	helper functions.
	(test_bad_endpoints): Likewise.
	* diagnostic-show-locus.c
	(exploc_with_display_col::exploc_with_display_col): Support
	tabstop parameter.
	(layout_point::layout_point): Make use of class
	exploc_with_display_col.
	(layout_range::layout_range): Likewise.
	(struct line_bounds): Clarify that the units are now always
	display columns.  Rename members accordingly.  Add constructor.
	(layout::print_source_line): Add support for tab expansion.
	(make_range): Adapt to class layout_range changes.
	(layout::maybe_add_location_range): Likewise.
	(layout::layout): Adapt to class exploc_with_display_col changes.
	(layout::calculate_x_offset_display): Support tabstop parameter.
	(layout::print_annotation_line): Adapt to struct line_bounds changes.
	(layout::print_line): Likewise.
	(line_label::line_label): Add diagnostic_context argument.
	(get_affected_range): Likewise.
	(get_printed_columns): Likewise.
	(layout::print_any_labels): Adapt to struct line_label changes.
	(class correction): Add m_tabstop member.
	(correction::correction): Add tabstop argument.
	(correction::compute_display_cols): Use m_tabstop.
	(class line_corrections): Add m_context member.
	(line_corrections::line_corrections): Add diagnostic_context argument.
	(line_corrections::add_hint): Use m_context to handle tabstops.
	(layout::print_trailing_fixits): Adapt to class line_corrections
	changes.
	(test_layout_x_offset_display_utf8): Support tabstop parameter.
	(test_layout_x_offset_display_tab): New selftest.
	(test_one_liner_colorized_utf8): Likewise.
	(test_tab_expansion): Likewise.
	(test_diagnostic_show_locus_one_liner_utf8): Call the new tests.
	(diagnostic_show_locus_c_tests): Likewise.
	(test_overlapped_fixit_printing): Adapt to helper class and
	function changes.
	(test_overlapped_fixit_printing_utf8): Likewise.
	(test_overlapped_fixit_printing_2): Likewise.
	* diagnostic.h (enum diagnostics_column_unit): New enum.
	(struct diagnostic_context): Add members for the new options.
	(diagnostic_converted_column): Declare.
	(json_from_expanded_location): Add new context argument.
	* diagnostic.c (diagnostic_initialize): Initialize new members.
	(diagnostic_converted_column): New function.
	(maybe_line_and_column): Be willing to output a column of 0.
	(diagnostic_get_location_text): Convert column number as per the new
	options.
	(diagnostic_report_current_module): Likewise.
	(assert_location_text): Add origin and column_unit arguments for
	testing the new functionality.
	(test_diagnostic_get_location_text): Test the new functionality.
	* doc/invoke.texi: Document the new options and behavior.
	* input.h (location_compute_display_column): Add tabstop argument.
	* input.c (location_compute_display_column): Likewise.
	(test_cpp_utf8): Add selftests for tab expansion.
	* tree-diagnostic-path.cc (default_tree_make_json_for_path): Pass the
	new context argument to json_from_expanded_location().

libcpp/ChangeLog:

	PR preprocessor/49973
	PR other/86904
	* include/cpplib.h (struct cpp_options):  Removed support for -ftabstop,
	which is now handled by diagnostic_context.
	(class cpp_display_width_computation): New class.
	(cpp_byte_column_to_display_column): Add optional tabstop argument.
	(cpp_display_width): Likewise.
	(cpp_display_column_to_byte_column): Likewise.
	* charset.c
	(cpp_display_width_computation::cpp_display_width_computation): New
	function.
	(cpp_display_width_computation::advance_display_cols): Likewise.
	(compute_next_display_width): Removed and implemented this
	functionality in a new function...
	(cpp_display_width_computation::process_next_codepoint): ...here.
	(cpp_byte_column_to_display_column): Added tabstop argument.
	Reimplemented in terms of class cpp_display_width_computation.
	(cpp_display_column_to_byte_column): Likewise.
	* init.c (cpp_create_reader): Remove handling of -ftabstop, which is now
	handled by diagnostic_context.

gcc/testsuite/ChangeLog:

	PR preprocessor/49973
	PR other/86904
	* c-c++-common/Wmisleading-indentation-3.c: Adjust expected output
	for new defaults.
	* c-c++-common/Wmisleading-indentation.c: Likewise.
	* c-c++-common/diagnostic-format-json-1.c: Likewise.
	* c-c++-common/diagnostic-format-json-2.c: Likewise.
	* c-c++-common/diagnostic-format-json-3.c: Likewise.
	* c-c++-common/diagnostic-format-json-4.c: Likewise.
	* c-c++-common/diagnostic-format-json-5.c: Likewise.
	* c-c++-common/missing-close-symbol.c: Likewise.
	* g++.dg/diagnostic/bad-binary-ops.C: Likewise.
	* g++.dg/parse/error4.C: Likewise.
	* g++.old-deja/g++.brendan/crash11.C: Likewise.
	* g++.old-deja/g++.pt/overload2.C: Likewise.
	* g++.old-deja/g++.robertl/eb109.C: Likewise.
	* gcc.dg/analyzer/malloc-paths-9.c: Likewise.
	* gcc.dg/bad-binary-ops.c: Likewise.
	* gcc.dg/format/branch-1.c: Likewise.
	* gcc.dg/format/pr79210.c: Likewise.
	* gcc.dg/plugin/diagnostic-test-expressions-1.c: Likewise.
	* gcc.dg/plugin/diagnostic-test-string-literals-1.c: Likewise.
	* gcc.dg/redecl-4.c: Likewise.
	* gfortran.dg/diagnostic-format-json-1.F90: Likewise.
	* gfortran.dg/diagnostic-format-json-2.F90: Likewise.
	* gfortran.dg/diagnostic-format-json-3.F90: Likewise.
	* go.dg/arrayclear.go: Add a comment explaining why adding a
	comment was necessary to work around a dejagnu bug.
	* c-c++-common/diagnostic-units-1.c: New test.
	* c-c++-common/diagnostic-units-2.c: New test.
	* c-c++-common/diagnostic-units-3.c: New test.
	* c-c++-common/diagnostic-units-4.c: New test.
	* c-c++-common/diagnostic-units-5.c: New test.
	* c-c++-common/diagnostic-units-6.c: New test.
	* c-c++-common/diagnostic-units-7.c: New test.
	* c-c++-common/diagnostic-units-8.c: New test.

1 files changed, 46 insertions, 26 deletions

diff --git a/gcc/input.c b/gcc/input.c
index dd1d23d..d573b90 100644
--- a/gcc/input.c
+++ b/gcc/input.c
@@ -913,7 +913,7 @@ make_location (location_t caret, source_range src_range)
    source line in order to calculate the display width.  If that cannot be done
    for any reason, then returns the byte column as a fallback.  */
 int
-location_compute_display_column (expanded_location exploc)
+location_compute_display_column (expanded_location exploc, int tabstop)
 {
   if (!(exploc.file && *exploc.file && exploc.line && exploc.column))
     return exploc.column;
@@ -921,7 +921,7 @@ location_compute_display_column (expanded_location exploc)
   /* If line is NULL, this function returns exploc.column which is the
      desired fallback.  */
   return cpp_byte_column_to_display_column (line.get_buffer (), line.length (),
-					    exploc.column);
+					    exploc.column, tabstop);
 }
 
 /* Dump statistics to stderr about the memory usage of the line_table
@@ -3608,33 +3608,46 @@ test_line_offset_overflow ()
 
 void test_cpp_utf8 ()
 {
+  const int def_tabstop = 8;
   /* Verify that wcwidth of invalid UTF-8 or control bytes is 1.  */
   {
-    int w_bad = cpp_display_width ("\xf0!\x9f!\x98!\x82!", 8);
+    int w_bad = cpp_display_width ("\xf0!\x9f!\x98!\x82!", 8, def_tabstop);
     ASSERT_EQ (8, w_bad);
-    int w_ctrl = cpp_display_width ("\r\t\n\v\0\1", 6);
-    ASSERT_EQ (6, w_ctrl);
+    int w_ctrl = cpp_display_width ("\r\n\v\0\1", 5, def_tabstop);
+    ASSERT_EQ (5, w_ctrl);
   }
 
   /* Verify that wcwidth of valid UTF-8 is as expected.  */
   {
-    const int w_pi = cpp_display_width ("\xcf\x80", 2);
+    const int w_pi = cpp_display_width ("\xcf\x80", 2, def_tabstop);
     ASSERT_EQ (1, w_pi);
-    const int w_emoji = cpp_display_width ("\xf0\x9f\x98\x82", 4);
+    const int w_emoji = cpp_display_width ("\xf0\x9f\x98\x82", 4, def_tabstop);
     ASSERT_EQ (2, w_emoji);
-    const int w_umlaut_precomposed = cpp_display_width ("\xc3\xbf", 2);
+    const int w_umlaut_precomposed = cpp_display_width ("\xc3\xbf", 2,
+							def_tabstop);
     ASSERT_EQ (1, w_umlaut_precomposed);
-    const int w_umlaut_combining = cpp_display_width ("y\xcc\x88", 3);
+    const int w_umlaut_combining = cpp_display_width ("y\xcc\x88", 3,
+						      def_tabstop);
     ASSERT_EQ (1, w_umlaut_combining);
-    const int w_han = cpp_display_width ("\xe4\xb8\xba", 3);
+    const int w_han = cpp_display_width ("\xe4\xb8\xba", 3, def_tabstop);
     ASSERT_EQ (2, w_han);
-    const int w_ascii = cpp_display_width ("GCC", 3);
+    const int w_ascii = cpp_display_width ("GCC", 3, def_tabstop);
     ASSERT_EQ (3, w_ascii);
     const int w_mixed = cpp_display_width ("\xcf\x80 = 3.14 \xf0\x9f\x98\x82"
-					   "\x9f! \xe4\xb8\xba y\xcc\x88", 24);
+					   "\x9f! \xe4\xb8\xba y\xcc\x88",
+					   24, def_tabstop);
     ASSERT_EQ (18, w_mixed);
   }
 
+  /* Verify that display width properly expands tabs.  */
+  {
+    const char *tstr = "\tabc\td";
+    ASSERT_EQ (6, cpp_display_width (tstr, 6, 1));
+    ASSERT_EQ (10, cpp_display_width (tstr, 6, 3));
+    ASSERT_EQ (17, cpp_display_width (tstr, 6, 8));
+    ASSERT_EQ (1, cpp_display_column_to_byte_column (tstr, 6, 7, 8));
+  }
+
   /* Verify that cpp_byte_column_to_display_column can go past the end,
      and similar edge cases.  */
   {
@@ -3645,10 +3658,13 @@ void test_cpp_utf8 ()
       /* 111122223456
 	 Byte columns.  */
 
-    ASSERT_EQ (5, cpp_display_width (str, 6));
-    ASSERT_EQ (105, cpp_byte_column_to_display_column (str, 6, 106));
-    ASSERT_EQ (10000, cpp_byte_column_to_display_column (NULL, 0, 10000));
-    ASSERT_EQ (0, cpp_byte_column_to_display_column (NULL, 10000, 0));
+    ASSERT_EQ (5, cpp_display_width (str, 6, def_tabstop));
+    ASSERT_EQ (105,
+	       cpp_byte_column_to_display_column (str, 6, 106, def_tabstop));
+    ASSERT_EQ (10000,
+	       cpp_byte_column_to_display_column (NULL, 0, 10000, def_tabstop));
+    ASSERT_EQ (0,
+	       cpp_byte_column_to_display_column (NULL, 10000, 0, def_tabstop));
   }
 
   /* Verify that cpp_display_column_to_byte_column can go past the end,
@@ -3662,21 +3678,25 @@ void test_cpp_utf8 ()
       /* 000000000000000000000000000000000111111
 	 111122223333444456666777788889999012345
 	 Byte columns.  */
-    ASSERT_EQ (4, cpp_display_column_to_byte_column (str, 15, 2));
-    ASSERT_EQ (15, cpp_display_column_to_byte_column (str, 15, 11));
-    ASSERT_EQ (115, cpp_display_column_to_byte_column (str, 15, 111));
-    ASSERT_EQ (10000, cpp_display_column_to_byte_column (NULL, 0, 10000));
-    ASSERT_EQ (0, cpp_display_column_to_byte_column (NULL, 10000, 0));
+    ASSERT_EQ (4, cpp_display_column_to_byte_column (str, 15, 2, def_tabstop));
+    ASSERT_EQ (15,
+	       cpp_display_column_to_byte_column (str, 15, 11, def_tabstop));
+    ASSERT_EQ (115,
+	       cpp_display_column_to_byte_column (str, 15, 111, def_tabstop));
+    ASSERT_EQ (10000,
+	       cpp_display_column_to_byte_column (NULL, 0, 10000, def_tabstop));
+    ASSERT_EQ (0,
+	       cpp_display_column_to_byte_column (NULL, 10000, 0, def_tabstop));
 
     /* Verify that we do not interrupt a UTF-8 sequence.  */
-    ASSERT_EQ (4, cpp_display_column_to_byte_column (str, 15, 1));
+    ASSERT_EQ (4, cpp_display_column_to_byte_column (str, 15, 1, def_tabstop));
 
     for (int byte_col = 1; byte_col <= 15; ++byte_col)
       {
-	const int disp_col = cpp_byte_column_to_display_column (str, 15,
-								byte_col);
-	const int byte_col2 = cpp_display_column_to_byte_column (str, 15,
-								 disp_col);
+	const int disp_col
+	  = cpp_byte_column_to_display_column (str, 15, byte_col, def_tabstop);
+	const int byte_col2
+	  = cpp_display_column_to_byte_column (str, 15, disp_col, def_tabstop);
 
 	/* If we ask for the display column in the middle of a UTF-8
 	   sequence, it will return the length of the partial sequence,