Linemap infrastructure for virtual locations

This is the first instalment of a set which goal is to track locations
of tokens across macro expansions.  Tom Tromey did the original work
and attached the patch to PR preprocessor/7263.  This opus is a
derivative of that original work.

This patch modifies the linemap module of libcpp to add virtual
locations support.

A virtual location is a mapped location that can resolve to several
different physical locations.  It can always resolve to the spelling
location of a token.  For tokens resulting from macro expansion it can
resolve to:
  - either the location of the expansion point of the macro.
  - or the location of the token in the definition of the
  macro
  - or, if the token is an argument of a function-like macro,
  the location of the use of the matching macro parameter in
  the definition of the macro

The patch creates a new type of line map called a macro map.  For every
single macro expansion, there is a macro map that generates a virtual
location for every single resulting token of the expansion.

The good old type of line map we all know is now called an ordinary
map.  That one still encodes spelling locations as it has always had.

As a result linemap_lookup as been extended to return a macro map when
given a virtual location resulting from a macro expansion.  The layout
of structs line_map has changed to support this new type of map.  So
did the layout of struct line_maps.  Accessor macros have been
introduced to avoid messing with the implementation details of these
datastructures directly.  This helped already as we have been testing
different ways of arranging these datastructure.  Having to constantly
adjust client code that is too tied with the internals of line_map and
line_maps would have been even more painful.

Of course, many new public functions have been added to the linemap
module to handle the resolution of virtual locations.

This patch introduces the infrastructure but no part of the compiler
uses virtual locations yet.

However the client code of the linemap data structures has been
adjusted as per the changes.  E.g, it's not anymore reliable for a
client code to manipulate struct line_map directly if it just wants to
deal with spelling locations, because struct line_map can now
represent a macro map as well.  In that case, it's better to use the
convenient API to resolve the initial (possibly virtual) location to a
spelling location (or to an ordinary map) and use that.

This is the reason why the patch adjusts the Java, Ada and Fortran
front ends.

Also, note that virtual locations are not supposed to be ordered for
relations '<' and '>' anymore.  To test if a virtual location appears
"before" another one, one has to use a new operator exposed by the
line map interface.  The patch updates the only spot (in the
diagnostics module) I have found that was making the assumption that
locations were ordered for these relations.  This is the only change
that introduces a use of the new line map API in this patch, so I am
adding a regression test for it only.

From-SVN: r180081

15 files changed, 133 insertions, 66 deletions

diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index 886ba44..a6f0e00 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,3 +1,18 @@
+2011-10-15  Tom Tromey  <tromey@redhat>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* input.h (struct expanded_location): Move to libcpp/line-map.h.
+	(LOCATION_COLUMN): New accessor
+	(in_system_header_at): Use linemap_location_in_system_header_p.
+	* diagnostic.c (diagnostic_report_current_module): Adjust to avoid
+	touching the internals of struct line_map.  Use the public API.
+	instead.
+	(diagnostic_report_diagnostic): Don't use relational operator '<'
+	on virtual locations.  Use linemap_location_before_p instead.
+	* input.c (expand_location): Adjust to expand to the tokens'
+	spelling location when macro location tracking is on.
+
+
 2011-10-08  Andi Kleen  <ak@linux.intel.com>
 
 	* ggc-page.c (GGC_QUIRE_SIZE): Increase to 512
diff --git a/gcc/ada/ChangeLog b/gcc/ada/ChangeLog
index 9c17e0e..f839cfb 100644
--- a/gcc/ada/ChangeLog
+++ b/gcc/ada/ChangeLog
@@ -1,3 +1,9 @@
+2011-10-15  Tom Tromey  <tromey@redhat.com> 
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* gcc-interface/trans.c (gigi, Sloc_to_locus): Adjust to use the
+	new public ordinary map interface.
+
 2011-10-16  Tristan Gingold  <gingold@adacore.com>
 
 	* link.c (_AIX): Add support for GNU ld.        
diff --git a/gcc/ada/gcc-interface/trans.c b/gcc/ada/gcc-interface/trans.c
index faf5eb3..69c66d1 100644
--- a/gcc/ada/gcc-interface/trans.c
+++ b/gcc/ada/gcc-interface/trans.c
@@ -314,7 +314,7 @@ gigi (Node_Id gnat_root, int max_gnat_node, int number_name ATTRIBUTE_UNUSED,
 	     (Get_Name_String (file_info_ptr[i].File_Name))));
 
       /* We rely on the order isomorphism between files and line maps.  */
-      gcc_assert ((int) line_table->used == i);
+      gcc_assert ((int) LINEMAPS_ORDINARY_USED (line_table) == i);
 
       /* We create the line map for a source file at once, with a fixed number
 	 of columns chosen to avoid jumping over the next power of 2.  */
@@ -8391,12 +8391,10 @@ Sloc_to_locus (Source_Ptr Sloc, location_t *locus)
       Source_File_Index file = Get_Source_File_Index (Sloc);
       Logical_Line_Number line = Get_Logical_Line_Number (Sloc);
       Column_Number column = Get_Column_Number (Sloc);
-      struct line_map *map = &line_table->maps[file - 1];
+      struct line_map *map = LINEMAPS_ORDINARY_MAP_AT (line_table, file - 1);
 
-      /* Translate the location according to the line-map.h formula.  */
-      *locus = map->start_location
-		+ ((line - map->to_line) << map->column_bits)
-		+ (column & ((1 << map->column_bits) - 1));
+      /* Translate the location.  */
+      *locus = linemap_position_for_line_and_column (map, line, column);
     }
 
   ref_filename
diff --git a/gcc/c-family/ChangeLog b/gcc/c-family/ChangeLog
index 3a26e17..b6ed818 100644
--- a/gcc/c-family/ChangeLog
+++ b/gcc/c-family/ChangeLog
@@ -1,3 +1,12 @@
+2011-10-15  Tom Tromey  <tromey@redhat.com>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* c-ppoutput.c (scan_translation_unit, maybe_print_line)
+	(print_line, cb_define, do_line_change): Adjust to avoid touching
+	the internals of struct line_map.  Use the public API instead.
+	* c-pch.c (c_common_read_pch): Likewise.
+	* c-lex.c (fe_file_change): Likewise.
+
 2011-10-11  Michael Meissner  <meissner@linux.vnet.ibm.com>
 
 	* c-common.c (def_builtin_1): Delete old interface with two
diff --git a/gcc/c-family/c-lex.c b/gcc/c-family/c-lex.c
index e60dcc5..be83b61 100644
--- a/gcc/c-family/c-lex.c
+++ b/gcc/c-family/c-lex.c
@@ -207,7 +207,7 @@ fe_file_change (const struct line_map *new_map)
 	    line = SOURCE_LINE (new_map - 1, included_at);
 
 	  input_location = new_map->start_location;
-	  (*debug_hooks->start_source_file) (line, new_map->to_file);
+	  (*debug_hooks->start_source_file) (line, LINEMAP_FILE (new_map));
 #ifndef NO_IMPLICIT_EXTERN_C
 	  if (c_header_level)
 	    ++c_header_level;
@@ -231,10 +231,10 @@ fe_file_change (const struct line_map *new_map)
 #endif
       input_location = new_map->start_location;
 
-      (*debug_hooks->end_source_file) (new_map->to_line);
+      (*debug_hooks->end_source_file) (LINEMAP_LINE (new_map));
     }
 
-  update_header_times (new_map->to_file);
+  update_header_times (LINEMAP_FILE (new_map));
   input_location = new_map->start_location;
 }
 
diff --git a/gcc/c-family/c-ppoutput.c b/gcc/c-family/c-ppoutput.c
index 16d4f7d..892f1ea 100644
--- a/gcc/c-family/c-ppoutput.c
+++ b/gcc/c-family/c-ppoutput.c
@@ -190,9 +190,7 @@ scan_translation_unit (cpp_reader *pfile)
       /* Subtle logic to output a space if and only if necessary.  */
       if (avoid_paste)
 	{
-	  const struct line_map *map
-	    = linemap_lookup (line_table, loc);
-	  int src_line = SOURCE_LINE (map, loc);
+	  int src_line = LOCATION_LINE (loc);
 
 	  if (print.source == NULL)
 	    print.source = token;
@@ -212,9 +210,7 @@ scan_translation_unit (cpp_reader *pfile)
 	}
       else if (token->flags & PREV_WHITE)
 	{
-	  const struct line_map *map
-	    = linemap_lookup (line_table, loc);
-	  int src_line = SOURCE_LINE (map, loc);
+	  int src_line = LOCATION_LINE (loc);
 
 	  if (src_line != print.src_line
 	      && do_line_adjustments
@@ -304,8 +300,9 @@ scan_translation_unit_trad (cpp_reader *pfile)
 static void
 maybe_print_line (source_location src_loc)
 {
-  const struct line_map *map = linemap_lookup (line_table, src_loc);
-  int src_line = SOURCE_LINE (map, src_loc);
+  int src_line = LOCATION_LINE (src_loc);
+  const char *src_file = LOCATION_FILE (src_loc);
+
   /* End the previous line of text.  */
   if (print.printed)
     {
@@ -317,7 +314,7 @@ maybe_print_line (source_location src_loc)
   if (!flag_no_line_commands
       && src_line >= print.src_line
       && src_line < print.src_line + 8
-      && strcmp (map->to_file, print.src_file) == 0)
+      && strcmp (src_file, print.src_file) == 0)
     {
       while (src_line > print.src_line)
 	{
@@ -341,28 +338,30 @@ print_line (source_location src_loc, const char *special_flags)
 
   if (!flag_no_line_commands)
     {
-      const struct line_map *map = linemap_lookup (line_table, src_loc);
-
-      size_t to_file_len = strlen (map->to_file);
+      const char *file_path = LOCATION_FILE (src_loc);
+      int sysp;
+      size_t to_file_len = strlen (file_path);
       unsigned char *to_file_quoted =
          (unsigned char *) alloca (to_file_len * 4 + 1);
       unsigned char *p;
 
-      print.src_line = SOURCE_LINE (map, src_loc);
-      print.src_file = map->to_file;
+      print.src_line = LOCATION_LINE (src_loc);
+      print.src_file = file_path;
 
       /* cpp_quote_string does not nul-terminate, so we have to do it
 	 ourselves.  */
       p = cpp_quote_string (to_file_quoted,
-			    (const unsigned char *) map->to_file, to_file_len);
+			    (const unsigned char *) file_path,
+			    to_file_len);
       *p = '\0';
       fprintf (print.outf, "# %u \"%s\"%s",
 	       print.src_line == 0 ? 1 : print.src_line,
 	       to_file_quoted, special_flags);
 
-      if (map->sysp == 2)
+      sysp = in_system_header_at (src_loc);
+      if (sysp == 2)
 	fputs (" 3 4", print.outf);
-      else if (map->sysp == 1)
+      else if (sysp == 1)
 	fputs (" 3", print.outf);
 
       putc ('\n', print.outf);
@@ -391,8 +390,7 @@ do_line_change (cpp_reader *pfile, const cpp_token *token,
      ought to care.  Some things do care; the fault lies with them.  */
   if (!CPP_OPTION (pfile, traditional))
     {
-      const struct line_map *map = linemap_lookup (line_table, src_loc);
-      int spaces = SOURCE_COLUMN (map, src_loc) - 2;
+      int spaces = LOCATION_COLUMN (src_loc) - 2;
       print.printed = 1;
 
       while (-- spaces >= 0)
@@ -421,6 +419,8 @@ cb_ident (cpp_reader *pfile ATTRIBUTE_UNUSED, source_location line,
 static void
 cb_define (cpp_reader *pfile, source_location line, cpp_hashnode *node)
 {
+  const struct line_map *map;
+
   maybe_print_line (line);
   fputs ("#define ", print.outf);
 
@@ -432,7 +432,10 @@ cb_define (cpp_reader *pfile, source_location line, cpp_hashnode *node)
     fputs ((const char *) NODE_NAME (node), print.outf);
 
   putc ('\n', print.outf);
-  if (linemap_lookup (line_table, line)->to_line != 0)
+  linemap_resolve_location (line_table, line,
+			    LRK_MACRO_DEFINITION_LOCATION,
+			    &map);
+  if (LINEMAP_LINE (map) != 0)
     print.src_line++;
 }
 
diff --git a/gcc/diagnostic.c b/gcc/diagnostic.c
index d297cdd..b46eb35 100644
--- a/gcc/diagnostic.c
+++ b/gcc/diagnostic.c
@@ -278,18 +278,18 @@ diagnostic_report_current_module (diagnostic_context *context)
 	  if (context->show_column)
 	    pp_verbatim (context->printer,
 			 "In file included from %s:%d:%d",
-			 map->to_file,
+			 LINEMAP_FILE (map),
 			 LAST_SOURCE_LINE (map), LAST_SOURCE_COLUMN (map));
 	  else
 	    pp_verbatim (context->printer,
 			 "In file included from %s:%d",
-			 map->to_file, LAST_SOURCE_LINE (map));
+			 LINEMAP_FILE (map), LAST_SOURCE_LINE (map));
 	  while (! MAIN_FILE_P (map))
 	    {
 	      map = INCLUDED_FROM (line_table, map);
 	      pp_verbatim (context->printer,
 			   ",\n                 from %s:%d",
-			   map->to_file, LAST_SOURCE_LINE (map));
+			   LINEMAP_FILE (map), LAST_SOURCE_LINE (map));
 	    }
 	  pp_verbatim (context->printer, ":");
 	  pp_newline (context->printer);
@@ -459,7 +459,10 @@ diagnostic_report_diagnostic (diagnostic_context *context,
 	  /* FIXME: Stupid search.  Optimize later. */
 	  for (i = context->n_classification_history - 1; i >= 0; i --)
 	    {
-	      if (context->classification_history[i].location <= location)
+	      if (linemap_location_before_p
+		  (line_table,
+		   context->classification_history[i].location,
+		   location))
 		{
 		  if (context->classification_history[i].kind == (int) DK_POP)
 		    {
diff --git a/gcc/fortran/ChangeLog b/gcc/fortran/ChangeLog
index 4866956..8d2f4d6 100644
--- a/gcc/fortran/ChangeLog
+++ b/gcc/fortran/ChangeLog
@@ -1,3 +1,9 @@
+2011-10-15  Tom Tromey  <tromey@redhat.com>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* cpp.c (print_line, cb_define): Adjust to avoid using internals
+	of struct line_map.  Use the public API instead.
+
 2011-10-17  Janus Weil  <janus@gcc.gnu.org>
 
 	PR fortran/47023
diff --git a/gcc/fortran/cpp.c b/gcc/fortran/cpp.c
index 9368d89..2f18893 100644
--- a/gcc/fortran/cpp.c
+++ b/gcc/fortran/cpp.c
@@ -818,27 +818,29 @@ print_line (source_location src_loc, const char *special_flags)
 
   if (!gfc_cpp_option.no_line_commands)
     {
-      const struct line_map *map = linemap_lookup (line_table, src_loc);
-
-      size_t to_file_len = strlen (map->to_file);
-      unsigned char *to_file_quoted =
-         (unsigned char *) alloca (to_file_len * 4 + 1);
+      expanded_location loc;
+      size_t to_file_len;
+      unsigned char *to_file_quoted;
       unsigned char *p;
 
-      print.src_line = SOURCE_LINE (map, src_loc);
+      loc = expand_location (src_loc);
+      to_file_len = strlen (loc.file);
+      to_file_quoted = (unsigned char *) alloca (to_file_len * 4 + 1);
+
+      print.src_line = loc.line;
 
       /* cpp_quote_string does not nul-terminate, so we have to do it
 	 ourselves.  */
       p = cpp_quote_string (to_file_quoted,
-			    (const unsigned char *) map->to_file, to_file_len);
+			    (const unsigned char *) loc.file, to_file_len);
       *p = '\0';
       fprintf (print.outf, "# %u \"%s\"%s",
 	       print.src_line == 0 ? 1 : print.src_line,
 	       to_file_quoted, special_flags);
 
-      if (map->sysp == 2)
+      if (loc.sysp == 2)
 	fputs (" 3 4", print.outf);
-      else if (map->sysp == 1)
+      else if (loc.sysp == 1)
 	fputs (" 3", print.outf);
 
       putc ('\n', print.outf);
@@ -935,7 +937,7 @@ cb_define (cpp_reader *pfile ATTRIBUTE_UNUSED, source_location line,
     fputs ((const char *) NODE_NAME (node), print.outf);
 
   putc ('\n', print.outf);
-  if (linemap_lookup (line_table, line)->to_line != 0)
+  if (LOCATION_LINE (line) != 0)
     print.src_line++;
 }
 
diff --git a/gcc/input.c b/gcc/input.c
index e5e051f..83344d7 100644
--- a/gcc/input.c
+++ b/gcc/input.c
@@ -42,12 +42,7 @@ expand_location (source_location loc)
       xloc.sysp = 0;
     }
   else
-    {
-      const struct line_map *map = linemap_lookup (line_table, loc);
-      xloc.file = map->to_file;
-      xloc.line = SOURCE_LINE (map, loc);
-      xloc.column = SOURCE_COLUMN (map, loc);
-      xloc.sysp = map->sysp != 0;
-    };
+    xloc = linemap_expand_location_full (line_table, loc,
+					 LRK_SPELLING_LOCATION);
   return xloc;
 }
diff --git a/gcc/input.h b/gcc/input.h
index 5929064..9fc55f3 100644
--- a/gcc/input.h
+++ b/gcc/input.h
@@ -37,20 +37,6 @@ extern GTY(()) struct line_maps *line_table;
 extern char builtins_location_check[(BUILTINS_LOCATION
 				     < RESERVED_LOCATION_COUNT) ? 1 : -1];
 
-typedef struct
-{
-  /* The name of the source file involved.  */
-  const char *file;
-
-  /* The line-location in the source file.  */
-  int line;
-
-  int column;
-
-  /* In a system header?. */
-  bool sysp;
-} expanded_location;
-
 extern expanded_location expand_location (source_location);
 
 /* Historically GCC used location_t, while cpp used source_location.
@@ -61,10 +47,12 @@ extern location_t input_location;
 
 #define LOCATION_FILE(LOC) ((expand_location (LOC)).file)
 #define LOCATION_LINE(LOC) ((expand_location (LOC)).line)
+#define LOCATION_COLUMN(LOC)((expand_location (LOC)).column)
 
 #define input_line LOCATION_LINE (input_location)
 #define input_filename LOCATION_FILE (input_location)
-#define in_system_header_at(LOC) ((expand_location (LOC)).sysp != 0)
+#define in_system_header_at(LOC) \
+  ((linemap_location_in_system_header_p (line_table, LOC)))
 #define in_system_header (in_system_header_at (input_location))
 
 #endif
diff --git a/gcc/java/ChangeLog b/gcc/java/ChangeLog
index dfb6500..ac83a40 100644
--- a/gcc/java/ChangeLog
+++ b/gcc/java/ChangeLog
@@ -1,3 +1,8 @@
+2011-10-15  Tom Tromey  <tromey@redhat.com>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* jcf-parse.c (set_source_filename): Adjust to the new map API.
+
 2011-10-11  Michael Meissner  <meissner@linux.vnet.ibm.com>
 
 	* class.c (build_static_field_ref): Delete old interface with two
diff --git a/gcc/java/jcf-parse.c b/gcc/java/jcf-parse.c
index 37cea28..04c04f5 100644
--- a/gcc/java/jcf-parse.c
+++ b/gcc/java/jcf-parse.c
@@ -355,7 +355,7 @@ set_source_filename (JCF *jcf, int index)
     }
       
   sfname = find_sourcefile (sfname);
-  line_table->maps[line_table->used-1].to_file = sfname;
+  ORDINARY_MAP_FILE_NAME (LINEMAPS_LAST_ORDINARY_MAP (line_table)) = sfname;
   if (current_class == main_class) main_input_filename = sfname;
 }
 
diff --git a/gcc/testsuite/ChangeLog b/gcc/testsuite/ChangeLog
index 11ae542..e2b4fe1 100644
--- a/gcc/testsuite/ChangeLog
+++ b/gcc/testsuite/ChangeLog
@@ -1,3 +1,8 @@
+2011-10-15  Tom Tromey  <tromey@redhat.com>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* gcc.dg/cpp/pragma-diagnostic-1.c: New test.
+
 2011-10-17  Paolo Carlini  <paolo.carlini@oracle.com>
 
 	PR c++/48489
diff --git a/gcc/testsuite/gcc.dg/cpp/pragma-diagnostic-1.c b/gcc/testsuite/gcc.dg/cpp/pragma-diagnostic-1.c
new file mode 100644
index 0000000..3a2f9da
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/cpp/pragma-diagnostic-1.c
@@ -0,0 +1,32 @@
+/*
+  { dg-options "-Wuninitialized" }
+  { dg-do compile }
+*/
+
+void f (unsigned);
+
+#define CODE_WITH_WARNING \
+  int a;		  \
+  f (a)
+
+#pragma GCC diagnostic ignored "-Wuninitialized"
+
+void
+g (void)
+{
+  CODE_WITH_WARNING;
+}
+
+#pragma GCC diagnostic push
+
+#pragma GCC diagnostic error "-Wuninitialized"
+
+void
+h (void)
+{
+  CODE_WITH_WARNING;		/* { dg-error "uninitialized" } */
+}
+
+/*
+  { dg-message "some warnings being treated as errors" "" {target *-*-*} 0 }
+*/