Source range tracking in libcpp and C FE, with bit-packing optimization

This patch combines:
  [PATCH 05/10] Add ranges to libcpp tokens (via ad-hoc data, unoptimized)
  [PATCH 06/10] Track expression ranges in C frontend
  [PATCH 07/10] Add plugin to recursively dump the source-ranges in a tree (v2)
  [PATCH 08/10] Wire things up so that libcpp users get token underlines
  [PATCH 09/10] Delay some resolution of ad-hoc locations, preserving ranges
  [PATCH 10/10] Compress short ranges into source_location
  [PATCH] libcpp: add examples to source_location description
along with fixes for the nits identified during review.

gcc/ChangeLog:
	* Makefile.in (OBJS): Add gcc-rich-location.o.
	* diagnostic.c (diagnostic_append_note): Pass line_table to
	rich_location ctor.
	(emit_diagnostic): Likewise.
	(inform): Likewise.
	(inform_n): Likewise.
	(warning): Likewise.
	(warning_at): Likewise.
	(warning_n): Likewise.
	(pedwarn): Likewise.
	(permerror): Likewise.
	(error): Likewise.
	(error_n): Likewise.
	(error_at): Likewise.
	(sorry): Likewise.
	(fatal_error): Likewise.
	(internal_error): Likewise.
	(internal_error_no_backtrace): Likewise.
	(source_range::debug): Likewise.
	* gcc-rich-location.c: New file.
	* gcc-rich-location.h: New file.
	* genmatch.c (fatal_at): Pass line_table to rich_location ctor.
	(warning_at): Likewise.
	* gimple.h (gimple_set_block): Use set_block function.
	* input.c (dump_line_table_statistics): Dump stats on how many
	ranges were optimized vs how many needed ad-hoc table.
	(write_digit_row): Add "map" param; use its range_bits
	to calculate the per-character offset.
	(dump_location_info): Print the range and column bits for each
	ordinary map.  Use the range bits to calculate the per-character
	offset.  Pass the map as a new param to the various calls to
	write_digit_row.  Eliminate uses of
	ORDINARY_MAP_NUMBER_OF_COLUMN_BITS.
	* print-tree.c (print_node): Print any source range information.
	* rtl-error.c (diagnostic_for_asm): Likewise.
	* toplev.c (general_init): Initialize line_table's
	default_range_bits.
	* tree-cfg.c (move_block_to_fn): Likewise.
	(move_block_to_fn): Likewise.
	* tree-inline.c (copy_phis_for_bb): Likewise.
	* tree.c (tree_set_block): Likewise.
	(get_pure_location): New function.
	(set_source_range): New functions.
	(set_block): New function.
	(set_source_range): New functions.
	* tree.h (CAN_HAVE_RANGE_P): New.
	(EXPR_LOCATION_RANGE): New.
	(EXPR_HAS_RANGE): New.
	(get_expr_source_range): New inline function.
	(DECL_LOCATION_RANGE): New.
	(set_source_range): New decls.
	(get_decl_source_range): New inline function.

gcc/ada/ChangeLog:
	* gcc-interface/trans.c (Sloc_to_locus): Add line_table param when
	calling linemap_position_for_line_and_column.

gcc/c-family/ChangeLog:
	* c-common.c (c_fully_fold_internal): Capture existing souce_range,
	and store it on the result.
	* c-opts.c (c_common_init_options): Set
	global_dc->colorize_source_p.

gcc/c/ChangeLog:
	* c-decl.c (warn_defaults_to): Pass line_table to
	rich_location ctor.
	* c-errors.c (pedwarn_c99): Likewise.
	(pedwarn_c90): Likewise.
	* c-parser.c (set_c_expr_source_range): New functions.
	(c_token::get_range): New method.
	(c_token::get_finish): New method.
	(c_parser_expr_no_commas): Call set_c_expr_source_range on the ret
	based on the range from the start of the LHS to the end of the
	RHS.
	(c_parser_conditional_expression): Likewise, based on the range
	from the start of the cond.value to the end of exp2.value.
	(c_parser_binary_expression): Call set_c_expr_source_range on
	the stack values for TRUTH_ANDIF_EXPR and TRUTH_ORIF_EXPR.
	(c_parser_cast_expression): Call set_c_expr_source_range on ret
	based on the cast_loc through to the end of the expr.
	(c_parser_unary_expression): Likewise, based on the
	op_loc through to the end of op.
	(c_parser_sizeof_expression) Likewise, based on the start of the
	sizeof token through to either the closing paren or the end of
	expr.
	(c_parser_postfix_expression): Likewise, using the token range,
	or from the open paren through to the close paren for
	parenthesized expressions.
	(c_parser_postfix_expression_after_primary): Likewise, for
	various kinds of expression.
	* c-tree.h (struct c_expr): Add field "src_range".
	(c_expr::get_start): New method.
	(c_expr::get_finish): New method.
	(set_c_expr_source_range): New decls.
	* c-typeck.c (parser_build_unary_op): Call set_c_expr_source_range
	on ret for prefix unary ops.
	(parser_build_binary_op): Likewise, running from the start of
	arg1.value through to the end of arg2.value.

gcc/cp/ChangeLog:
	* error.c (pedwarn_cxx98): Pass line_table to rich_location ctor.

gcc/fortran/ChangeLog:
	* error.c (gfc_warning): Pass line_table to rich_location ctor.
	(gfc_warning_now_at): Likewise.
	(gfc_warning_now): Likewise.
	(gfc_error_now): Likewise.
	(gfc_fatal_error): Likewise.
	(gfc_error): Likewise.
	(gfc_internal_error): Likewise.

gcc/testsuite/ChangeLog:
	* gcc.dg/diagnostic-token-ranges.c: New file.
	* gcc.dg/diagnostic-tree-expr-ranges-2.c: New file.
	* gcc.dg/plugin/diagnostic-test-expressions-1.c: New file.
	* gcc.dg/plugin/diagnostic-test-show-trees-1.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_show_trees.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (get_loc): Add
	line_table param when calling
	linemap_position_for_line_and_column.
	(test_show_locus): Pass line_table to rich_location ctors.
	(plugin_init): Remove setting of global_dc->colorize_source_p.
	* gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c:
	New file.
	* gcc.dg/plugin/plugin.exp (plugin_test_list): Add
	diagnostic_plugin_test_tree_expression_range.c,
	diagnostic-test-expressions-1.c, diagnostic_plugin_show_trees.c,
	and diagnostic-test-show-trees-1.c.

libcpp/ChangeLog:
	* errors.c (cpp_diagnostic): Pass pfile->line_table to
	rich_location ctor.
	(cpp_diagnostic_with_line): Likewise.
	* include/cpplib.h (struct cpp_token): Update comment for src_loc
	to indicate that the range of the token is "baked into" the
	source_location.
	* include/line-map.h (source_location): Update the descriptive
	comment to reflect the packing scheme for short ranges, adding
	worked examples of location encoding.
	(struct line_map_ordinary): Drop field "column_bits" in favor
	of field "m_column_and_range_bits"; add field "m_range_bits".
	(ORDINARY_MAP_NUMBER_OF_COLUMN_BITS): Delete.
	(location_adhoc_data): Add source_range field.
	(struct line_maps): Add fields "default_range_bits",
	"num_optimized_ranges" and "num_unoptimized_ranges".
	(get_combined_adhoc_loc): Add source_range param.
	(get_range_from_loc): New declaration.
	(pure_location_p): New prototype.
	(COMBINE_LOCATION_DATA):  Add source_range param.
	(SOURCE_LINE): Update for renaming of column_bits.
	(SOURCE_COLUMN): Likewise.  Shift the column right by the map's
	range_bits.
	(LAST_SOURCE_LINE_LOCATION): Update for renaming of column_bits.
	(linemap_position_for_line_and_column): Add line_maps * params.
	(rich_location::rich_location): Likewise.
	* lex.c (_cpp_lex_direct): Capture the range of the token, baking
	it into token->src_loc via a call to COMBINE_LOCATION_DATA.
	* line-map.c (LINE_MAP_MAX_COLUMN_NUMBER): Reduce from 1U << 17 to
	1U << 12.
	(location_adhoc_data_hash): Add the src_range into
	the hash value.
	(location_adhoc_data_eq): Require equality of the src_range
	values.
	(can_be_stored_compactly_p): New function.
	(get_combined_adhoc_loc): Add src_range param, and store it,
	via a bit-packing scheme for short ranges, otherwise within the
	lookaside table.  Remove the requirement that data is non-NULL.
	(get_range_from_adhoc_loc): New function.
	(get_range_from_loc): New function.
	(pure_location_p): New function.
	(linemap_add): Ensure that start_location has zero for the
	range_bits, unless we're past LINE_MAP_MAX_LOCATION_WITH_COLS.
	Initialize range_bits to zero.  Assert that the start_location
	is "pure".
	(linemap_line_start): Assert that the
	column_and_range_bits >= range_bits.
	Update determinination of whether we need to start a new map
	using the effective column bits, without the range bits.
	Use the set's default_range_bits in new maps, apart from
	those with column_bits == 0, which should also have 0 range_bits.
	Increase the column bits for new maps by the range bits.
	When adding lines to an existing map, use set->highest_line
	directly rather than offsetting highest by SOURCE_COLUMN.
	Add assertions to sanity-check the return value.
	(linemap_position_for_column): Offset to_column by range_bits.
	Update set->highest_location if necessary.
	(linemap_position_for_line_and_column): Add line_maps * param.
	Update the calculation to offset the column by range_bits, and
	conditionalize it on being <= LINE_MAP_MAX_LOCATION_WITH_COLS.
	Bound it by LINEMAPS_MACRO_LOWEST_LOCATION.  Update
	set->highest_location if necessary.
	(linemap_position_for_loc_and_offset): Handle ad-hoc locations;
	pass "set" to linemap_position_for_line_and_column.
	(linemap_macro_map_loc_unwind_toward_spelling): Add line_maps
	param.  Handle ad-hoc locations.
	(linemap_location_in_system_header_p): Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_macro_loc_to_spelling_point): Retain ad-hoc locations.
	Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_resolve_location): Retain ad-hoc locations.  Pass on
	"set" to call to linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_unwind_toward_expansion):  Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_expand_location): Extract the data pointer before
	extracting the location.
	(rich_location::rich_location): Add line_maps param; use it to
	extract the range from the source_location.
	* location-example.txt: Regenerate, showing new representation.

From-SVN: r230331

1 files changed, 233 insertions, 41 deletions

diff --git a/libcpp/line-map.c b/libcpp/line-map.c
index 3c19f93..c5aa422 100644
--- a/libcpp/line-map.c
+++ b/libcpp/line-map.c
@@ -27,9 +27,9 @@ along with this program; see the file COPYING3.  If not see
 #include "hashtab.h"
 
 /* Do not track column numbers higher than this one.  As a result, the
-   range of column_bits is [7, 18] (or 0 if column numbers are
+   range of column_bits is [12, 18] (or 0 if column numbers are
    disabled).  */
-const unsigned int LINE_MAP_MAX_COLUMN_NUMBER = (1U << 17);
+const unsigned int LINE_MAP_MAX_COLUMN_NUMBER = (1U << 12);
 
 /* Do not track column numbers if locations get higher than this.  */
 const source_location LINE_MAP_MAX_LOCATION_WITH_COLS = 0x60000000;
@@ -46,7 +46,7 @@ static const line_map_macro* linemap_macro_map_lookup (struct line_maps *,
 static source_location linemap_macro_map_loc_to_def_point
 (const line_map_macro *, source_location);
 static source_location linemap_macro_map_loc_unwind_toward_spelling
-(const line_map_macro *, source_location);
+(line_maps *set, const line_map_macro *, source_location);
 static source_location linemap_macro_map_loc_to_exp_point
 (const line_map_macro *, source_location);
 static source_location linemap_macro_loc_to_spelling_point
@@ -69,7 +69,10 @@ location_adhoc_data_hash (const void *l)
 {
   const struct location_adhoc_data *lb =
       (const struct location_adhoc_data *) l;
-  return (hashval_t) lb->locus + (size_t) lb->data;
+  return ((hashval_t) lb->locus
+	  + (hashval_t) lb->src_range.m_start
+	  + (hashval_t) lb->src_range.m_finish
+	  + (size_t) lb->data);
 }
 
 /* Compare function for location_adhoc_data hashtable.  */
@@ -81,7 +84,10 @@ location_adhoc_data_eq (const void *l1, const void *l2)
       (const struct location_adhoc_data *) l1;
   const struct location_adhoc_data *lb2 =
       (const struct location_adhoc_data *) l2;
-  return lb1->locus == lb2->locus && lb1->data == lb2->data;
+  return (lb1->locus == lb2->locus
+	  && lb1->src_range.m_start == lb2->src_range.m_start
+	  && lb1->src_range.m_finish == lb2->src_range.m_finish
+	  && lb1->data == lb2->data);
 }
 
 /* Update the hashtable when location_adhoc_data is reallocated.  */
@@ -106,23 +112,103 @@ rebuild_location_adhoc_htab (struct line_maps *set)
 		    set->location_adhoc_data_map.data + i, INSERT);
 }
 
+/* Helper function for get_combined_adhoc_loc.
+   Can the given LOCUS + SRC_RANGE and DATA pointer be stored compactly
+   within a source_location, without needing to use an ad-hoc location.  */
+
+static bool
+can_be_stored_compactly_p (struct line_maps *set,
+			   source_location locus,
+			   source_range src_range,
+			   void *data)
+{
+  /* If there's an ad-hoc pointer, we can't store it directly in the
+     source_location, we need the lookaside.  */
+  if (data)
+    return false;
+
+  /* We only store ranges that begin at the locus and that are sufficiently
+     "sane".  */
+  if (src_range.m_start != locus)
+    return false;
+
+  if (src_range.m_finish < src_range.m_start)
+    return false;
+
+  if (src_range.m_start < RESERVED_LOCATION_COUNT)
+    return false;
+
+  if (locus >= LINE_MAP_MAX_LOCATION_WITH_COLS)
+    return false;
+
+  /* All 3 locations must be within ordinary maps, typically, the same
+     ordinary map.  */
+  source_location lowest_macro_loc = LINEMAPS_MACRO_LOWEST_LOCATION (set);
+  if (locus >= lowest_macro_loc)
+    return false;
+  if (src_range.m_start >= lowest_macro_loc)
+    return false;
+  if (src_range.m_finish >= lowest_macro_loc)
+    return false;
+
+  /* Passed all tests.  */
+  return true;
+}
+
 /* Combine LOCUS and DATA to a combined adhoc loc.  */
 
 source_location
 get_combined_adhoc_loc (struct line_maps *set,
-			source_location locus, void *data)
+			source_location locus,
+			source_range src_range,
+			void *data)
 {
   struct location_adhoc_data lb;
   struct location_adhoc_data **slot;
 
-  linemap_assert (data);
-
   if (IS_ADHOC_LOC (locus))
     locus
       = set->location_adhoc_data_map.data[locus & MAX_SOURCE_LOCATION].locus;
   if (locus == 0 && data == NULL)
     return 0;
+
+  /* Any ordinary locations ought to be "pure" at this point: no
+     compressed ranges.  */
+  linemap_assert (locus < RESERVED_LOCATION_COUNT
+		  || locus >= LINE_MAP_MAX_LOCATION_WITH_COLS
+		  || locus >= LINEMAPS_MACRO_LOWEST_LOCATION (set)
+		  || pure_location_p (set, locus));
+
+  /* Consider short-range optimization.  */
+  if (can_be_stored_compactly_p (set, locus, src_range, data))
+    {
+      /* The low bits ought to be clear.  */
+      linemap_assert (pure_location_p (set, locus));
+      const line_map *map = linemap_lookup (set, locus);
+      const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+      unsigned int int_diff = src_range.m_finish - src_range.m_start;
+      unsigned int col_diff = (int_diff >> ordmap->m_range_bits);
+      if (col_diff < (1U << ordmap->m_range_bits))
+	{
+	  source_location packed = locus | col_diff;
+	  set->num_optimized_ranges++;
+	  return packed;
+	}
+    }
+
+  /* We can also compactly store the reserved locations
+     when locus == start == finish (and data is NULL).  */
+  if (locus < RESERVED_LOCATION_COUNT
+      && locus == src_range.m_start
+      && locus == src_range.m_finish
+      && !data)
+    return locus;
+
+  if (!data)
+    set->num_unoptimized_ranges++;
+
   lb.locus = locus;
+  lb.src_range = src_range;
   lb.data = data;
   slot = (struct location_adhoc_data **)
       htab_find_slot (set->location_adhoc_data_map.htab, &lb, INSERT);
@@ -177,6 +263,60 @@ get_location_from_adhoc_loc (struct line_maps *set, source_location loc)
   return set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
 }
 
+/* Return the source_range for adhoc location LOC.  */
+
+static source_range
+get_range_from_adhoc_loc (struct line_maps *set, source_location loc)
+{
+  linemap_assert (IS_ADHOC_LOC (loc));
+  return set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].src_range;
+}
+
+/* Get the source_range of location LOC, either from the ad-hoc
+   lookaside table, or embedded inside LOC itself.  */
+
+source_range
+get_range_from_loc (struct line_maps *set,
+		    source_location loc)
+{
+  if (IS_ADHOC_LOC (loc))
+    return get_range_from_adhoc_loc (set, loc);
+
+  /* For ordinary maps, extract packed range.  */
+  if (loc >= RESERVED_LOCATION_COUNT
+      && loc < LINEMAPS_MACRO_LOWEST_LOCATION (set)
+      && loc <= LINE_MAP_MAX_LOCATION_WITH_COLS)
+    {
+      const line_map *map = linemap_lookup (set, loc);
+      const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+      source_range result;
+      int offset = loc & ((1 << ordmap->m_range_bits) - 1);
+      result.m_start = loc - offset;
+      result.m_finish = result.m_start + (offset << ordmap->m_range_bits);
+      return result;
+    }
+
+  return source_range::from_location (loc);
+}
+
+/* Get whether location LOC is a "pure" location, or
+   whether it is an ad-hoc location, or embeds range information.  */
+
+bool
+pure_location_p (line_maps *set, source_location loc)
+{
+  if (IS_ADHOC_LOC (loc))
+    return false;
+
+  const line_map *map = linemap_lookup (set, loc);
+  const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+
+  if (loc & ((1U << ordmap->m_range_bits) - 1))
+    return false;
+
+  return true;
+}
+
 /* Finalize the location_adhoc_data structure.  */
 void
 location_adhoc_data_fini (struct line_maps *set)
@@ -319,7 +459,19 @@ const struct line_map *
 linemap_add (struct line_maps *set, enum lc_reason reason,
 	     unsigned int sysp, const char *to_file, linenum_type to_line)
 {
-  source_location start_location = set->highest_location + 1;
+  /* Generate a start_location above the current highest_location.
+     If possible, make the low range bits be zero.  */
+  source_location start_location;
+  if (set->highest_location < LINE_MAP_MAX_LOCATION_WITH_COLS)
+    {
+      start_location = set->highest_location + (1 << set->default_range_bits);
+      if (set->default_range_bits)
+	start_location &= ~((1 << set->default_range_bits) - 1);
+      linemap_assert (0 == (start_location
+			    & ((1 << set->default_range_bits) - 1)));
+    }
+  else
+    start_location = set->highest_location + 1;
 
   linemap_assert (!(LINEMAPS_ORDINARY_USED (set)
 		    && (start_location
@@ -398,11 +550,18 @@ linemap_add (struct line_maps *set, enum lc_reason reason,
   map->to_file = to_file;
   map->to_line = to_line;
   LINEMAPS_ORDINARY_CACHE (set) = LINEMAPS_ORDINARY_USED (set) - 1;
-  map->column_bits = 0;
+  map->m_column_and_range_bits = 0;
+  map->m_range_bits = 0;
   set->highest_location = start_location;
   set->highest_line = start_location;
   set->max_column_hint = 0;
 
+  /* This assertion is placed after set->highest_location has
+     been updated, since the latter affects
+     linemap_location_from_macro_expansion_p, which ultimately affects
+     pure_location_p.  */
+  linemap_assert (pure_location_p (set, start_location));
+
   if (reason == LC_ENTER)
     {
       map->included_from =
@@ -549,13 +708,14 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
     SOURCE_LINE (map, set->highest_line);
   int line_delta = to_line - last_line;
   bool add_map = false;
+  linemap_assert (map->m_column_and_range_bits >= map->m_range_bits);
+  int effective_column_bits = map->m_column_and_range_bits - map->m_range_bits;
 
   if (line_delta < 0
       || (line_delta > 10
-	  && line_delta * ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) > 1000)
-      || (max_column_hint >= (1U << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)))
-      || (max_column_hint <= 80
-	  && ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) >= 10)
+	  && line_delta * map->m_column_and_range_bits > 1000)
+      || (max_column_hint >= (1U << effective_column_bits))
+      || (max_column_hint <= 80 && effective_column_bits >= 10)
       || (highest > LINE_MAP_MAX_LOCATION_WITH_COLS
 	  && (set->max_column_hint || highest >= LINE_MAP_MAX_SOURCE_LOCATION)))
     add_map = true;
@@ -564,22 +724,27 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
   if (add_map)
     {
       int column_bits;
+      int range_bits;
       if (max_column_hint > LINE_MAP_MAX_COLUMN_NUMBER
 	  || highest > LINE_MAP_MAX_LOCATION_WITH_COLS)
 	{
 	  /* If the column number is ridiculous or we've allocated a huge
-	     number of source_locations, give up on column numbers. */
+	     number of source_locations, give up on column numbers
+	     (and on packed ranges).  */
 	  max_column_hint = 0;
 	  column_bits = 0;
+	  range_bits = 0;
 	  if (highest > LINE_MAP_MAX_SOURCE_LOCATION)
 	    return 0;
 	}
       else
 	{
 	  column_bits = 7;
+	  range_bits = set->default_range_bits;
 	  while (max_column_hint >= (1U << column_bits))
 	    column_bits++;
 	  max_column_hint = 1U << column_bits;
+	  column_bits += range_bits;
 	}
       /* Allocate the new line_map.  However, if the current map only has a
 	 single line we can sometimes just increase its column_bits instead. */
@@ -592,14 +757,14 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
 				ORDINARY_MAP_IN_SYSTEM_HEADER_P (map),
 				ORDINARY_MAP_FILE_NAME (map),
 				to_line)));
-      map->column_bits = column_bits;
+      map->m_column_and_range_bits = column_bits;
+      map->m_range_bits = range_bits;
       r = (MAP_START_LOCATION (map)
 	   + ((to_line - ORDINARY_MAP_STARTING_LINE_NUMBER (map))
 	      << column_bits));
     }
   else
-    r = highest - SOURCE_COLUMN (map, highest)
-      + (line_delta << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map));
+    r = set->highest_line + (line_delta << map->m_column_and_range_bits);
 
   /* Locations of ordinary tokens are always lower than locations of
      macro tokens.  */
@@ -610,6 +775,18 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
   if (r > set->highest_location)
     set->highest_location = r;
   set->max_column_hint = max_column_hint;
+
+  /* At this point, we expect one of:
+     (a) the normal case: a "pure" location with 0 range bits, or
+     (b) we've gone past LINE_MAP_MAX_LOCATION_WITH_COLS so can't track
+        columns anymore (or ranges), or
+     (c) we're in a region with a column hint exceeding
+        LINE_MAP_MAX_COLUMN_NUMBER, so column-tracking is off,
+	with column_bits == 0.  */
+  linemap_assert (pure_location_p (set, r)
+		  || r >= LINE_MAP_MAX_LOCATION_WITH_COLS
+		  || map->m_column_and_range_bits == 0);
+  linemap_assert (SOURCE_LINE (map, r) == to_line);
   return r;
 }
 
@@ -640,7 +817,8 @@ linemap_position_for_column (struct line_maps *set, unsigned int to_column)
 	  r = linemap_line_start (set, SOURCE_LINE (map, r), to_column + 50);
 	}
     }
-  r = r + to_column;
+  line_map_ordinary *map = LINEMAPS_LAST_ORDINARY_MAP (set);
+  r = r + (to_column << map->m_range_bits);
   if (r >= set->highest_location)
     set->highest_location = r;
   return r;
@@ -650,16 +828,25 @@ linemap_position_for_column (struct line_maps *set, unsigned int to_column)
    column.  */
 
 source_location
-linemap_position_for_line_and_column (const line_map_ordinary *ord_map,
+linemap_position_for_line_and_column (line_maps *set,
+				      const line_map_ordinary *ord_map,
 				      linenum_type line,
 				      unsigned column)
 {
   linemap_assert (ORDINARY_MAP_STARTING_LINE_NUMBER (ord_map) <= line);
 
-  return (MAP_START_LOCATION (ord_map)
-	  + ((line - ORDINARY_MAP_STARTING_LINE_NUMBER (ord_map))
-	     << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (ord_map))
-	  + (column & ((1 << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (ord_map)) - 1)));
+  source_location r = MAP_START_LOCATION (ord_map);
+  r += ((line - ORDINARY_MAP_STARTING_LINE_NUMBER (ord_map))
+	<< ord_map->m_column_and_range_bits);
+  if (r <= LINE_MAP_MAX_LOCATION_WITH_COLS)
+    r += ((column & ((1 << ord_map->m_column_and_range_bits) - 1))
+	  << ord_map->m_range_bits);
+  source_location upper_limit = LINEMAPS_MACRO_LOWEST_LOCATION (set);
+  if (r >= upper_limit)
+    r = upper_limit - 1;
+  if (r > set->highest_location)
+    set->highest_location = r;
+  return r;
 }
 
 /* Encode and return a source_location starting from location LOC and
@@ -673,6 +860,9 @@ linemap_position_for_loc_and_offset (struct line_maps *set,
 {
   const line_map_ordinary * map = NULL;
 
+  if (IS_ADHOC_LOC (loc))
+    loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
+
   /* This function does not support virtual locations yet.  */
   if (linemap_assert_fails
       (!linemap_location_from_macro_expansion_p (set, loc)))
@@ -711,11 +901,11 @@ linemap_position_for_loc_and_offset (struct line_maps *set,
     }
 
   offset += column;
-  if (linemap_assert_fails (offset < (1u << map->column_bits)))
+  if (linemap_assert_fails (offset < (1u << map->m_column_and_range_bits)))
     return loc;
 
   source_location r = 
-    linemap_position_for_line_and_column (map, line, offset);
+    linemap_position_for_line_and_column (set, map, line, offset);
   if (linemap_assert_fails (r <= set->highest_location)
       || linemap_assert_fails (map == linemap_lookup (set, r)))
     return loc;
@@ -893,14 +1083,19 @@ linemap_macro_map_loc_to_def_point (const line_map_macro *map,
    In other words, this returns the xI location presented in the
    comments of line_map_macro above.  */
 source_location
-linemap_macro_map_loc_unwind_toward_spelling (const line_map_macro* map,
+linemap_macro_map_loc_unwind_toward_spelling (line_maps *set,
+					      const line_map_macro* map,
 					      source_location location)
 {
   unsigned token_no;
 
+  if (IS_ADHOC_LOC (location))
+    location = get_location_from_adhoc_loc (set, location);
+
   linemap_assert (linemap_macro_expansion_map_p (map)
 		  && location >= MAP_START_LOCATION (map));
   linemap_assert (location >= RESERVED_LOCATION_COUNT);
+  linemap_assert (!IS_ADHOC_LOC (location));
 
   token_no = location - MAP_START_LOCATION (map);
   linemap_assert (token_no < MACRO_MAP_NUM_MACRO_TOKENS (map));
@@ -1010,7 +1205,7 @@ linemap_location_in_system_header_p (struct line_maps *set,
 
 	      /* It's a token resulting from a macro expansion.  */
 	      source_location loc =
-		linemap_macro_map_loc_unwind_toward_spelling (macro_map, location);
+		linemap_macro_map_loc_unwind_toward_spelling (set, macro_map, location);
 	      if (loc < RESERVED_LOCATION_COUNT)
 		/* This token might come from a built-in macro.  Let's
 		   look at where that macro got expanded.  */
@@ -1183,11 +1378,6 @@ linemap_macro_loc_to_spelling_point (struct line_maps *set,
 				     const line_map_ordinary **original_map)
 {
   struct line_map *map;
-
-  if (IS_ADHOC_LOC (location))
-    location = set->location_adhoc_data_map.data[location
-						 & MAX_SOURCE_LOCATION].locus;
-
   linemap_assert (set && location >= RESERVED_LOCATION_COUNT);
 
   while (true)
@@ -1198,7 +1388,7 @@ linemap_macro_loc_to_spelling_point (struct line_maps *set,
 
       location
 	= linemap_macro_map_loc_unwind_toward_spelling
-	    (linemap_check_macro (map),
+	    (set, linemap_check_macro (map),
 	     location);
     }
 
@@ -1341,10 +1531,11 @@ linemap_resolve_location (struct line_maps *set,
 			  enum location_resolution_kind lrk,
 			  const line_map_ordinary **map)
 {
+  source_location locus = loc;
   if (IS_ADHOC_LOC (loc))
-    loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
+    locus = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
 
-  if (loc < RESERVED_LOCATION_COUNT)
+  if (locus < RESERVED_LOCATION_COUNT)
     {
       /* A reserved location wasn't encoded in a map.  Let's return a
 	 NULL map here, just like what linemap_ordinary_map_lookup
@@ -1396,7 +1587,7 @@ linemap_unwind_toward_expansion (struct line_maps *set,
     loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
 
   resolved_location =
-    linemap_macro_map_loc_unwind_toward_spelling (macro_map, loc);
+    linemap_macro_map_loc_unwind_toward_spelling (set, macro_map, loc);
   resolved_map = linemap_lookup (set, resolved_location);
 
   if (!linemap_macro_expansion_map_p (resolved_map))
@@ -1478,9 +1669,9 @@ linemap_expand_location (struct line_maps *set,
   memset (&xloc, 0, sizeof (xloc));
   if (IS_ADHOC_LOC (loc))
     {
-      loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
       xloc.data
 	= set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].data;
+      loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
     }
 
   if (loc < RESERVED_LOCATION_COUNT)
@@ -1760,13 +1951,14 @@ line_table_dump (FILE *stream, struct line_maps *set, unsigned int num_ordinary,
 
 /* Construct a rich_location with location LOC as its initial range.  */
 
-rich_location::rich_location (source_location loc) :
+rich_location::rich_location (line_maps *set, source_location loc) :
   m_loc (loc),
   m_num_ranges (0),
   m_have_expanded_location (false)
 {
-  /* Set up the 0th range: */
-  add_range (loc, loc, true);
+  /* Set up the 0th range, extracting any range from LOC.  */
+  source_range src_range = get_range_from_loc (set, loc);
+  add_range (src_range, true);
   m_ranges[0].m_caret = lazily_expand_location ();
 }