New IPA-SRA

2019-09-20  Martin Jambor  <mjambor@suse.cz>

        * coretypes.h (cgraph_edge): Declare.
        * ipa-param-manipulation.c: Rewrite.
        * ipa-param-manipulation.h: Likewise.
        * Makefile.in (GTFILES): Added ipa-param-manipulation.h and ipa-sra.c.
        (OBJS): Added ipa-sra.o.
        * cgraph.h (ipa_replace_map): Removed fields old_tree, replace_p
        and ref_p, added fields param_adjustments and performed_splits.
        (struct cgraph_clone_info): Remove ags_to_skip and
        combined_args_to_skip, new field param_adjustments.
        (cgraph_node::create_clone): Changed parameters to use
        ipa_param_adjustments.
        (cgraph_node::create_virtual_clone): Likewise.
        (cgraph_node::create_virtual_clone_with_body): Likewise.
        (tree_function_versioning): Likewise.
        (cgraph_build_function_type_skip_args): Removed.
        * cgraph.c (cgraph_edge::redirect_call_stmt_to_callee): Convert to
        using ipa_param_adjustments.
        (clone_of_p): Likewise.
        * cgraphclones.c (cgraph_build_function_type_skip_args): Removed.
        (build_function_decl_skip_args): Likewise.
        (duplicate_thunk_for_node): Adjust parameters using
        ipa_param_body_adjustments, copy param_adjustments instead of
        args_to_skip.
        (cgraph_node::create_clone): Convert to using ipa_param_adjustments.
        (cgraph_node::create_virtual_clone): Likewise.
        (cgraph_node::create_version_clone_with_body): Likewise.
        (cgraph_materialize_clone): Likewise.
        (symbol_table::materialize_all_clones): Likewise.
        * ipa-fnsummary.c (ipa_fn_summary_t::duplicate): Simplify
        ipa_replace_map check.
        * ipa-cp.c (get_replacement_map): Do not initialize removed fields.
        (initialize_node_lattices): Make aware that some parameters might have
        already been removed.
        (want_remove_some_param_p): New function.
        (create_specialized_node): Convert to using ipa_param_adjustments and
        deal with possibly pre-existing adjustments.
        * lto-cgraph.c (output_cgraph_opt_summary_p): Likewise.
        (output_node_opt_summary): Do not stream removed fields.  Stream
        parameter adjustments instead of argumetns to skip.
        (input_node_opt_summary): Likewise.
        (input_node_opt_summary): Likewise.
        * lto-section-in.c (lto_section_name): Added ipa-sra section.
        * lto-streamer.h (lto_section_type): Likewise.
        * tree-inline.h (copy_body_data): New fields killed_new_ssa_names and
	param_body_adjs.
        (copy_decl_to_var): Declare.
        * tree-inline.c (update_clone_info): Do not remap old_tree.
        (remap_gimple_stmt): Use ipa_param_body_adjustments to modify gimple
        statements, walk all extra generated statements and remap their
        operands.
        (redirect_all_calls): Add killed SSA names to a hash set.
        (remap_ssa_name): Do not remap killed SSA names.
        (copy_arguments_for_versioning): Renames to copy_arguments_nochange,
        half of functionality moved to ipa_param_body_adjustments.
        (copy_decl_to_var): Make exported.
        (copy_body): Destroy killed_new_ssa_names hash set.
        (expand_call_inline): Remap performed splits.
        (update_clone_info): Likewise.
        (tree_function_versioning): Simplify tree_map processing.  Updated to
        accept ipa_param_adjustments and use ipa_param_body_adjustments.
        * omp-simd-clone.c (simd_clone_vector_of_formal_parm_types): Adjust
        for the new interface.
        (simd_clone_clauses_extract): Likewise, make args an auto_vec.
        (simd_clone_compute_base_data_type): Likewise.
        (simd_clone_init_simd_arrays): Adjust for the new interface.
        (simd_clone_adjust_argument_types): Likewise.
        (struct modify_stmt_info): Likewise.
        (ipa_simd_modify_stmt_ops): Likewise.
        (ipa_simd_modify_function_body): Likewise.
        (simd_clone_adjust): Likewise.
        * tree-sra.c: Removed IPA-SRA.  Include tree-sra.h.
        (type_internals_preclude_sra_p): Make public.
        * tree-sra.h: New file.
        * ipa-inline-transform.c (save_inline_function_body): Update to
        refelct new tree_function_versioning signature.
        * ipa-prop.c (adjust_agg_replacement_values): Use a helper from
        ipa_param_adjustments to get current parameter indices.
        (ipcp_modif_dom_walker::before_dom_children): Likewise.
        (ipcp_update_bits): Likewise.
        (ipcp_update_vr): Likewise.
        * ipa-split.c (split_function): Convert to using ipa_param_adjustments.
        * ipa-sra.c: New file.
        * multiple_target.c (create_target_clone): Update to reflet new type
        of create_version_clone_with_body.
        * trans-mem.c (ipa_tm_create_version): Update to reflect new type of
        tree_function_versioning.
	(modify_function): Update to reflect new type of
	tree_function_versioning.
        * params.def (PARAM_IPA_SRA_MAX_REPLACEMENTS): New.
        * passes.def: Remove old IPA-SRA and add new one.
        * tree-pass.h (make_pass_early_ipa_sra): Remove declaration.
        (make_pass_ipa_sra): Declare.
	* dbgcnt.def: Remove eipa_sra.  Added ipa_sra_params and
	ipa_sra_retvalues.
	* doc/invoke.texi (ipa-sra-max-replacements): New.

        testsuite/
        * g++.dg/ipa/pr81248.C: Adjust dg-options and dump-scan.
        * gcc.dg/ipa/ipa-sra-1.c: Likewise.
        * gcc.dg/ipa/ipa-sra-10.c: Likewise.
        * gcc.dg/ipa/ipa-sra-11.c: Likewise.
        * gcc.dg/ipa/ipa-sra-3.c: Likewise.
        * gcc.dg/ipa/ipa-sra-4.c: Likewise.
        * gcc.dg/ipa/ipa-sra-5.c: Likewise.
        * gcc.dg/ipa/ipacost-2.c: Disable ipa-sra.
        * gcc.dg/ipa/ipcp-agg-9.c: Likewise.
        * gcc.dg/ipa/pr78121.c: Adjust scan pattern.
        * gcc.dg/ipa/vrp1.c: Likewise.
        * gcc.dg/ipa/vrp2.c: Likewise.
        * gcc.dg/ipa/vrp3.c: Likewise.
        * gcc.dg/ipa/vrp7.c: Likewise.
        * gcc.dg/ipa/vrp8.c: Likewise.
        * gcc.dg/noreorder.c: use noipa attribute instead of noinline.
        * gcc.dg/ipa/20040703-wpa.c: New test.
	* gcc.dg/ipa/ipa-sra-12.c: New test.
	* gcc.dg/ipa/ipa-sra-13.c: Likewise.
	* gcc.dg/ipa/ipa-sra-14.c: Likewise.
	* gcc.dg/ipa/ipa-sra-15.c: Likewise.
	* gcc.dg/ipa/ipa-sra-16.c: Likewise.
	* gcc.dg/ipa/ipa-sra-17.c: Likewise.
	* gcc.dg/ipa/ipa-sra-18.c: Likewise.
	* gcc.dg/ipa/ipa-sra-19.c: Likewise.
	* gcc.dg/ipa/ipa-sra-20.c: Likewise.
	* gcc.dg/ipa/ipa-sra-21.c: Likewise.
	* gcc.dg/ipa/ipa-sra-22.c: Likewise.
	* gcc.dg/sso/ipa-sra-1.c: Likewise.
	* g++.dg/ipa/ipa-sra-2.C: Likewise.
	* g++.dg/ipa/ipa-sra-3.C: Likewise.
	* gcc.dg/tree-ssa/ipa-cp-1.c: Make return value used.
	* g++.dg/ipa/devirt-19.C: Add missing return, add -fipa-cp-clone
	option.
	* g++.dg/lto/devirt-19_0.C: Add -fipa-cp-clone option.
	* gcc.dg/ipa/ipa-sra-2.c: Removed.
	* gcc.dg/ipa/ipa-sra-6.c: Likewise.

From-SVN: r275982

1 files changed, 382 insertions, 67 deletions

diff --git a/gcc/ipa-param-manipulation.h b/gcc/ipa-param-manipulation.h
index 71fc4a2..34477da 100644
--- a/gcc/ipa-param-manipulation.h
+++ b/gcc/ipa-param-manipulation.h
@@ -16,101 +16,416 @@ for more details.
 
 You should have received a copy of the GNU General Public License
 along with GCC; see the file COPYING3.  If not see
-<http://www.gnu.org/licenses/>.  */
+<http://www.gnu.org/licenses/>.
+
+
+
+This file defines classes and other data structures that are used to manipulate
+the prototype of a function, especially to create, remove or split its formal
+parameters, but also to remove its return value, and also its call statements
+correspondingly.
+
+The most basic one is a vector of structures ipa_adjusted_param.  It is simply
+a description how the new parameters should look like after the transformation
+in what way they relate to the previous ones (if in any).  Such relation to an
+old parameter can be an outright copy or an IPA-SRA replacement. If an old
+parameter is not listed or otherwise mentioned, it is removed as unused or at
+least unnecessary.  Note that this most basic structure does not work for
+modifying calls of functions with variable number of arguments.
+
+Class ipa_param_adjustments is only a little more than a thin encapsulation of
+a vector of ipa_param_adjustments.  Along with this vector it contains an index
+of the first potential vararg argument and a boolean flag whether the return
+value should be removed or not.  Moreover, the class contains method
+modify_call which can transform a call statement so that it correctly calls a
+modified function.  These two data structures were designed to have a small
+memory footprint because they are allocated for each clone of a call graph node
+that has its prototype changed and live until the end of IPA clone
+materialization and call redirection phase.
+
+On the other hand, class ipa_param_body_adjustments can afford to allocate more
+data because its life span is much smaller, it is allocated and destroyed in
+the course of materialization of each single clone that needs it or only when a
+particular pass needs to change a function it is operating on.  This class has
+various methods required to change function declaration and the body of the
+function according to instructions given either by class ipa_param_adjustments
+or only a vector of ipa_adjusted_params.
+
+When these classes are used in the context of call graph clone materialization
+and subsequent call statement redirection - which is the point at which we
+modify arguments in call statements - they need to cooperate with each other in
+order to handle what we refer to as transitive (IPA-SRA) splits.  These are
+situations when a formal parameter of one function is split into several
+smaller ones and some of them are then passed on in a call to another function
+because the formal parameter of this callee has also been split.
+
+Consider a simple example:
+
+struct S {int a, b, c;};
+struct Z {int x; S s;};
+
+foo (S s)
+{
+  use (s.b);
+}
+
+bar (Z z)
+{
+  use (z.s.a);
+  foo (z.s);
+}
+
+baz ()
+{
+  bar (*global);
+}
+
+Both bar and foo would have their parameter split.  Foo would receive one
+replacement representing s.b.  Function bar would see its parameter split into
+one replacement representing z.s.a and another representing z.s.b which would
+be passed on to foo.  It would be a so called transitive split IPA-SRA
+replacement, one which is passed in a call as an actual argument to another
+IPA-SRA replacement in another function.
+
+Note that the call chain the example can be arbitrarily long and recursive and
+that any function in it can be cloned by another IPA pass and any number of
+adjacent functions in the call chain can be inlined into each other.  Call
+redirection takes place only after bodies of the function have been modified by
+all of the above.
+
+Call redirection has to be able to find the right decl or SSA_NAME that
+corresponds to the transitive split in the caller.  The SSA names are assigned
+right after clone materialization/ modification and cannot be "added"
+afterwards.  Moreover, if the caller has been inlined the SSA_NAMEs in question
+no longer belong to PARM_DECLs but to VAR_DECLs, indistinguishable from any
+others.
+
+Therefore, when clone materialization finds a call statement which it knows is
+a part of a transitive split, it will modify it into:
+
+  foo (DUMMY_Z_VAR.s, repl_for_a, repl_for_b, <rest of original arguments>);
+
+It will also store {DUMMY_S_VAR, 32} and {DUMMY_S_VAR, 64} representing offsets
+of z.s.a and z.s.b (assuming a 32-bit int) into foo's cgraph node
+clone->performed_splits vector (which is storing structures of type
+ipa_param_performed_split also defined in this header file).
+
+Call redirection will identify that expression DUMMY_Z_VAR.s is based on a
+variable stored in performed_splits vector and learn that the following
+arguments, already in SSA form, represent offsets 32 and 64 in a split original
+parameter.  It subtracts offset of DUMMY_Z_VAR.s from 32 and 64 and arrives at
+offsets 0 and 32 within callee's original parameter.  At this point it also
+knows from the call graph that only the bit with offset 32 is needed and so
+changes the call statement into final:
+
+bar (repl_for_b, <rest of original arguments>);  */
 
 #ifndef IPA_PARAM_MANIPULATION_H
 #define IPA_PARAM_MANIPULATION_H
 
+/* Indices into ipa_param_prefixes to identify a human-readable prefix for newly
+   synthesized parameters.  Keep in sync with the array.  */
+enum ipa_param_name_prefix_indices
+  {
+   IPA_PARAM_PREFIX_SYNTH,
+   IPA_PARAM_PREFIX_ISRA,
+   IPA_PARAM_PREFIX_SIMD,
+   IPA_PARAM_PREFIX_MASK,
+   IPA_PARAM_PREFIX_COUNT
+};
+
+/* We do not support manipulating functions with more than
+   1<<IPA_PARAM_MAX_INDEX_BITS parameters.  */
+#define IPA_PARAM_MAX_INDEX_BITS 16
+
 /* Operation to be performed for the parameter in ipa_parm_adjustment
    below.  */
-enum ipa_parm_op {
-  IPA_PARM_OP_NONE,
-
-  /* This describes a brand new parameter.
 
-     The field `type' should be set to the new type, `arg_prefix'
-     should be set to the string prefix for the new DECL_NAME, and
-     `new_decl' will ultimately hold the newly created argument.  */
-  IPA_PARM_OP_NEW,
+enum ipa_parm_op
+{
+  /* Do not use or you will trigger an assert.  */
+  IPA_PARAM_OP_UNDEFINED,
 
   /* This new parameter is an unmodified parameter at index base_index. */
-  IPA_PARM_OP_COPY,
+  IPA_PARAM_OP_COPY,
+
+  /* This describes a brand new parameter.  If it somehow relates to any
+     original parameters, the user needs to manage the transition itself.  */
+  IPA_PARAM_OP_NEW,
 
-  /* This adjustment describes a parameter that is about to be removed
-     completely.  Most users will probably need to book keep those so that they
-     don't leave behinfd any non default def ssa names belonging to them.  */
-  IPA_PARM_OP_REMOVE
+    /* Split parameter as indicated by fields base_index, offset and type.  */
+  IPA_PARAM_OP_SPLIT
 };
 
-/* Structure to describe transformations of formal parameters and actual
-   arguments.  Each instance describes one new parameter and they are meant to
-   be stored in a vector.  Additionally, most users will probably want to store
-   adjustments about parameters that are being removed altogether so that SSA
-   names belonging to them can be replaced by SSA names of an artificial
-   variable.  */
-struct ipa_parm_adjustment
-{
-  /* The original PARM_DECL itself, helpful for processing of the body of the
-     function itself.  Intended for traversing function bodies.
-     ipa_modify_formal_parameters, ipa_modify_call_arguments and
-     ipa_combine_adjustments ignore this and use base_index.
-     ipa_modify_formal_parameters actually sets this.  */
-  tree base;
+/* Structure that describes one parameter of a function after transformation.
+   Omitted parameters will be removed.  */
 
-  /* Type of the new parameter.  However, if by_ref is true, the real type will
-     be a pointer to this type.  */
+struct GTY(()) ipa_adjusted_param
+{
+  /* Type of the new parameter.  Required for all operations except
+     IPA_PARM_OP_COPY when the original type will be preserved.  */
   tree type;
 
-  /* Alias refrerence type to be used in MEM_REFs when adjusting caller
-     arguments.  */
+  /* Alias reference type to be used in MEM_REFs when adjusting caller
+     arguments.  Required for IPA_PARM_OP_SPLIT operation.  */
   tree alias_ptr_type;
 
-  /* The new declaration when creating/replacing a parameter.  Created
-     by ipa_modify_formal_parameters, useful for functions modifying
-     the body accordingly.  For brand new arguments, this is the newly
-     created argument.  */
-  tree new_decl;
+  /* Offset into the original parameter (for the cases when the new parameter
+     is a component of an original one).  Required for IPA_PARM_OP_SPLIT
+     operation.  */
+  unsigned unit_offset;
 
-  /* New declaration of a substitute variable that we may use to replace all
-     non-default-def ssa names when a parm decl is going away.  */
-  tree new_ssa_base;
+  /* Zero based index of the original parameter this one is based on.  Required
+     for IPA_PARAM_OP_COPY and IPA_PARAM_OP_SPLIT, users of IPA_PARAM_OP_NEW
+     only need to specify it if they use replacement lookup provided by
+     ipa_param_body_adjustments.  */
+  unsigned base_index : IPA_PARAM_MAX_INDEX_BITS;
 
-  /* This holds the prefix to be used for the new DECL_NAME.  */
-  const char *arg_prefix;
+  /* Zero based index of the parameter this one is based on in the previous
+     clone.  If there is no previous clone, it must be equal to base_index.  */
+  unsigned prev_clone_index : IPA_PARAM_MAX_INDEX_BITS;
 
-  /* Offset into the original parameter (for the cases when the new parameter
-     is a component of an original one).  */
-  poly_int64_pod offset;
+  /* Specify the operation, if any, to be performed on the parameter.  */
+  enum ipa_parm_op op : 2;
 
-  /* Zero based index of the original parameter this one is based on.  */
-  int base_index;
+  /* If set, this structure describes a parameter copied over from a previous
+     IPA clone, any transformations are thus not to be re-done.  */
+  unsigned prev_clone_adjustment : 1;
 
-  /* Whether this parameter is a new parameter, a copy of an old one,
-     or one about to be removed.  */
-  enum ipa_parm_op op;
+  /* Index into ipa_param_prefixes specifying a prefix to be used with
+     DECL_NAMEs of newly synthesized parameters.  */
+  unsigned param_prefix_index : 2;
 
   /* Storage order of the original parameter (for the cases when the new
      parameter is a component of an original one).  */
   unsigned reverse : 1;
 
-  /* The parameter is to be passed by reference.  */
-  unsigned by_ref : 1;
+  /* A bit free for the user.  */
+  unsigned user_flag : 1;
+};
+
+void ipa_dump_adjusted_parameters (FILE *f,
+				   vec<ipa_adjusted_param, va_gc> *adj_params);
+
+/* Structure to remember the split performed on a node so that edge redirection
+   (i.e. splitting arguments of call statements) know how split formal
+   parameters of the caller are represented.  */
+
+struct GTY(()) ipa_param_performed_split
+{
+  /* The dummy VAR_DECL that was created instead of the split parameter that
+     sits in the call in the meantime between clone materialization and call
+     redirection.  All entries in a vector of performed splits that correspond
+     to the same dumy decl must be grouped together.  */
+  tree dummy_decl;
+  /* Offset into the original parameter.  */
+  unsigned unit_offset;
+};
+
+/* Class used to record planned modifications to parameters of a function and
+   also to perform necessary modifications at the caller side at the gimple
+   level.  Used to describe all cgraph node clones that have their parameters
+   changed, therefore the class should only have a small memory footprint.  */
+
+class GTY(()) ipa_param_adjustments
+{
+public:
+  /* Constructor from NEW_PARAMS showing how new parameters should look like
+      plus copying any pre-existing actual arguments starting from argument
+      with index ALWAYS_COPY_START (if non-negative, negative means do not copy
+      anything beyond what is described in NEW_PARAMS), and SKIP_RETURN, which
+      indicates that the function should return void after transformation.  */
+
+  ipa_param_adjustments (vec<ipa_adjusted_param, va_gc> *new_params,
+			 int always_copy_start, bool skip_return)
+    : m_adj_params (new_params), m_always_copy_start (always_copy_start),
+    m_skip_return (skip_return)
+    {}
+
+  /* Modify a call statement arguments (and possibly remove the return value)
+     as described in the data fields of this class.  */
+  gcall *modify_call (gcall *stmt,
+		      vec<ipa_param_performed_split, va_gc> *performed_splits,
+		      tree callee_decl, bool update_references);
+  /* Return if the first parameter is left intact.  */
+  bool first_param_intact_p ();
+  /* Build a function type corresponding to the modified call.  */
+  tree build_new_function_type (tree old_type, bool type_is_original_p);
+  /* Build a declaration corresponding to the target of the modified call.  */
+  tree adjust_decl (tree orig_decl);
+  /* Fill a vector marking which parameters are intact by the described
+     modifications. */
+  void get_surviving_params (vec<bool> *surviving_params);
+  /* Fill a vector with new indices of surviving original parameters.  */
+  void get_updated_indices (vec<int> *new_indices);
+
+  void dump (FILE *f);
+  void debug ();
+
+  /* How the known part of arguments should look like.  */
+  vec<ipa_adjusted_param, va_gc> *m_adj_params;
+
+  /* If non-negative, copy any arguments starting at this offset without any
+     modifications so that functions with variable number of arguments can be
+     modified. This number should be equal to the number of original forma
+     parameters.  */
+  int m_always_copy_start;
+  /* If true, make the function not return any value.  */
+  bool m_skip_return;
+
+private:
+  ipa_param_adjustments () {}
+
+  void init (vec<tree> *cur_params);
+  int get_max_base_index ();
+  bool method2func_p (tree orig_type);
+};
+
+/* Structure used to map expressions accessing split or replaced parameters to
+   new PARM_DECLs.  */
+
+struct ipa_param_body_replacement
+{
+  /* The old decl of the original parameter.   */
+  tree base;
+  /* The new decl it should be replaced with.  */
+  tree repl;
+  /* When modifying clones during IPA clone materialization, this is a dummy
+     decl used to mark calls in which we need to apply transitive splitting,
+     these dummy delcls are inserted as arguments to such calls and then
+     followed by all the replacements with offset info stored in
+     ipa_param_performed_split.
+
+     Users of ipa_param_body_adjustments that modify standalone functions
+     outside of IPA clone materialization can use this field for their internal
+     purposes.  */
+  tree dummy;
+  /* The offset within BASE that REPL represents.  */
+  unsigned unit_offset;
+};
+
+struct ipa_replace_map;
+
+/* Class used when actually performing adjustments to formal parameters of a
+   function to map accesses that need to be replaced to replacements.  The
+   class attempts to work in two very different sets of circumstances: as a
+   part of tree-inine.c's tree_function_versioning machinery to clone functions
+   (when M_ID is not NULL) and in s standalone fashion, modifying an existing
+   function in place (when M_ID is NULL).  While a lot of stuff handled in a
+   unified way in both modes, there are many aspects of the processs that
+   requires distinct paths.  */
+
+class ipa_param_body_adjustments
+{
+public:
+  /* Constructor to use from within tree-inline.  */
+  ipa_param_body_adjustments (ipa_param_adjustments *adjustments,
+			      tree fndecl, tree old_fndecl,
+			      struct copy_body_data *id, tree *vars,
+			      vec<ipa_replace_map *, va_gc> *tree_map);
+  /* Constructor to use for modifying a function outside of tree-inline from an
+     instance of ipa_param_adjustments.  */
+  ipa_param_body_adjustments (ipa_param_adjustments *adjustments,
+			      tree fndecl);
+  /* Constructor to use for modifying a function outside of tree-inline from a
+     simple vector of desired parameter modification.  */
+  ipa_param_body_adjustments (vec<ipa_adjusted_param, va_gc> *adj_params,
+			      tree fndecl);
+
+  /* The do-it-all function for modifying a function outside of
+     tree-inline.  */
+  bool perform_cfun_body_modifications ();
+
+  /* Change the PARM_DECLs.  */
+  void modify_formal_parameters ();
+  /* Register a replacement decl for the transformation done in APM.  */
+  void register_replacement (ipa_adjusted_param *apm, tree replacement,
+			     tree dummy = NULL_TREE);
+  /* Lookup a replacement for a given offset within a given parameter.  */
+  tree lookup_replacement (tree base, unsigned unit_offset);
+  /* Lookup a replacement for an expression, if there is one.  */
+  ipa_param_body_replacement *get_expr_replacement (tree expr,
+						    bool ignore_default_def);
+  /* Lookup the new base for surviving names previously belonging to a
+     parameter. */
+  tree get_replacement_ssa_base (tree old_decl);
+  /* Modify a statement.  */
+  bool modify_gimple_stmt (gimple **stmt, gimple_seq *extra_stmts);
+  /* Return the new chain of parameters.  */
+  tree get_new_param_chain ();
+
+  /* Pointers to data structures defining how the function should be
+     modified.  */
+  vec<ipa_adjusted_param, va_gc> *m_adj_params;
+  ipa_param_adjustments *m_adjustments;
+
+  /* Vector of old parameter declarations that must have their debug bind
+     statements re-mapped and debug decls created.  */
+
+  auto_vec<tree, 16> m_reset_debug_decls;
+
+  /* Set to true if there are any IPA_PARAM_OP_SPLIT adjustments among stored
+     adjustments.  */
+  bool m_split_modifications_p;
+private:
+  void common_initialization (tree old_fndecl, tree *vars,
+			      vec<ipa_replace_map *, va_gc> *tree_map);
+  unsigned get_base_index (ipa_adjusted_param *apm);
+  ipa_param_body_replacement *lookup_replacement_1 (tree base,
+						    unsigned unit_offset);
+  tree replace_removed_params_ssa_names (tree old_name, gimple *stmt);
+  bool modify_expression (tree *expr_p, bool convert);
+  bool modify_assignment (gimple *stmt, gimple_seq *extra_stmts);
+  bool modify_call_stmt (gcall **stmt_p);
+  bool modify_cfun_body ();
+  void reset_debug_stmts ();
+
+  /* Declaration of the function that is being transformed.  */
+
+  tree m_fndecl;
+
+  /* If non-NULL, the tree-inline master data structure guiding materialization
+     of the current clone.  */
+  struct copy_body_data *m_id;
+
+  /* Vector of old parameter declarations (before changing them).  */
+
+  auto_vec<tree, 16> m_oparms;
+
+  /* Vector of parameter declarations the function will have after
+     transformation.  */
+
+  auto_vec<tree, 16> m_new_decls;
+
+  /* If the function type has non-NULL TYPE_ARG_TYPES, this is the vector of
+     these types after transformation, otherwise an empty one.  */
+
+  auto_vec<tree, 16> m_new_types;
+
+  /* Vector of structures telling how to replace old parameters in in the
+     function body.  TODO: Even though there usually be only few, but should we
+     use a hash?  */
+
+  auto_vec<ipa_param_body_replacement, 16> m_replacements;
+
+  /* Vector for remapping SSA_BASES from old parameter declarations that are
+     being removed as a part of the transformation.  Before a new VAR_DECL is
+     created, it holds the old PARM_DECL, once the variable is built it is
+     stored here.  */
+
+  auto_vec<tree> m_removed_decls;
+
+  /* Hash to quickly lookup the item in m_removed_decls given the old decl.  */
+
+  hash_map<tree, unsigned> m_removed_map;
+
+  /* True iff the transformed function is a class method that is about to loose
+     its this pointer and must be converted to a normal function.  */
+
+  bool m_method2func;
 };
 
-typedef vec<ipa_parm_adjustment> ipa_parm_adjustment_vec;
-
-vec<tree> ipa_get_vector_of_formal_parms (tree fndecl);
-vec<tree> ipa_get_vector_of_formal_parm_types (tree fntype);
-void ipa_modify_formal_parameters (tree fndecl, ipa_parm_adjustment_vec);
-void ipa_modify_call_arguments (struct cgraph_edge *, gcall *,
-				ipa_parm_adjustment_vec);
-ipa_parm_adjustment_vec ipa_combine_adjustments (ipa_parm_adjustment_vec,
-						 ipa_parm_adjustment_vec);
-void ipa_dump_param_adjustments (FILE *, ipa_parm_adjustment_vec, tree);
-
-bool ipa_modify_expr (tree *, bool, ipa_parm_adjustment_vec);
-ipa_parm_adjustment *ipa_get_adjustment_candidate (tree **, bool *,
-						   ipa_parm_adjustment_vec,
-						   bool);
+void push_function_arg_decls (vec<tree> *args, tree fndecl);
+void push_function_arg_types (vec<tree> *types, tree fntype);
 
 #endif	/* IPA_PARAM_MANIPULATION_H */