| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This addresses pr97929. The case for WIDEN_PLUS and WIDEN_MINUS were
missing in vect_get_smallest_scalar_type.
gcc/ChangeLog:
PR tree-optimization/97929
* tree-vect-data-refs.c (vect_get_smallest_scalar_type): Add
WIDEN_PLUS/WIDEN_MINUS case.
gcc/testsuite/ChangeLog:
* gcc.dg/vect/pr97929.c: New test.
|
|
Currently we have three vector cost models: cheap, dynamic and
unlimited. -O2 -ftree-vectorize uses “cheap” by default, but that's
still relatively aggressive about peeling and aliasing checks,
and can lead to significant code size growth.
This patch adds an even more conservative choice, which for lack of
imagination I've called “very cheap”. It only allows vectorisation
if the vector code entirely replaces the scalar code. It also
requires one iteration of the vector loop to pay for itself,
regardless of how often the loop iterates. (If the vector loop
needs multiple iterations to be beneficial then things are
probably too close to call, and the conservative thing would
be to stick with the scalar code.)
The idea is that this should be suitable for -O2, although the patch
doesn't change any defaults itself.
I tested this by building and running a bunch of workloads for SVE,
with three options:
(1) -O2
(2) -O2 -ftree-vectorize -fvect-cost-model=very-cheap
(3) -O2 -ftree-vectorize [-fvect-cost-model=cheap]
All three builds used the default -msve-vector-bits=scalable and
ran with the minimum vector length of 128 bits, which should give
a worst-case bound for the performance impact.
The workloads included a mixture of microbenchmarks and full
applications. Because it's quite an eclectic mix, there's not
much point giving exact figures. The aim was more to get a general
impression.
Code size growth with (2) was much lower than with (3). Only a
handful of tests increased by more than 5%, and all of them were
microbenchmarks.
In terms of performance, (2) was significantly faster than (1)
on microbenchmarks (as expected) but also on some full apps.
Again, performance only regressed on a handful of tests.
As expected, the performance of (3) vs. (1) and (3) vs. (2) is more
of a mixed bag. There are several significant improvements with (3)
over (2), but also some (smaller) regressions. That seems to be in
line with -O2 -ftree-vectorize being a kind of -O2.5.
The patch reorders vect_cost_model so that values are in order
of increasing aggressiveness, which makes it possible to use
range checks. The value 0 still represents “unlimited”,
so “if (flag_vect_cost_model)” is still a meaningful check.
gcc/
* doc/invoke.texi (-fvect-cost-model): Add a very-cheap model.
* common.opt (fvect-cost-model=): Add very-cheap as a possible option.
(fsimd-cost-model=): Likewise.
(vect_cost_model): Add very-cheap.
* flag-types.h (vect_cost_model): Add VECT_COST_MODEL_VERY_CHEAP.
Put the values in order of increasing aggressiveness.
* tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Use
range checks when comparing against VECT_COST_MODEL_CHEAP.
(vect_prune_runtime_alias_test_list): Do not allow any alias
checks for the very-cheap cost model.
* tree-vect-loop.c (vect_analyze_loop_costing): Do not allow
any peeling for the very-cheap cost model. Also require one
iteration of the vector loop to pay for itself.
gcc/testsuite/
* gcc.dg/vect/vect-cost-model-1.c: New test.
* gcc.dg/vect/vect-cost-model-2.c: Likewise.
* gcc.dg/vect/vect-cost-model-3.c: Likewise.
* gcc.dg/vect/vect-cost-model-4.c: Likewise.
* gcc.dg/vect/vect-cost-model-5.c: Likewise.
* gcc.dg/vect/vect-cost-model-6.c: Likewise.
|
|
The following removes an assert that can not easily be adjusted to
cover the additional cases we now handle after the removal of
the same-align DRs vector.
2020-11-10 Richard Biener <rguenther@suse.de>
PR tree-optimization/97769
* tree-vect-data-refs.c (vect_update_misalignment_for_peel):
Remove assert.
* gcc.dg/vect/pr97769.c: New testcase.
|
|
The following fixes SLP vectorization of stores that were
pattern recognized. Since in SLP vectorization pattern analysis
happens after dataref group analysis we have to adjust the groups
with the pattern stmts. This has some effects down the pipeline
and exposes cases where we looked at the wrong pattern/non-pattern
stmts.
2020-11-05 Richard Biener <rguenther@suse.de>
* tree-vect-data-refs.c (vect_slp_analyze_node_dependences):
Use the original stmts.
(vect_slp_analyze_node_alignment): Use the pattern stmt.
* tree-vect-slp.c (vect_fixup_store_groups_with_patterns):
New function.
(vect_slp_analyze_bb_1): Call it.
* gcc.dg/vect/bb-slp-69.c: New testcase.
|
|
This adds dumping to vect_slp_analyze_node_alignment when it fails
an SLP instance due to shared vector type conflicts.
2020-10-28 Richard Biener <rguenther@suse.de>
* tree-vect-data-refs.c (vect_slp_analyze_node_alignment):
Dump when vect_update_shared_vectype fails.
|
|
This makes the only consumer of STMT_VINFO_SAME_ALIGN_REFS, the
loop peeling for alignment code, use locally computed data and
then removes STMT_VINFO_SAME_ALIGN_REFS and its computation.
It also adjusts the auto_vec<> move CTOR/assignment so you
can write
auto_vec<..> foo = bar.copy ();
and have foo own the generated copy.
2020-10-13 Richard Biener <rguenther@suse.de>
PR tree-optimization/97382
* tree-vectorizer.h (_stmt_vec_info::same_align_refs): Remove.
(STMT_VINFO_SAME_ALIGN_REFS): Likewise.
* tree-vectorizer.c (vec_info::new_stmt_vec_info): Do not
allocate STMT_VINFO_SAME_ALIGN_REFS.
(vec_info::free_stmt_vec_info): Do not release
STMT_VINFO_SAME_ALIGN_REFS.
* tree-vect-data-refs.c (vect_analyze_data_ref_dependences):
Do not compute self and read-read dependences.
(vect_dr_aligned_if_related_peeled_dr_is): New helper.
(vect_dr_aligned_if_peeled_dr_is): Likewise.
(vect_update_misalignment_for_peel): Use it instead of
iterating over STMT_VINFO_SAME_ALIGN_REFS.
(dr_align_group_sort_cmp): New function.
(vect_enhance_data_refs_alignment): Count the number of
same aligned refs here and elide uses of STMT_VINFO_SAME_ALIGN_REFS.
(vect_find_same_alignment_drs): Remove.
(vect_analyze_data_refs_alignment): Do not call it.
* vec.h (auto_vec<T, 0>::auto_vec): Adjust CTOR to take
a vec<>&&, assert it isn't using auto storage.
(auto_vec& operator=): Apply a similar change.
* gcc.dg/vect/no-vfa-vect-dv-2.c: Remove same align dump
scanning.
* gcc.dg/vect/vect-103.c: Likewise.
* gcc.dg/vect/vect-91.c: Likewise.
* gfortran.dg/vect/vect-4.f90: Likewise.
|
|
This introduces a permute optimization phase for SLP which is
intended to cover the existing permute eliding for SLP reductions
plus handling commonizing the easy cases.
It currently uses graphds to compute a postorder on the reverse
SLP graph and it handles all cases vect_attempt_slp_rearrange_stmts
did (hopefully - I've adjusted most testcases that triggered it
a few days ago). It restricts itself to move around bijective
permutations to simplify things for now, mainly around constant nodes.
As a prerequesite it makes the SLP graph cyclic (ugh). It looks
like it would pay off to compute a PRE/POST order visit array
once and elide all the recursive SLP graph walks and their
visited hash-set. At least for the time where we do not change
the SLP graph during such walk.
I do not like using graphds too much but at least I don't have to
re-implement yet another RPO walk, so maybe it isn't too bad.
It now computes permute placement during iteration and thus should
get cycles more obviously correct.
Richard.
2020-10-06 Richard Biener <rguenther@suse.de>
* tree-vect-data-refs.c (vect_slp_analyze_instance_dependence):
Use SLP_TREE_REPRESENTATIVE.
* tree-vectorizer.h (_slp_tree::vertex): New member used
for graphds interfacing.
* tree-vect-slp.c (vect_build_slp_tree_2): Allocate space
for PHI SLP children.
(vect_analyze_slp_backedges): New function filling in SLP
node children for PHIs that correspond to backedge values.
(vect_analyze_slp): Call vect_analyze_slp_backedges for the
graph.
(vect_slp_analyze_node_operations): Deal with a cyclic graph.
(vect_schedule_slp_instance): Likewise.
(vect_schedule_slp): Likewise.
(slp_copy_subtree): Remove.
(vect_slp_rearrange_stmts): Likewise.
(vect_attempt_slp_rearrange_stmts): Likewise.
(vect_slp_build_vertices): New functions.
(vect_slp_permute): Likewise.
(vect_slp_perms_eq): Likewise.
(vect_optimize_slp): Remove special code to elide
permutations with SLP reductions. Implement generic
permute optimization.
* gcc.dg/vect/bb-slp-50.c: New testcase.
* gcc.dg/vect/bb-slp-51.c: Likewise.
|
|
This fixes leaks discovered checking whether I introduced new ones
with the last vectorizer changes.
2020-10-09 Richard Biener <rguenther@suse.de>
* cgraphunit.c (expand_all_functions): Free tp_first_run_order.
* ipa-modref.c (pass_ipa_modref::execute): Free order.
* tree-ssa-loop-niter.c (estimate_numbers_of_iterations): Free
loop body.
* tree-vect-data-refs.c (vect_find_stmt_data_reference): Free
data references upon failure.
* tree-vect-loop.c (update_epilogue_loop_vinfo): Free BBs
array of the original loop.
* tree-vect-slp.c (vect_slp_bbs): Use an auto_vec for
dataref_groups to release its memory.
|
|
This removes STMT_VINFO_NUM_SLP_USES by pushing the setting of
the shared stmt_vec_info vector type to where we actually need it
which is alignment analysis and vectorizable_* analysis (where
we could eventually elide it for non-load/store operations).
In particular "uses" in the cache and in disqualified SLP
subgraphs should no longer provide conflicting vector types
this way.
2020-09-16 Richard Biener <rguenther@suse.de>
* tree-vectorizer.h (_stmt_vec_info::num_slp_uses): Remove.
(STMT_VINFO_NUM_SLP_USES): Likewise.
(vect_free_slp_instance): Adjust.
(vect_update_shared_vectype): Declare.
* tree-vectorizer.c (vec_info::~vec_info): Adjust.
* tree-vect-loop.c (vect_analyze_loop_2): Likewise.
(vectorizable_live_operation): Use vector type from
SLP_TREE_REPRESENTATIVE.
(vect_transform_loop): Adjust.
* tree-vect-data-refs.c (vect_slp_analyze_node_alignment):
Set the shared vector type.
* tree-vect-slp.c (vect_free_slp_tree): Remove final_p
parameter, remove STMT_VINFO_NUM_SLP_USES updating.
(vect_free_slp_instance): Adjust.
(vect_create_new_slp_node): Remove STMT_VINFO_NUM_SLP_USES
updating.
(vect_update_shared_vectype): Always compare with the
present vector type, update if NULL.
(vect_build_slp_tree_1): Do not update the shared vector
type here.
(vect_build_slp_tree_2): Adjust.
(slp_copy_subtree): Likewise.
(vect_attempt_slp_rearrange_stmts): Likewise.
(vect_analyze_slp_instance): Likewise.
(vect_analyze_slp): Likewise.
(vect_slp_analyze_node_operations_1): Update the shared
vector type.
(vect_slp_analyze_operations): Adjust.
(vect_slp_analyze_bb_1): Likewise.
|
|
gcc/ChangeLog:
* tree-vect-data-refs.c (dr_group_sort_cmp): Work on
data_ref_pair.
(vect_analyze_data_ref_accesses): Work on groups.
(vect_find_stmt_data_reference): Add group_id argument and fill
up dataref_groups vector.
* tree-vect-loop.c (vect_get_datarefs_in_loop): Pass new
arguments.
(vect_analyze_loop_2): Likewise.
* tree-vect-slp.c (vect_slp_analyze_bb_1): Pass argument.
(vect_slp_bb_region): Likewise.
(vect_slp_region): Likewise.
(vect_slp_bb):Work on the entire BB.
* tree-vectorizer.h (vect_analyze_data_ref_accesses): Add new
argument.
(vect_find_stmt_data_reference): Likewise.
gcc/testsuite/ChangeLog:
* gcc.dg/vect/bb-slp-38.c: Adjust pattern as now we only process
a single vectorization and now 2 partial.
* gcc.dg/vect/bb-slp-45.c: New test.
|
|
This is a follow up to commit 5c9669a0e6c respectively discussion
https://gcc.gnu.org/pipermail/gcc-patches/2020-June/549132.html
In case a type has a lower alignment than its size, ensure that we
advance. For example, on s390x we have for a long double an alignment
constraint of 8 bytes whereas the size is 16 bytes. Increasing the loop
variable only by TARGET_ALIGN / DR_SIZE which equals zero in case of a
long double results in an infinite loop. This is fixed by ensuring that
we advance at least by one.
gcc/ChangeLog:
* tree-vect-data-refs.c (vect_enhance_data_refs_alignment):
Ensure that loop variable npeel_tmp advances in each iteration.
|
|
This followup removes vect_verify_datarefs_alignment and its
premature cancellation of vectorization leaving the actual
decision whether alignment is supported to the functions
deciding whether we can vectorize a load or store.
2020-07-08 Richard Biener <rguenther@suse.de>
* tree-vectorizer.h (vect_verify_datarefs_alignment): Remove.
(vect_slp_analyze_and_verify_instance_alignment): Rename to ...
(vect_slp_analyze_instance_alignment): ... this.
* tree-vect-data-refs.c (verify_data_ref_alignment): Remove.
(vect_verify_datarefs_alignment): Likewise.
(vect_enhance_data_refs_alignment): Do not call
vect_verify_datarefs_alignment.
(vect_slp_analyze_node_alignment): Rename from
vect_slp_analyze_and_verify_node_alignment and do not
call verify_data_ref_alignment.
(vect_slp_analyze_instance_alignment): Rename from
vect_slp_analyze_and_verify_instance_alignment.
* tree-vect-stmts.c (vectorizable_store): Dump when
we vectorize an unaligned access.
(vectorizable_load): Likewise.
* tree-vect-loop.c (vect_analyze_loop_2): Do not call
vect_verify_datarefs_alignment.
* tree-vect-slp.c (vect_slp_analyze_bb_1): Adjust.
* gcc.dg/vect/bb-slp-10.c: Adjust.
* gcc.dg/vect/slp-45.c: Likewise.
* gcc.dg/vect/vect-109.c: Likewise.
|
|
This fixes a condition that caused all negative step DR groups to
be detected as single element interleaving. Such groups are
rejected by interleaving vectorization but miscompiled by SLP
which is fixed by forcing VMAT_STRIDED_SLP for now.
2020-07-07 Richard Biener <rguenther@suse.de>
* tree-vect-data-refs.c (vect_analyze_data_ref_accesses): Fix
group overlap condition to allow negative step DR groups.
* tree-vect-stmts.c (get_group_load_store_type): For
multi element SLP groups force VMAT_STRIDED_SLP when the step
is negative.
* gcc.dg/vect/slp-47.c: New testcase.
* gcc.dg/vect/slp-48.c: Likewise.
|
|
This fixes bogus misalignment calculation for negative steps
since an assertion a previous comment indicated no longer holds:
/* DR_STEP(dr) is the same as -TYPE_SIZE of the scalar type,
otherwise we wouldn't be here. */
Thus the following replaces DR_STEP by -TYPE_SIZE.
2020-07-06 Richard Biener <rguenther@suse.de>
PR tree-optimization/96075
* tree-vect-data-refs.c (vect_compute_data_ref_alignment): Use
TYPE_SIZE_UNIT of the vector component type instead of DR_STEP
for the misalignment calculation for negative step.
* gcc.dg/vect/slp-46.c: New testcase.
|
|
In the test case for PR95961, vectorization factor computed
by vect_determine_vectorization_factor is [8,8]. But this is
updated to [1,1] later by vect_update_vf_for_slp. When we call
vect_get_num_vectors in vect_enhance_data_refs_alignment, the number
of scalars which is based on the vectorization factor is not a multiple
of the the number of elements in the vector type. This leads to
the ICE. This isn't a simple stream of contiguous vector accesses.
It's hard to predict from the available information how many vector
accesses we'll actually need per iteration. As discussed, here we
should use the number of scalars instead of the number of vectors as
an upper bound for the loop saving info about DR in the hash table.
2020-07-02 Felix Yang <felix.yang@huawei.com>
gcc/
PR tree-optimization/95961
* tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Use the
number of scalars instead of the number of vectors as an upper bound
for the loop saving info about DR in the hash table. Remove unused
local variables.
gcc/testsuite/
PR tree-optimization/95961
* gcc.target/aarch64/sve/pr95961.c: New test.
|
|
This makes sure to emit SLP vectorized loads where the first scalar
load is. This makes SLP dependence checking more powerful because
hoisting loads can use TBAA and it increases the freedom for
vector placement when there are constraints from live lanes.
Vectorized shifts block inserting vectorized stmts always after
vectorized defs because it ends up using the original scalar
operand even when the SLP graph indicates the shift operand
is vectorized (and we actually emit and cost those stmts).
vect_slp_analyze_and_verify_node_alignment shows we need alignment
for too many places, this is a temporary solution and my plan
is to have a single meta-info for a dataref group instead
(also getting rid of DR_GROUP_FIRST/NEXT_ELEMENT).
2020-06-24 Richard Biener <rguenther@suse.de>
* tree-vectorizer.h (vect_find_first_scalar_stmt_in_slp):
Declare.
* tree-vect-data-refs.c (vect_preserves_scalar_order_p):
Simplify for new position of vectorized SLP loads.
(vect_slp_analyze_node_dependences): Adjust for it.
(vect_slp_analyze_and_verify_node_alignment): Compute alignment
for the first stmts dataref.
* tree-vect-slp.c (vect_find_first_scalar_stmt_in_slp): New.
(vect_schedule_slp_instance): Emit loads before the
first scalar stmt.
* tree-vect-stmts.c (vectorizable_load): Do what the comment
says and use vect_find_first_scalar_stmt_in_slp.
|
|
Minor code refactorings in tree-vect-data-refs.c. When trying loop peeling
to force alignment, call vect_relevant_for_alignment_p to filter out data
references in the loop whose alignment is irrelevant.
2020-06-18 Felix Yang <felix.yang@huawei.com>
gcc/
* tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Call
vect_relevant_for_alignment_p to filter out data references in
the loop whose alignment is irrelevant when trying loop peeling
to force alignment.
|
|
Minor code refactorings in tree-vect-data-refs.c and tree-vect-loop.c.
Use LOOP_VINFO_DATAREFS and LOOP_VINFO_DDRS when possible and rename
several parameters to make code more consistent.
2020-06-13 Felix Yang <felix.yang@huawei.com>
gcc/
* tree-vect-data-refs.c (vect_verify_datarefs_alignment): Rename
parameter to loop_vinfo and update uses. Use LOOP_VINFO_DATAREFS
when possible.
(vect_analyze_data_refs_alignment): Likewise, and use LOOP_VINFO_DDRS
when possible.
* tree-vect-loop.c (vect_dissolve_slp_only_groups): Use
LOOP_VINFO_DATAREFS when possible.
(update_epilogue_loop_vinfo): Likewise.
|
|
In the test case for PR95570, the only data reference in the loop is a
gather-statter access. Scalar evolution analysis for this data reference
failed, so DR_STEP is NULL_TREE. This leads to the segmentation fault.
We should filter out scatter-gather access in vect_enhance_data_refs_alignment.
2020-06-12 Felix Yang <felix.yang@huawei.com>
gcc/
PR tree-optimization/95570
* tree-vect-data-refs.c (vect_relevant_for_alignment_p): New function.
(vect_verify_datarefs_alignment): Call it to filter out data references
in the loop whose alignment is irrelevant.
(vect_get_peeling_costs_all_drs): Likewise.
(vect_peeling_supportable): Likewise.
(vect_enhance_data_refs_alignment): Likewise.
gcc/testsuite/
PR tree-optimization/95570
* gcc.dg/vect/pr95570.c: New test.
|
|
This makes {SLP_TREE,STMT_VINFO}_VEC_STMTS a vector of gimple * and
not allocate a stmt_vec_info for vectorizer generated stmts since
this is now possible after removing the only use which was chaining
of vector stmts via STMT_VINFO_RELATED_STMT.
This also removes all stmt_vec_info allocations done for vector
stmts, the remaining ones are for stmts in the scalar IL and for
patterns which are not part of the IL. Thus after this the stmt
UIDs inside a basic-block are suitable for dominance checking
if you ignore (or lazy-fill) UIDs of zero of the vector stmts
inserted during transform. This property is ensured by a new
flag set when pattern analysis is complete.
2020-06-10 Richard Biener <rguenther@suse.de>
* tree-vectorizer.h (_slp_tree::vec_stmts): Make it a vector
of gimple * stmts.
(_stmt_vec_info::vec_stmts): Likewise.
(vec_info::stmt_vec_info_ro): New flag.
(vect_finish_replace_stmt): Adjust declaration.
(vect_finish_stmt_generation): Likewise.
(vectorizable_induction): Likewise.
(vect_transform_reduction): Likewise.
(vectorizable_lc_phi): Likewise.
* tree-vect-data-refs.c (vect_create_data_ref_ptr): Do not
allocate stmt infos for increments.
(vect_record_grouped_load_vectors): Adjust.
* tree-vect-loop.c (vect_create_epilog_for_reduction): Likewise.
(vectorize_fold_left_reduction): Likewise.
(vect_transform_reduction): Likewise.
(vect_transform_cycle_phi): Likewise.
(vectorizable_lc_phi): Likewise.
(vectorizable_induction): Likewise.
(vectorizable_live_operation): Likewise.
(vect_transform_loop): Likewise.
* tree-vect-patterns.c (vect_pattern_recog): Set stmt_vec_info_ro.
* tree-vect-slp.c (vect_get_slp_vect_def): Adjust.
(vect_get_slp_defs): Likewise.
(vect_transform_slp_perm_load): Likewise.
(vect_schedule_slp_instance): Likewise.
(vectorize_slp_instance_root_stmt): Likewise.
* tree-vect-stmts.c (vect_get_vec_defs_for_operand): Likewise.
(vect_finish_stmt_generation_1): Do not allocate a stmt info.
(vect_finish_replace_stmt): Do not return anything.
(vect_finish_stmt_generation): Likewise.
(vect_build_gather_load_calls): Adjust.
(vectorizable_bswap): Likewise.
(vectorizable_call): Likewise.
(vectorizable_simd_clone_call): Likewise.
(vect_create_vectorized_demotion_stmts): Likewise.
(vectorizable_conversion): Likewise.
(vectorizable_assignment): Likewise.
(vectorizable_shift): Likewise.
(vectorizable_operation): Likewise.
(vectorizable_scan_store): Likewise.
(vectorizable_store): Likewise.
(vectorizable_load): Likewise.
(vectorizable_condition): Likewise.
(vectorizable_comparison): Likewise.
(vect_transform_stmt): Likewise.
* tree-vectorizer.c (vec_info::vec_info): Initialize
stmt_vec_info_ro.
(vec_info::replace_stmt): Copy over stmt UID rather than
unsetting/setting a stmt info allocating a new UID.
(vec_info::set_vinfo_for_stmt): Assert !stmt_vec_info_ro.
|
|
This gets rid of the linked list of STMT_VINFO_VECT_STMT and
STMT_VINFO_RELATED_STMT in preparation for vectorized stmts no
longer needing a stmt_vec_info (just for this chaining). This
has ripple-down effects in all places we gather vectorized
defs. For this new interfaces are introduced and used
throughout vectorization, simplifying code in a lot of places
and merging it with the SLP way of gathering vectorized
operands. There is vect_get_vec_defs as the new recommended
unified interface and vect_get_vec_defs_for_operand as one
for non-SLP operation. I've resorted to keep the structure
of the code the same where using vect_get_vec_defs would have
been too disruptive for this already large patch.
2020-06-10 Richard Biener <rguenther@suse.de>
* tree-vect-data-refs.c (vect_vfa_access_size): Adjust.
(vect_record_grouped_load_vectors): Likewise.
* tree-vect-loop.c (vect_create_epilog_for_reduction): Likewise.
(vectorize_fold_left_reduction): Likewise.
(vect_transform_reduction): Likewise.
(vect_transform_cycle_phi): Likewise.
(vectorizable_lc_phi): Likewise.
(vectorizable_induction): Likewise.
(vectorizable_live_operation): Likewise.
(vect_transform_loop): Likewise.
* tree-vect-slp.c (vect_get_slp_defs): New function, split out
from overload.
* tree-vect-stmts.c (vect_get_vec_def_for_operand_1): Remove.
(vect_get_vec_def_for_operand): Likewise.
(vect_get_vec_def_for_stmt_copy): Likewise.
(vect_get_vec_defs_for_stmt_copy): Likewise.
(vect_get_vec_defs_for_operand): New function.
(vect_get_vec_defs): Likewise.
(vect_build_gather_load_calls): Adjust.
(vect_get_gather_scatter_ops): Likewise.
(vectorizable_bswap): Likewise.
(vectorizable_call): Likewise.
(vectorizable_simd_clone_call): Likewise.
(vect_get_loop_based_defs): Remove.
(vect_create_vectorized_demotion_stmts): Adjust.
(vectorizable_conversion): Likewise.
(vectorizable_assignment): Likewise.
(vectorizable_shift): Likewise.
(vectorizable_operation): Likewise.
(vectorizable_scan_store): Likewise.
(vectorizable_store): Likewise.
(vectorizable_load): Likewise.
(vectorizable_condition): Likewise.
(vectorizable_comparison): Likewise.
(vect_transform_stmt): Adjust and remove no longer applicable
sanity checks.
* tree-vectorizer.c (vec_info::new_stmt_vec_info): Initialize
STMT_VINFO_VEC_STMTS.
(vec_info::free_stmt_vec_info): Relase it.
* tree-vectorizer.h (_stmt_vec_info::vectorized_stmt): Remove.
(_stmt_vec_info::vec_stmts): Add.
(STMT_VINFO_VEC_STMT): Remove.
(STMT_VINFO_VEC_STMTS): New.
(vect_get_vec_def_for_operand_1): Remove.
(vect_get_vec_def_for_operand): Likewise.
(vect_get_vec_defs_for_stmt_copy): Likewise.
(vect_get_vec_def_for_stmt_copy): Likewise.
(vect_get_vec_defs): New overloads.
(vect_get_vec_defs_for_operand): New.
(vect_get_slp_defs): Declare.
|
|
This fixes a disconnect between the stmt_info used for dr_info
analysis and the one in SLP_TREE_REPRESENTATIVE with a temporary
workaround.
2020-06-05 Richard Biener <rguenther@suse.de>
PR tree-optimization/95539
* tree-vect-data-refs.c
(vect_slp_analyze_and_verify_instance_alignment): Use
SLP_TREE_REPRESENTATIVE for the data-ref check.
* tree-vect-stmts.c (vectorizable_load): Reset stmt_info
back to the first scalar stmt rather than the
SLP_TREE_REPRESENTATIVE to match previous behavior.
* gcc.dg/vect/pr95539.c: New testcase.
|
|
A lot of code that wants to know the number of bits in a vector
element gets that information from the element's TYPE_SIZE,
which is always equal to TYPE_SIZE_UNIT * BITS_PER_UNIT.
This doesn't work for SVE and AVX512-style packed boolean vectors,
where several elements can occupy a single byte.
This patch introduces a new pair of helpers for getting the true
(possibly sub-byte) size. I made a token attempt to convert obvious
element size calculations, but I'm sure I missed some.
2020-05-12 Richard Sandiford <richard.sandiford@arm.com>
gcc/
PR tree-optimization/94980
* tree.h (vector_element_bits, vector_element_bits_tree): Declare.
* tree.c (vector_element_bits, vector_element_bits_tree): New.
* match.pd: Use the new functions instead of determining the
vector element size directly from TYPE_SIZE(_UNIT).
* tree-vect-data-refs.c (vect_gather_scatter_fn_p): Likewise.
* tree-vect-patterns.c (vect_recog_mask_conversion_pattern): Likewise.
* tree-vect-stmts.c (vect_is_simple_cond): Likewise.
* tree-vect-generic.c (expand_vector_piecewise): Likewise.
(expand_vector_conversion): Likewise.
(expand_vector_addition): Likewise for a TYPE_SIZE_UNIT used as
a divisor. Convert the dividend to bits to compensate.
* tree-vect-loop.c (vectorizable_live_operation): Call
vector_element_bits instead of open-coding it.
|
|
This removes trivial instances of SLP_INSTANCE_GROUP_SIZE and refrains
from using a "SLP instance" which nowadays is just one of the possibly
many entries into the SLP graph.
2020-05-06 Richard Biener <rguenther@suse.de>
* tree-vectorizer.h (vect_transform_slp_perm_load): Adjust.
* tree-vect-data-refs.c (vect_slp_analyze_node_dependences):
Remove slp_instance parameter, just iterate over all scalar stmts.
(vect_slp_analyze_instance_dependence): Adjust and likewise.
* tree-vect-slp.c (vect_bb_slp_scalar_cost): Remove unused BB
parameter.
(vect_schedule_slp): Just iterate over all scalar stmts.
(vect_supported_load_permutation_p): Adjust.
(vect_transform_slp_perm_load): Remove slp_instance parameter,
instead use the number of lanes in the node as group size.
* tree-vect-stmts.c (vect_model_load_cost): Get vectorization
factor instead of slp_instance as parameter.
(vectorizable_load): Adjust.
|
|
Soonish we'll get SLP nodes which have no corresponding scalar
stmt and thus not stmt_vec_info and thus no way to get back to
the associated vec_info. This patch makes the vec_info available
as part of the APIs instead of putting in that back-pointer into
the leaf data structures.
2020-05-05 Richard Biener <rguenther@suse.de>
* tree-vectorizer.h (_stmt_vec_info::vinfo): Remove.
(STMT_VINFO_LOOP_VINFO): Likewise.
(STMT_VINFO_BB_VINFO): Likewise.
* tree-vect-data-refs.c: Adjust for the above, adding vec_info *
parameters and adjusting calls.
* tree-vect-loop-manip.c: Likewise.
* tree-vect-loop.c: Likewise.
* tree-vect-patterns.c: Likewise.
* tree-vect-slp.c: Likewise.
* tree-vect-stmts.c: Likewise.
* tree-vectorizer.c: Likewise.
* target.def (add_stmt_cost): Add vec_info * parameter.
* target.h (stmt_in_inner_loop_p): Likewise.
* targhooks.c (default_add_stmt_cost): Adjust.
* doc/tm.texi: Re-generate.
* config/aarch64/aarch64.c (aarch64_extending_load_p): Add
vec_info * parameter and adjust.
(aarch64_sve_adjust_stmt_cost): Likewise.
(aarch64_add_stmt_cost): Likewise.
* config/arm/arm.c (arm_add_stmt_cost): Likewise.
* config/i386/i386.c (ix86_add_stmt_cost): Likewise.
* config/rs6000/rs6000.c (rs6000_add_stmt_cost): Likewise.
|
|
This PR is a regression caused by r256644, which added support for alias
checks involving variable strides. One of the changes in that commit
was to split the access size out of the segment length. The PR shows
that I hadn't done that correctly for the handling of negative strides
in vect_compile_time_alias. The old code was:
const_length_a = (-wi::to_poly_wide (segment_length_a)).force_uhwi ();
offset_a = (offset_a + vect_get_scalar_dr_size (a)) - const_length_a;
where vect_get_scalar_dr_size (a) was cancelling out the subtraction
of the access size inherent in "- const_length_a". Taking the access
size out of the segment length meant that the addition was no longer
needed/correct.
2020-02-19 Richard Sandiford <richard.sandiford@arm.com>
gcc/
PR tree-optimization/93767
* tree-vect-data-refs.c (vect_compile_time_alias): Remove the
access-size bias from the offset calculations for negative strides.
gcc/testsuite/
PR tree-optimization/93767
* gcc.dg/vect/pr93767.c: New test.
|
|
gcc/ChangeLog:
2020-01-10 Andre Vieira <andre.simoesdiasvieira@arm.com>
* tree-vect-data-refs.c (vect_create_addr_base_for_vector_ref): Use
get_dr_vinfo_offset
* tree-vect-loop.c (update_epilogue_loop_vinfo): Remove orig_drs_init
parameter and its use to reset DR_OFFSET's.
(vect_transform_loop): Remove orig_drs_init argument.
* tree-vect-loop-manip.c (vect_update_init_of_dr): Update the offset
member of dr_vec_info rather than the offset of the associated
data_reference's innermost_loop_behavior.
(vect_update_init_of_dr): Pass dr_vec_info instead of data_reference.
(vect_do_peeling): Remove orig_drs_init parameter and its construction.
* tree-vect-stmts.c (check_scan_store): Replace use of DR_OFFSET with
get_dr_vinfo_offset.
(vectorizable_store): Likewise.
(vectorizable_load): Likewise.
From-SVN: r280107
|
|
From-SVN: r279813
|
|
Allowing mixed vector sizes broke the assumption in the following assert,
since it's now possible for different accesses to require different
levels of alignment:
/* FORNOW: use the same mask to test all potentially unaligned
references in the loop. The vectorizer currently supports
a single vector size, see the reference to
GET_MODE_NUNITS (TYPE_MODE (vectype)) where the
vectorization factor is computed. */
gcc_assert (!LOOP_VINFO_PTR_MASK (loop_vinfo)
|| LOOP_VINFO_PTR_MASK (loop_vinfo) == mask);
I guess we could try to over-align smaller accesses so that all
of them are consistent, or try to support multiple alignment masks,
but for now the easiest fix seems to be to turn the assert into a
bail-out check.
2019-11-21 Richard Sandiford <richard.sandiford@arm.com>
gcc/
PR tree-optimization/92526
* tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Reject
versioning for alignment if the accesses do not have a consistent
mask, rather than asserting that the masks are consistent.
gcc/testsuite/
PR tree-optimization/92526
* gcc.target/aarch64/pr92526.c: New test.
From-SVN: r278592
|
|
This patch adds a bunch of flags to dr_with_seg_len_pair_t,
for use by later patches. The update to tree-loop-distribution.c
is conservatively correct, but might be tweakable later.
2019-11-16 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-data-ref.h (DR_ALIAS_RAW, DR_ALIAS_WAR, DR_ALIAS_WAW)
(DR_ALIAS_ARBITRARY, DR_ALIAS_SWAPPED, DR_ALIAS_UNSWAPPED): New flags.
(dr_with_seg_len_pair_t::sequencing): New enum.
(dr_with_seg_len_pair_t::flags): New member variable.
(dr_with_seg_len_pair_t::dr_with_seg_len_pair_t): Take a sequencing
parameter and initialize the flags member variable.
* tree-loop-distribution.c (compute_alias_check_pairs): Update
call accordingly.
* tree-vect-data-refs.c (vect_prune_runtime_alias_test_list): Likewise.
Ensure the two data references in an alias pair are in statement
order, if there is a defined order.
* tree-data-ref.c (prune_runtime_alias_test_list): Use
DR_ALIAS_SWAPPED and DR_ALIAS_UNSWAPPED to record whether we've
swapped the references in a dr_with_seg_len_pair_t. OR together
the flags when merging two dr_with_seg_len_pair_ts. After merging,
try to restore the original dr_with_seg_len order, updating the
flags if that fails.
From-SVN: r278350
|
|
The two users of tree-data-ref's runtime alias checks both canonicalise
the order of the dr_with_seg_lens in a pair before passing them to
prune_runtime_alias_test_list. It's more convenient for later patches
if prune_runtime_alias_test_list does that itself.
2019-11-16 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-data-ref.c (prune_runtime_alias_test_list): Sort the
two accesses in each dr_with_seg_len_pair_t before trying to
combine separate dr_with_seg_len_pair_ts.
* tree-loop-distribution.c (compute_alias_check_pairs): Don't do
that here.
* tree-vect-data-refs.c (vect_prune_runtime_alias_test_list): Likewise.
From-SVN: r278348
|
|
The BB vectoriser picked vector types in the same way as the loop
vectoriser: it picked a vector mode/size for the region and then
based all the vector types off that choice. This meant we could
end up trying to use vector types that had too many elements for
the group size.
The main part of this patch is therefore about passing the SLP
group size down to routines like get_vectype_for_scalar_type and
ensuring that each vector type in the SLP tree is chosen wrt the
group size. That part in itself is pretty easy and mechanical.
The main warts are:
(1) We normally pick a STMT_VINFO_VECTYPE for data references at an
early stage (vect_analyze_data_refs). However, nothing in the
BB vectoriser relied on this, or on the min_vf calculated from it.
I couldn't see anything other than vect_recog_bool_pattern that
tried to access the vector type before the SLP tree is built.
(2) It's possible for the same statement to be used in groups of
different sizes. Taking the group size into account meant that
we could try to pick different vector types for the same statement.
This problem should go away with the move to doing everything on
SLP trees, where presumably we would attach the vector type to the
SLP node rather than the stmt_vec_info. Until then, the patch just
uses a first-come, first-served approach.
(3) A similar problem exists for grouped data references, where
different statements in the same dataref group could be used
in SLP nodes that have different group sizes. The patch copes
with that by making sure that all vector types in a dataref
group remain consistent.
The patch means that:
void
f (int *x, short *y)
{
x[0] += y[0];
x[1] += y[1];
x[2] += y[2];
x[3] += y[3];
}
now produces:
ldr q0, [x0]
ldr d1, [x1]
saddw v0.4s, v0.4s, v1.4h
str q0, [x0]
ret
instead of:
ldrsh w2, [x1]
ldrsh w3, [x1, 2]
fmov s0, w2
ldrsh w2, [x1, 4]
ldrsh w1, [x1, 6]
ins v0.s[1], w3
ldr q1, [x0]
ins v0.s[2], w2
ins v0.s[3], w1
add v0.4s, v0.4s, v1.4s
str q0, [x0]
ret
Unfortunately it also means we start to vectorise
gcc.target/i386/pr84101.c for -m32. That seems like a target
cost issue though; see PR92265 for details.
2019-11-16 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (vect_get_vector_types_for_stmt): Take an
optional maximum nunits.
(get_vectype_for_scalar_type): Likewise. Also declare a form that
takes an slp_tree.
(get_mask_type_for_scalar_type): Take an optional slp_tree.
(vect_get_mask_type_for_stmt): Likewise.
* tree-vect-data-refs.c (vect_analyze_data_refs): Don't store
the vector type in STMT_VINFO_VECTYPE for BB vectorization.
* tree-vect-patterns.c (vect_recog_bool_pattern): Use
vect_get_vector_types_for_stmt instead of STMT_VINFO_VECTYPE
to get an assumed vector type for data references.
* tree-vect-slp.c (vect_update_shared_vectype): New function.
(vect_update_all_shared_vectypes): Likewise.
(vect_build_slp_tree_1): Pass the group size to
vect_get_vector_types_for_stmt. Use vect_update_shared_vectype
for BB vectorization.
(vect_build_slp_tree_2): Call vect_update_all_shared_vectypes
before building the vectof from scalars.
(vect_analyze_slp_instance): Pass the group size to
get_vectype_for_scalar_type.
(vect_slp_analyze_node_operations_1): Don't recompute the vector
types for BB vectorization here; just handle the case in which
we deferred the choice for booleans.
(vect_get_constant_vectors): Pass the slp_tree to
get_vectype_for_scalar_type.
* tree-vect-stmts.c (vect_prologue_cost_for_slp_op): Likewise.
(vectorizable_call): Likewise.
(vectorizable_simd_clone_call): Likewise.
(vectorizable_conversion): Likewise.
(vectorizable_shift): Likewise.
(vectorizable_operation): Likewise.
(vectorizable_comparison): Likewise.
(vect_is_simple_cond): Take the slp_tree as argument and
pass it to get_vectype_for_scalar_type.
(vectorizable_condition): Update call accordingly.
(get_vectype_for_scalar_type): Take a group_size argument.
For BB vectorization, limit the the vector to that number
of elements. Also define an overload that takes an slp_tree.
(get_mask_type_for_scalar_type): Add an slp_tree argument and
pass it to get_vectype_for_scalar_type.
(vect_get_vector_types_for_stmt): Add a group_size argument
and pass it to get_vectype_for_scalar_type. Don't use the
cached vector type for BB vectorization if a group size is given.
Handle data references in that case.
(vect_get_mask_type_for_stmt): Take an slp_tree argument and
pass it to get_mask_type_for_scalar_type.
gcc/testsuite/
* gcc.dg/vect/bb-slp-4.c: Expect the block to be vectorized
with -fno-vect-cost-model.
* gcc.dg/vect/bb-slp-bool-1.c: New test.
* gcc.target/aarch64/vect_mixed_sizes_14.c: Likewise.
* gcc.target/i386/pr84101.c: XFAIL for -m32.
From-SVN: r278334
|
|
2019-11-12 Martin Liska <mliska@suse.cz>
* Makefile.in: Remove PARAMS_H and params.list
and params.options.
* params-enum.h: Remove.
* params-list.h: Remove.
* params-options.h: Remove.
* params.c: Remove.
* params.def: Remove.
* params.h: Remove.
* asan.c: Do not include params.h.
* auto-profile.c: Likewise.
* bb-reorder.c: Likewise.
* builtins.c: Likewise.
* cfgcleanup.c: Likewise.
* cfgexpand.c: Likewise.
* cfgloopanal.c: Likewise.
* cgraph.c: Likewise.
* combine.c: Likewise.
* common/config/aarch64/aarch64-common.c: Likewise.
* common/config/gcn/gcn-common.c: Likewise.
* common/config/ia64/ia64-common.c: Likewise.
* common/config/powerpcspe/powerpcspe-common.c: Likewise.
* common/config/rs6000/rs6000-common.c: Likewise.
* common/config/sh/sh-common.c: Likewise.
* config/aarch64/aarch64.c: Likewise.
* config/alpha/alpha.c: Likewise.
* config/arm/arm.c: Likewise.
* config/avr/avr.c: Likewise.
* config/csky/csky.c: Likewise.
* config/i386/i386-builtins.c: Likewise.
* config/i386/i386-expand.c: Likewise.
* config/i386/i386-features.c: Likewise.
* config/i386/i386-options.c: Likewise.
* config/i386/i386.c: Likewise.
* config/ia64/ia64.c: Likewise.
* config/rs6000/rs6000-logue.c: Likewise.
* config/rs6000/rs6000.c: Likewise.
* config/s390/s390.c: Likewise.
* config/sparc/sparc.c: Likewise.
* config/visium/visium.c: Likewise.
* coverage.c: Likewise.
* cprop.c: Likewise.
* cse.c: Likewise.
* cselib.c: Likewise.
* dse.c: Likewise.
* emit-rtl.c: Likewise.
* explow.c: Likewise.
* final.c: Likewise.
* fold-const.c: Likewise.
* gcc.c: Likewise.
* gcse.c: Likewise.
* ggc-common.c: Likewise.
* ggc-page.c: Likewise.
* gimple-loop-interchange.cc: Likewise.
* gimple-loop-jam.c: Likewise.
* gimple-loop-versioning.cc: Likewise.
* gimple-ssa-split-paths.c: Likewise.
* gimple-ssa-sprintf.c: Likewise.
* gimple-ssa-store-merging.c: Likewise.
* gimple-ssa-strength-reduction.c: Likewise.
* gimple-ssa-warn-alloca.c: Likewise.
* gimple-ssa-warn-restrict.c: Likewise.
* graphite-isl-ast-to-gimple.c: Likewise.
* graphite-optimize-isl.c: Likewise.
* graphite-scop-detection.c: Likewise.
* graphite-sese-to-poly.c: Likewise.
* graphite.c: Likewise.
* haifa-sched.c: Likewise.
* hsa-gen.c: Likewise.
* ifcvt.c: Likewise.
* ipa-cp.c: Likewise.
* ipa-fnsummary.c: Likewise.
* ipa-inline-analysis.c: Likewise.
* ipa-inline.c: Likewise.
* ipa-polymorphic-call.c: Likewise.
* ipa-profile.c: Likewise.
* ipa-prop.c: Likewise.
* ipa-split.c: Likewise.
* ipa-sra.c: Likewise.
* ira-build.c: Likewise.
* ira-conflicts.c: Likewise.
* loop-doloop.c: Likewise.
* loop-invariant.c: Likewise.
* loop-unroll.c: Likewise.
* lra-assigns.c: Likewise.
* lra-constraints.c: Likewise.
* modulo-sched.c: Likewise.
* opt-suggestions.c: Likewise.
* opts.c: Likewise.
* postreload-gcse.c: Likewise.
* predict.c: Likewise.
* reload.c: Likewise.
* reorg.c: Likewise.
* resource.c: Likewise.
* sanopt.c: Likewise.
* sched-deps.c: Likewise.
* sched-ebb.c: Likewise.
* sched-rgn.c: Likewise.
* sel-sched-ir.c: Likewise.
* sel-sched.c: Likewise.
* shrink-wrap.c: Likewise.
* stmt.c: Likewise.
* targhooks.c: Likewise.
* toplev.c: Likewise.
* tracer.c: Likewise.
* trans-mem.c: Likewise.
* tree-chrec.c: Likewise.
* tree-data-ref.c: Likewise.
* tree-if-conv.c: Likewise.
* tree-inline.c: Likewise.
* tree-loop-distribution.c: Likewise.
* tree-parloops.c: Likewise.
* tree-predcom.c: Likewise.
* tree-profile.c: Likewise.
* tree-scalar-evolution.c: Likewise.
* tree-sra.c: Likewise.
* tree-ssa-ccp.c: Likewise.
* tree-ssa-dom.c: Likewise.
* tree-ssa-dse.c: Likewise.
* tree-ssa-ifcombine.c: Likewise.
* tree-ssa-loop-ch.c: Likewise.
* tree-ssa-loop-im.c: Likewise.
* tree-ssa-loop-ivcanon.c: Likewise.
* tree-ssa-loop-ivopts.c: Likewise.
* tree-ssa-loop-manip.c: Likewise.
* tree-ssa-loop-niter.c: Likewise.
* tree-ssa-loop-prefetch.c: Likewise.
* tree-ssa-loop-unswitch.c: Likewise.
* tree-ssa-math-opts.c: Likewise.
* tree-ssa-phiopt.c: Likewise.
* tree-ssa-pre.c: Likewise.
* tree-ssa-reassoc.c: Likewise.
* tree-ssa-sccvn.c: Likewise.
* tree-ssa-scopedtables.c: Likewise.
* tree-ssa-sink.c: Likewise.
* tree-ssa-strlen.c: Likewise.
* tree-ssa-structalias.c: Likewise.
* tree-ssa-tail-merge.c: Likewise.
* tree-ssa-threadbackward.c: Likewise.
* tree-ssa-threadedge.c: Likewise.
* tree-ssa-uninit.c: Likewise.
* tree-switch-conversion.c: Likewise.
* tree-vect-data-refs.c: Likewise.
* tree-vect-loop.c: Likewise.
* tree-vect-slp.c: Likewise.
* tree-vrp.c: Likewise.
* tree.c: Likewise.
* value-prof.c: Likewise.
* var-tracking.c: Likewise.
2019-11-12 Martin Liska <mliska@suse.cz>
* gimple-parser.c: Do not include params.h.
2019-11-12 Martin Liska <mliska@suse.cz>
* name-lookup.c: Do not include params.h.
* typeck.c: Likewise.
2019-11-12 Martin Liska <mliska@suse.cz>
* lto-common.c: Do not include params.h.
* lto-partition.c: Likewise.
* lto.c: Likewise.
From-SVN: r278086
|
|
2019-11-12 Martin Liska <mliska@suse.cz>
* asan.c (asan_sanitize_stack_p): Replace old parameter syntax
with the new one, include opts.h if needed. Use SET_OPTION_IF_UNSET
macro.
(asan_sanitize_allocas_p): Likewise.
(asan_emit_stack_protection): Likewise.
(asan_protect_global): Likewise.
(instrument_derefs): Likewise.
(instrument_builtin_call): Likewise.
(asan_expand_mark_ifn): Likewise.
* auto-profile.c (auto_profile): Likewise.
* bb-reorder.c (copy_bb_p): Likewise.
(duplicate_computed_gotos): Likewise.
* builtins.c (inline_expand_builtin_string_cmp): Likewise.
* cfgcleanup.c (try_crossjump_to_edge): Likewise.
(try_crossjump_bb): Likewise.
* cfgexpand.c (defer_stack_allocation): Likewise.
(stack_protect_classify_type): Likewise.
(pass_expand::execute): Likewise.
* cfgloopanal.c (expected_loop_iterations_unbounded): Likewise.
(estimate_reg_pressure_cost): Likewise.
* cgraph.c (cgraph_edge::maybe_hot_p): Likewise.
* combine.c (combine_instructions): Likewise.
(record_value_for_reg): Likewise.
* common/config/aarch64/aarch64-common.c (aarch64_option_validate_param): Likewise.
(aarch64_option_default_params): Likewise.
* common/config/ia64/ia64-common.c (ia64_option_default_params): Likewise.
* common/config/powerpcspe/powerpcspe-common.c (rs6000_option_default_params): Likewise.
* common/config/rs6000/rs6000-common.c (rs6000_option_default_params): Likewise.
* common/config/sh/sh-common.c (sh_option_default_params): Likewise.
* config/aarch64/aarch64.c (aarch64_output_probe_stack_range): Likewise.
(aarch64_allocate_and_probe_stack_space): Likewise.
(aarch64_expand_epilogue): Likewise.
(aarch64_override_options_internal): Likewise.
* config/alpha/alpha.c (alpha_option_override): Likewise.
* config/arm/arm.c (arm_option_override): Likewise.
(arm_valid_target_attribute_p): Likewise.
* config/i386/i386-options.c (ix86_option_override_internal): Likewise.
* config/i386/i386.c (get_probe_interval): Likewise.
(ix86_adjust_stack_and_probe_stack_clash): Likewise.
(ix86_max_noce_ifcvt_seq_cost): Likewise.
* config/ia64/ia64.c (ia64_adjust_cost): Likewise.
* config/rs6000/rs6000-logue.c (get_stack_clash_protection_probe_interval): Likewise.
(get_stack_clash_protection_guard_size): Likewise.
* config/rs6000/rs6000.c (rs6000_option_override_internal): Likewise.
* config/s390/s390.c (allocate_stack_space): Likewise.
(s390_emit_prologue): Likewise.
(s390_option_override_internal): Likewise.
* config/sparc/sparc.c (sparc_option_override): Likewise.
* config/visium/visium.c (visium_option_override): Likewise.
* coverage.c (get_coverage_counts): Likewise.
(coverage_compute_profile_id): Likewise.
(coverage_begin_function): Likewise.
(coverage_end_function): Likewise.
* cse.c (cse_find_path): Likewise.
(cse_extended_basic_block): Likewise.
(cse_main): Likewise.
* cselib.c (cselib_invalidate_mem): Likewise.
* dse.c (dse_step1): Likewise.
* emit-rtl.c (set_new_first_and_last_insn): Likewise.
(get_max_insn_count): Likewise.
(make_debug_insn_raw): Likewise.
(init_emit): Likewise.
* explow.c (compute_stack_clash_protection_loop_data): Likewise.
* final.c (compute_alignments): Likewise.
* fold-const.c (fold_range_test): Likewise.
(fold_truth_andor): Likewise.
(tree_single_nonnegative_warnv_p): Likewise.
(integer_valued_real_single_p): Likewise.
* gcse.c (want_to_gcse_p): Likewise.
(prune_insertions_deletions): Likewise.
(hoist_code): Likewise.
(gcse_or_cprop_is_too_expensive): Likewise.
* ggc-common.c: Likewise.
* ggc-page.c (ggc_collect): Likewise.
* gimple-loop-interchange.cc (MAX_NUM_STMT): Likewise.
(MAX_DATAREFS): Likewise.
(OUTER_STRIDE_RATIO): Likewise.
* gimple-loop-jam.c (tree_loop_unroll_and_jam): Likewise.
* gimple-loop-versioning.cc (loop_versioning::max_insns_for_loop): Likewise.
* gimple-ssa-split-paths.c (is_feasible_trace): Likewise.
* gimple-ssa-store-merging.c (imm_store_chain_info::try_coalesce_bswap): Likewise.
(imm_store_chain_info::coalesce_immediate_stores): Likewise.
(imm_store_chain_info::output_merged_store): Likewise.
(pass_store_merging::process_store): Likewise.
* gimple-ssa-strength-reduction.c (find_basis_for_base_expr): Likewise.
* graphite-isl-ast-to-gimple.c (class translate_isl_ast_to_gimple): Likewise.
(scop_to_isl_ast): Likewise.
* graphite-optimize-isl.c (get_schedule_for_node_st): Likewise.
(optimize_isl): Likewise.
* graphite-scop-detection.c (build_scops): Likewise.
* haifa-sched.c (set_modulo_params): Likewise.
(rank_for_schedule): Likewise.
(model_add_to_worklist): Likewise.
(model_promote_insn): Likewise.
(model_choose_insn): Likewise.
(queue_to_ready): Likewise.
(autopref_multipass_dfa_lookahead_guard): Likewise.
(schedule_block): Likewise.
(sched_init): Likewise.
* hsa-gen.c (init_prologue): Likewise.
* ifcvt.c (bb_ok_for_noce_convert_multiple_sets): Likewise.
(cond_move_process_if_block): Likewise.
* ipa-cp.c (ipcp_lattice::add_value): Likewise.
(merge_agg_lats_step): Likewise.
(devirtualization_time_bonus): Likewise.
(hint_time_bonus): Likewise.
(incorporate_penalties): Likewise.
(good_cloning_opportunity_p): Likewise.
(ipcp_propagate_stage): Likewise.
* ipa-fnsummary.c (decompose_param_expr): Likewise.
(set_switch_stmt_execution_predicate): Likewise.
(analyze_function_body): Likewise.
(compute_fn_summary): Likewise.
* ipa-inline-analysis.c (estimate_growth): Likewise.
* ipa-inline.c (caller_growth_limits): Likewise.
(inline_insns_single): Likewise.
(inline_insns_auto): Likewise.
(can_inline_edge_by_limits_p): Likewise.
(want_early_inline_function_p): Likewise.
(big_speedup_p): Likewise.
(want_inline_small_function_p): Likewise.
(want_inline_self_recursive_call_p): Likewise.
(edge_badness): Likewise.
(recursive_inlining): Likewise.
(compute_max_insns): Likewise.
(early_inliner): Likewise.
* ipa-polymorphic-call.c (csftc_abort_walking_p): Likewise.
* ipa-profile.c (ipa_profile): Likewise.
* ipa-prop.c (determine_known_aggregate_parts): Likewise.
(ipa_analyze_node): Likewise.
(ipcp_transform_function): Likewise.
* ipa-split.c (consider_split): Likewise.
* ipa-sra.c (allocate_access): Likewise.
(process_scan_results): Likewise.
(ipa_sra_summarize_function): Likewise.
(pull_accesses_from_callee): Likewise.
* ira-build.c (loop_compare_func): Likewise.
(mark_loops_for_removal): Likewise.
* ira-conflicts.c (build_conflict_bit_table): Likewise.
* loop-doloop.c (doloop_optimize): Likewise.
* loop-invariant.c (gain_for_invariant): Likewise.
(move_loop_invariants): Likewise.
* loop-unroll.c (decide_unroll_constant_iterations): Likewise.
(decide_unroll_runtime_iterations): Likewise.
(decide_unroll_stupid): Likewise.
(expand_var_during_unrolling): Likewise.
* lra-assigns.c (spill_for): Likewise.
* lra-constraints.c (EBB_PROBABILITY_CUTOFF): Likewise.
* modulo-sched.c (sms_schedule): Likewise.
(DFA_HISTORY): Likewise.
* opts.c (default_options_optimization): Likewise.
(finish_options): Likewise.
(common_handle_option): Likewise.
* postreload-gcse.c (eliminate_partially_redundant_load): Likewise.
(if): Likewise.
* predict.c (get_hot_bb_threshold): Likewise.
(maybe_hot_count_p): Likewise.
(probably_never_executed): Likewise.
(predictable_edge_p): Likewise.
(predict_loops): Likewise.
(expr_expected_value_1): Likewise.
(tree_predict_by_opcode): Likewise.
(handle_missing_profiles): Likewise.
* reload.c (find_equiv_reg): Likewise.
* reorg.c (redundant_insn): Likewise.
* resource.c (mark_target_live_regs): Likewise.
(incr_ticks_for_insn): Likewise.
* sanopt.c (pass_sanopt::execute): Likewise.
* sched-deps.c (sched_analyze_1): Likewise.
(sched_analyze_2): Likewise.
(sched_analyze_insn): Likewise.
(deps_analyze_insn): Likewise.
* sched-ebb.c (schedule_ebbs): Likewise.
* sched-rgn.c (find_single_block_region): Likewise.
(too_large): Likewise.
(haifa_find_rgns): Likewise.
(extend_rgns): Likewise.
(new_ready): Likewise.
(schedule_region): Likewise.
(sched_rgn_init): Likewise.
* sel-sched-ir.c (make_region_from_loop): Likewise.
* sel-sched-ir.h (MAX_WS): Likewise.
* sel-sched.c (process_pipelined_exprs): Likewise.
(sel_setup_region_sched_flags): Likewise.
* shrink-wrap.c (try_shrink_wrapping): Likewise.
* targhooks.c (default_max_noce_ifcvt_seq_cost): Likewise.
* toplev.c (print_version): Likewise.
(process_options): Likewise.
* tracer.c (tail_duplicate): Likewise.
* trans-mem.c (tm_log_add): Likewise.
* tree-chrec.c (chrec_fold_plus_1): Likewise.
* tree-data-ref.c (split_constant_offset): Likewise.
(compute_all_dependences): Likewise.
* tree-if-conv.c (MAX_PHI_ARG_NUM): Likewise.
* tree-inline.c (remap_gimple_stmt): Likewise.
* tree-loop-distribution.c (MAX_DATAREFS_NUM): Likewise.
* tree-parloops.c (MIN_PER_THREAD): Likewise.
(create_parallel_loop): Likewise.
* tree-predcom.c (determine_unroll_factor): Likewise.
* tree-scalar-evolution.c (instantiate_scev_r): Likewise.
* tree-sra.c (analyze_all_variable_accesses): Likewise.
* tree-ssa-ccp.c (fold_builtin_alloca_with_align): Likewise.
* tree-ssa-dse.c (setup_live_bytes_from_ref): Likewise.
(dse_optimize_redundant_stores): Likewise.
(dse_classify_store): Likewise.
* tree-ssa-ifcombine.c (ifcombine_ifandif): Likewise.
* tree-ssa-loop-ch.c (ch_base::copy_headers): Likewise.
* tree-ssa-loop-im.c (LIM_EXPENSIVE): Likewise.
* tree-ssa-loop-ivcanon.c (try_unroll_loop_completely): Likewise.
(try_peel_loop): Likewise.
(tree_unroll_loops_completely): Likewise.
* tree-ssa-loop-ivopts.c (avg_loop_niter): Likewise.
(CONSIDER_ALL_CANDIDATES_BOUND): Likewise.
(MAX_CONSIDERED_GROUPS): Likewise.
(ALWAYS_PRUNE_CAND_SET_BOUND): Likewise.
* tree-ssa-loop-manip.c (can_unroll_loop_p): Likewise.
* tree-ssa-loop-niter.c (MAX_ITERATIONS_TO_TRACK): Likewise.
* tree-ssa-loop-prefetch.c (PREFETCH_BLOCK): Likewise.
(L1_CACHE_SIZE_BYTES): Likewise.
(L2_CACHE_SIZE_BYTES): Likewise.
(should_issue_prefetch_p): Likewise.
(schedule_prefetches): Likewise.
(determine_unroll_factor): Likewise.
(volume_of_references): Likewise.
(add_subscript_strides): Likewise.
(self_reuse_distance): Likewise.
(mem_ref_count_reasonable_p): Likewise.
(insn_to_prefetch_ratio_too_small_p): Likewise.
(loop_prefetch_arrays): Likewise.
(tree_ssa_prefetch_arrays): Likewise.
* tree-ssa-loop-unswitch.c (tree_unswitch_single_loop): Likewise.
* tree-ssa-math-opts.c (gimple_expand_builtin_pow): Likewise.
(convert_mult_to_fma): Likewise.
(math_opts_dom_walker::after_dom_children): Likewise.
* tree-ssa-phiopt.c (cond_if_else_store_replacement): Likewise.
(hoist_adjacent_loads): Likewise.
(gate_hoist_loads): Likewise.
* tree-ssa-pre.c (translate_vuse_through_block): Likewise.
(compute_partial_antic_aux): Likewise.
* tree-ssa-reassoc.c (get_reassociation_width): Likewise.
* tree-ssa-sccvn.c (vn_reference_lookup_pieces): Likewise.
(vn_reference_lookup): Likewise.
(do_rpo_vn): Likewise.
* tree-ssa-scopedtables.c (avail_exprs_stack::lookup_avail_expr): Likewise.
* tree-ssa-sink.c (select_best_block): Likewise.
* tree-ssa-strlen.c (new_stridx): Likewise.
(new_addr_stridx): Likewise.
(get_range_strlen_dynamic): Likewise.
(class ssa_name_limit_t): Likewise.
* tree-ssa-structalias.c (push_fields_onto_fieldstack): Likewise.
(create_variable_info_for_1): Likewise.
(init_alias_vars): Likewise.
* tree-ssa-tail-merge.c (find_clusters_1): Likewise.
(tail_merge_optimize): Likewise.
* tree-ssa-threadbackward.c (thread_jumps::profitable_jump_thread_path): Likewise.
(thread_jumps::fsm_find_control_statement_thread_paths): Likewise.
(thread_jumps::find_jump_threads_backwards): Likewise.
* tree-ssa-threadedge.c (record_temporary_equivalences_from_stmts_at_dest): Likewise.
* tree-ssa-uninit.c (compute_control_dep_chain): Likewise.
* tree-switch-conversion.c (switch_conversion::check_range): Likewise.
(jump_table_cluster::can_be_handled): Likewise.
* tree-switch-conversion.h (jump_table_cluster::case_values_threshold): Likewise.
(SWITCH_CONVERSION_BRANCH_RATIO): Likewise.
(param_switch_conversion_branch_ratio): Likewise.
* tree-vect-data-refs.c (vect_mark_for_runtime_alias_test): Likewise.
(vect_enhance_data_refs_alignment): Likewise.
(vect_prune_runtime_alias_test_list): Likewise.
* tree-vect-loop.c (vect_analyze_loop_costing): Likewise.
(vect_get_datarefs_in_loop): Likewise.
(vect_analyze_loop): Likewise.
* tree-vect-slp.c (vect_slp_bb): Likewise.
* tree-vectorizer.h: Likewise.
* tree-vrp.c (find_switch_asserts): Likewise.
(vrp_prop::check_mem_ref): Likewise.
* tree.c (wide_int_to_tree_1): Likewise.
(cache_integer_cst): Likewise.
* var-tracking.c (EXPR_USE_DEPTH): Likewise.
(reverse_op): Likewise.
(vt_find_locations): Likewise.
2019-11-12 Martin Liska <mliska@suse.cz>
* gimple-parser.c (c_parser_parse_gimple_body): Replace old parameter syntax
with the new one, include opts.h if needed. Use SET_OPTION_IF_UNSET
macro.
2019-11-12 Martin Liska <mliska@suse.cz>
* name-lookup.c (namespace_hints::namespace_hints): Replace old parameter syntax
with the new one, include opts.h if needed. Use SET_OPTION_IF_UNSET
macro.
* typeck.c (comptypes): Likewise.
2019-11-12 Martin Liska <mliska@suse.cz>
* lto-partition.c (lto_balanced_map): Replace old parameter syntax
with the new one, include opts.h if needed. Use SET_OPTION_IF_UNSET
macro.
* lto.c (do_whole_program_analysis): Likewise.
From-SVN: r278085
|
|
gcc/ChangeLog:
2019-11-08 Andre Vieira <andre.simoesdiasvieira@arm.com>
PR tree-optimization/92351
* tree-vect-data-refs.c (vect_compute_data_ref_alignment): When we are
peeling the main loop for alignment, make sure to set the misalignment
of the epilogue's data references to DR_MISALIGNMENT_UNKNOWN.
gcc/testsuite/ChangeLog:
2019-11-08 Andre Vieira <andre.simoesdiasvieira@arm.com>
PR tree-optimization/92351
* gcc.dg/vect/vect-peel-2.c: Disable epilogue vectorization and
split the source of this test to...
* gcc.dg/vect/vect-peel-2-src.c: ... This.
* gcc.dg/vect/vect-peel-2-epilogues.c: New test.
From-SVN: r277974
|
|
The gather and scatter optabs required the vector offset to be
the integer equivalent of the vector mode being loaded or stored.
This patch generalises them so that the two vectors can have different
element sizes, although they still need to have the same number of
elements.
One consequence of this is that it's possible (if unlikely)
for two IFN_GATHER_LOADs to have the same arguments but different
return types. E.g. the same scalar base and vector of 32-bit offsets
could be used to load 8-bit elements and to load 16-bit elements.
From just looking at the arguments, we could wrongly deduce that
they're equivalent.
I know we saw this happen at one point with IFN_WHILE_ULT,
and we dealt with it there by passing a zero of the return type
as an extra argument. Doing the same here also makes the load
and store functions have the same argument assignment.
For now this patch should be a no-op, but later SVE patches take
advantage of the new flexibility.
2019-11-08 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* optabs.def (gather_load_optab, mask_gather_load_optab)
(scatter_store_optab, mask_scatter_store_optab): Turn into
conversion optabs, with the offset mode given explicitly.
* doc/md.texi: Update accordingly.
* config/aarch64/aarch64-sve-builtins-base.cc
(svld1_gather_impl::expand): Likewise.
(svst1_scatter_impl::expand): Likewise.
* internal-fn.c (gather_load_direct, scatter_store_direct): Likewise.
(expand_scatter_store_optab_fn): Likewise.
(direct_gather_load_optab_supported_p): Likewise.
(direct_scatter_store_optab_supported_p): Likewise.
(expand_gather_load_optab_fn): Likewise. Expect the mask argument
to be argument 4.
(internal_fn_mask_index): Return 4 for IFN_MASK_GATHER_LOAD.
(internal_gather_scatter_fn_supported_p): Replace the offset sign
argument with the offset vector type. Require the two vector
types to have the same number of elements but allow their element
sizes to be different. Treat the optabs as conversion optabs.
* internal-fn.h (internal_gather_scatter_fn_supported_p): Update
prototype accordingly.
* optabs-query.c (supports_at_least_one_mode_p): Replace with...
(supports_vec_convert_optab_p): ...this new function.
(supports_vec_gather_load_p): Update accordingly.
(supports_vec_scatter_store_p): Likewise.
* tree-vectorizer.h (vect_gather_scatter_fn_p): Take a vec_info.
Replace the offset sign and bits parameters with a scalar type tree.
* tree-vect-data-refs.c (vect_gather_scatter_fn_p): Likewise.
Pass back the offset vector type instead of the scalar element type.
Allow the offset to be wider than the memory elements. Search for
an offset type that the target supports, stopping once we've
reached the maximum of the element size and pointer size.
Update call to internal_gather_scatter_fn_supported_p.
(vect_check_gather_scatter): Update calls accordingly.
When testing a new scale before knowing the final offset type,
check whether the scale is supported for any signed or unsigned
offset type. Check whether the target supports the source and
target types of a conversion before deciding whether to look
through the conversion. Record the chosen offset_vectype.
* tree-vect-patterns.c (vect_get_gather_scatter_offset_type): Delete.
(vect_recog_gather_scatter_pattern): Get the scalar offset type
directly from the gs_info's offset_vectype instead. Pass a zero
of the result type to IFN_GATHER_LOAD and IFN_MASK_GATHER_LOAD.
* tree-vect-stmts.c (check_load_store_masking): Update call to
internal_gather_scatter_fn_supported_p, passing the offset vector
type recorded in the gs_info.
(vect_truncate_gather_scatter_offset): Update call to
vect_check_gather_scatter, leaving it to search for a valid
offset vector type.
(vect_use_strided_gather_scatters_p): Convert the offset to the
element type of the gs_info's offset_vectype.
(vect_get_gather_scatter_ops): Get the offset vector type directly
from the gs_info.
(vect_get_strided_load_store_ops): Likewise.
(vectorizable_load): Pass a zero of the result type to IFN_GATHER_LOAD
and IFN_MASK_GATHER_LOAD.
* config/aarch64/aarch64-sve.md (gather_load<mode>): Rename to...
(gather_load<mode><v_int_equiv>): ...this.
(mask_gather_load<mode>): Rename to...
(mask_gather_load<mode><v_int_equiv>): ...this.
(scatter_store<mode>): Rename to...
(scatter_store<mode><v_int_equiv>): ...this.
(mask_scatter_store<mode>): Rename to...
(mask_scatter_store<mode><v_int_equiv>): ...this.
From-SVN: r277949
|
|
* config/arc/arc.c (hwloop_optimize): Add missing space in string
literal.
* config/rx/rx.c (rx_print_operand): Likewise.
* tree-vect-data-refs.c (vect_analyze_data_refs): Likewise.
* tree-ssa-loop-ch.c (should_duplicate_loop_header_p): Likewise.
* ipa-sra.c (create_parameter_descriptors, process_scan_results):
Likewise.
* genemit.c (emit_c_code): Likewise.
* plugin.c (try_init_one_plugin): Likewise. Formatting fix.
cp/
* call.c (convert_arg_to_ellipsis): Add missing space in string
literal.
From-SVN: r277426
|
|
2019-10-21 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (get_vectype_for_scalar_type): Take a vec_info.
* tree-vect-stmts.c (get_vectype_for_scalar_type): Likewise.
(vect_prologue_cost_for_slp_op): Update call accordingly.
(vect_get_vec_def_for_operand, vect_get_gather_scatter_ops)
(vect_get_strided_load_store_ops, vectorizable_simd_clone_call)
(vect_supportable_shift, vect_is_simple_cond, vectorizable_comparison)
(get_mask_type_for_scalar_type): Likewise.
(vect_get_vector_types_for_stmt): Likewise.
* tree-vect-data-refs.c (vect_analyze_data_refs): Likewise.
* tree-vect-loop.c (vect_determine_vectorization_factor): Likewise.
(get_initial_def_for_reduction, build_vect_cond_expr): Likewise.
* tree-vect-patterns.c (vect_supportable_direct_optab_p): Likewise.
(vect_split_statement, vect_convert_input): Likewise.
(vect_recog_widen_op_pattern, vect_recog_pow_pattern): Likewise.
(vect_recog_over_widening_pattern, vect_recog_mulhs_pattern): Likewise.
(vect_recog_average_pattern, vect_recog_cast_forwprop_pattern)
(vect_recog_rotate_pattern, vect_recog_vector_vector_shift_pattern)
(vect_synth_mult_by_constant, vect_recog_mult_pattern): Likewise.
(vect_recog_divmod_pattern, vect_recog_mixed_size_cond_pattern)
(check_bool_pattern, adjust_bool_pattern_cast, adjust_bool_pattern)
(search_type_for_mask_1, vect_recog_bool_pattern): Likewise.
(vect_recog_mask_conversion_pattern): Likewise.
(vect_add_conversion_to_pattern): Likewise.
(vect_recog_gather_scatter_pattern): Likewise.
* tree-vect-slp.c (vect_build_slp_tree_2): Likewise.
(vect_analyze_slp_instance, vect_get_constant_vectors): Likewise.
From-SVN: r277227
|
|
2019-10-11 Richard Biener <rguenther@suse.de>
PR tree-optimization/92066
PR tree-optimization/92046
* tree-vect-data-refs.c (vect_enhance_data_refs_alignment):
Fix bogus cost model check.
From-SVN: r276876
|
|
affecting --params which are global.)
2019-10-10 Richard Biener <rguenther@suse.de>
PR middle-end/92046
* opts.c (finish_options): Do not influence global --params
from options that are adjustable per function.
* tree-vect-data-refs.c (vect_enhance_data_refs_alignment):
Apply --param adjustment based on active cost-model.
* tree-ssa-phiopt.c (cond_if_else_store_replacement): Disable
further store-sinking when vectorization or if-conversion
are not enabled.
From-SVN: r276807
|
|
non-bugs
gcc/c/ChangeLog:
PR c++/61339
* c-decl.c (xref_tag): Change class-key of PODs to struct and others
to class.
(field_decl_cmp): Same.
* c-parser.c (c_parser_struct_or_union_specifier): Same.
* c-tree.h: Same.
* gimple-parser.c (c_parser_gimple_compound_statement): Same.
gcc/c-family/ChangeLog:
PR c++/61339
* c-opts.c (handle_deferred_opts): : Change class-key of PODs to struct
and others to class.
* c-pretty-print.h: Same.
gcc/cp/ChangeLog:
PR c++/61339
* cp-tree.h: Change class-key of PODs to struct and others to class.
* search.c: Same.
* semantics.c (finalize_nrv_r): Same.
gcc/lto/ChangeLog:
PR c++/61339
* lto-common.c (lto_splay_tree_new): : Change class-key of PODs
to struct and others to class.
(mentions_vars_p): Same.
(register_resolution): Same.
(lto_register_var_decl_in_symtab): Same.
(lto_register_function_decl_in_symtab): Same.
(cmp_tree): Same.
(lto_read_decls): Same.
gcc/ChangeLog:
PR c++/61339
* auto-profile.c: Change class-key of PODs to struct and others
to class.
* basic-block.h: Same.
* bitmap.c (bitmap_alloc): Same.
* bitmap.h: Same.
* builtins.c (expand_builtin_prefetch): Same.
(expand_builtin_interclass_mathfn): Same.
(expand_builtin_strlen): Same.
(expand_builtin_mempcpy_args): Same.
(expand_cmpstr): Same.
(expand_builtin___clear_cache): Same.
(expand_ifn_atomic_bit_test_and): Same.
(expand_builtin_thread_pointer): Same.
(expand_builtin_set_thread_pointer): Same.
* caller-save.c (setup_save_areas): Same.
(replace_reg_with_saved_mem): Same.
(insert_restore): Same.
(insert_save): Same.
(add_used_regs): Same.
* cfg.c (get_bb_copy): Same.
(set_loop_copy): Same.
* cfg.h: Same.
* cfganal.h: Same.
* cfgexpand.c (alloc_stack_frame_space): Same.
(add_stack_var): Same.
(add_stack_var_conflict): Same.
(add_scope_conflicts_1): Same.
(update_alias_info_with_stack_vars): Same.
(expand_used_vars): Same.
* cfghooks.c (redirect_edge_and_branch_force): Same.
(delete_basic_block): Same.
(split_edge): Same.
(make_forwarder_block): Same.
(force_nonfallthru): Same.
(duplicate_block): Same.
(lv_flush_pending_stmts): Same.
* cfghooks.h: Same.
* cfgloop.c (flow_loops_cfg_dump): Same.
(flow_loop_nested_p): Same.
(superloop_at_depth): Same.
(get_loop_latch_edges): Same.
(flow_loop_dump): Same.
(flow_loops_dump): Same.
(flow_loops_free): Same.
(flow_loop_nodes_find): Same.
(establish_preds): Same.
(flow_loop_tree_node_add): Same.
(flow_loop_tree_node_remove): Same.
(flow_loops_find): Same.
(find_subloop_latch_edge_by_profile): Same.
(find_subloop_latch_edge_by_ivs): Same.
(mfb_redirect_edges_in_set): Same.
(form_subloop): Same.
(merge_latch_edges): Same.
(disambiguate_multiple_latches): Same.
(disambiguate_loops_with_multiple_latches): Same.
(flow_bb_inside_loop_p): Same.
(glb_enum_p): Same.
(get_loop_body_with_size): Same.
(get_loop_body): Same.
(fill_sons_in_loop): Same.
(get_loop_body_in_dom_order): Same.
(get_loop_body_in_custom_order): Same.
(release_recorded_exits): Same.
(get_loop_exit_edges): Same.
(num_loop_branches): Same.
(remove_bb_from_loops): Same.
(find_common_loop): Same.
(delete_loop): Same.
(cancel_loop): Same.
(verify_loop_structure): Same.
(loop_preheader_edge): Same.
(loop_exit_edge_p): Same.
(single_exit): Same.
(loop_exits_to_bb_p): Same.
(loop_exits_from_bb_p): Same.
(get_loop_location): Same.
(record_niter_bound): Same.
(get_estimated_loop_iterations_int): Same.
(max_stmt_executions_int): Same.
(likely_max_stmt_executions_int): Same.
(get_estimated_loop_iterations): Same.
(get_max_loop_iterations): Same.
(get_max_loop_iterations_int): Same.
(get_likely_max_loop_iterations): Same.
* cfgloop.h (simple_loop_desc): Same.
(get_loop): Same.
(loop_depth): Same.
(loop_outer): Same.
(loop_iterator::next): Same.
(loop_outermost): Same.
* cfgloopanal.c (mark_irreducible_loops): Same.
(num_loop_insns): Same.
(average_num_loop_insns): Same.
(expected_loop_iterations_unbounded): Same.
(expected_loop_iterations): Same.
(mark_loop_exit_edges): Same.
(single_likely_exit): Same.
* cfgloopmanip.c (fix_bb_placement): Same.
(fix_bb_placements): Same.
(remove_path): Same.
(place_new_loop): Same.
(add_loop): Same.
(scale_loop_frequencies): Same.
(scale_loop_profile): Same.
(create_empty_if_region_on_edge): Same.
(create_empty_loop_on_edge): Same.
(loopify): Same.
(unloop): Same.
(fix_loop_placements): Same.
(copy_loop_info): Same.
(duplicate_loop): Same.
(duplicate_subloops): Same.
(loop_redirect_edge): Same.
(can_duplicate_loop_p): Same.
(duplicate_loop_to_header_edge): Same.
(mfb_keep_just): Same.
(has_preds_from_loop): Same.
(create_preheader): Same.
(create_preheaders): Same.
(lv_adjust_loop_entry_edge): Same.
(loop_version): Same.
* cfgloopmanip.h: Same.
* cgraph.h: Same.
* cgraphbuild.c: Same.
* combine.c (make_extraction): Same.
* config/i386/i386-features.c: Same.
* config/i386/i386-features.h: Same.
* config/i386/i386.c (ix86_emit_outlined_ms2sysv_save): Same.
(ix86_emit_outlined_ms2sysv_restore): Same.
(ix86_noce_conversion_profitable_p): Same.
(ix86_init_cost): Same.
(ix86_simd_clone_usable): Same.
* configure.ac: Same.
* coretypes.h: Same.
* data-streamer-in.c (string_for_index): Same.
(streamer_read_indexed_string): Same.
(streamer_read_string): Same.
(bp_unpack_indexed_string): Same.
(bp_unpack_string): Same.
(streamer_read_uhwi): Same.
(streamer_read_hwi): Same.
(streamer_read_gcov_count): Same.
(streamer_read_wide_int): Same.
* data-streamer.h (streamer_write_bitpack): Same.
(bp_unpack_value): Same.
(streamer_write_char_stream): Same.
(streamer_write_hwi_in_range): Same.
(streamer_write_record_start): Same.
* ddg.c (create_ddg_dep_from_intra_loop_link): Same.
(add_cross_iteration_register_deps): Same.
(build_intra_loop_deps): Same.
* df-core.c (df_analyze): Same.
(loop_post_order_compute): Same.
(loop_inverted_post_order_compute): Same.
* df-problems.c (df_rd_alloc): Same.
(df_rd_simulate_one_insn): Same.
(df_rd_local_compute): Same.
(df_rd_init_solution): Same.
(df_rd_confluence_n): Same.
(df_rd_transfer_function): Same.
(df_rd_free): Same.
(df_rd_dump_defs_set): Same.
(df_rd_top_dump): Same.
(df_lr_alloc): Same.
(df_lr_reset): Same.
(df_lr_local_compute): Same.
(df_lr_init): Same.
(df_lr_confluence_n): Same.
(df_lr_free): Same.
(df_lr_top_dump): Same.
(df_lr_verify_transfer_functions): Same.
(df_live_alloc): Same.
(df_live_reset): Same.
(df_live_init): Same.
(df_live_confluence_n): Same.
(df_live_finalize): Same.
(df_live_free): Same.
(df_live_top_dump): Same.
(df_live_verify_transfer_functions): Same.
(df_mir_alloc): Same.
(df_mir_reset): Same.
(df_mir_init): Same.
(df_mir_confluence_n): Same.
(df_mir_free): Same.
(df_mir_top_dump): Same.
(df_word_lr_alloc): Same.
(df_word_lr_reset): Same.
(df_word_lr_init): Same.
(df_word_lr_confluence_n): Same.
(df_word_lr_free): Same.
(df_word_lr_top_dump): Same.
(df_md_alloc): Same.
(df_md_simulate_one_insn): Same.
(df_md_reset): Same.
(df_md_init): Same.
(df_md_free): Same.
(df_md_top_dump): Same.
* df-scan.c (df_insn_delete): Same.
(df_insn_rescan): Same.
(df_notes_rescan): Same.
(df_sort_and_compress_mws): Same.
(df_install_mws): Same.
(df_refs_add_to_chains): Same.
(df_ref_create_structure): Same.
(df_ref_record): Same.
(df_def_record_1): Same.
(df_find_hard_reg_defs): Same.
(df_uses_record): Same.
(df_get_conditional_uses): Same.
(df_get_call_refs): Same.
(df_recompute_luids): Same.
(df_get_entry_block_def_set): Same.
(df_entry_block_defs_collect): Same.
(df_get_exit_block_use_set): Same.
(df_exit_block_uses_collect): Same.
(df_mws_verify): Same.
(df_bb_verify): Same.
* df.h (df_scan_get_bb_info): Same.
* doc/tm.texi: Same.
* dse.c (record_store): Same.
* dumpfile.h: Same.
* emit-rtl.c (const_fixed_hasher::equal): Same.
(set_mem_attributes_minus_bitpos): Same.
(change_address): Same.
(adjust_address_1): Same.
(offset_address): Same.
* emit-rtl.h: Same.
* except.c (dw2_build_landing_pads): Same.
(sjlj_emit_dispatch_table): Same.
* explow.c (allocate_dynamic_stack_space): Same.
(emit_stack_probe): Same.
(probe_stack_range): Same.
* expmed.c (store_bit_field_using_insv): Same.
(store_bit_field_1): Same.
(store_integral_bit_field): Same.
(extract_bit_field_using_extv): Same.
(extract_bit_field_1): Same.
(emit_cstore): Same.
* expr.c (emit_block_move_via_cpymem): Same.
(expand_cmpstrn_or_cmpmem): Same.
(set_storage_via_setmem): Same.
(emit_single_push_insn_1): Same.
(expand_assignment): Same.
(store_constructor): Same.
(expand_expr_real_2): Same.
(expand_expr_real_1): Same.
(try_casesi): Same.
* flags.h: Same.
* function.c (try_fit_stack_local): Same.
(assign_stack_local_1): Same.
(assign_stack_local): Same.
(cut_slot_from_list): Same.
(insert_slot_to_list): Same.
(max_slot_level): Same.
(move_slot_to_level): Same.
(temp_address_hasher::equal): Same.
(remove_unused_temp_slot_addresses): Same.
(assign_temp): Same.
(combine_temp_slots): Same.
(update_temp_slot_address): Same.
(preserve_temp_slots): Same.
* function.h: Same.
* fwprop.c: Same.
* gcc-rich-location.h: Same.
* gcov.c: Same.
* genattrtab.c (check_attr_test): Same.
(check_attr_value): Same.
(convert_set_attr_alternative): Same.
(convert_set_attr): Same.
(check_defs): Same.
(copy_boolean): Same.
(get_attr_value): Same.
(expand_delays): Same.
(make_length_attrs): Same.
(min_fn): Same.
(make_alternative_compare): Same.
(simplify_test_exp): Same.
(tests_attr_p): Same.
(get_attr_order): Same.
(clear_struct_flag): Same.
(gen_attr): Same.
(compares_alternatives_p): Same.
(gen_insn): Same.
(gen_delay): Same.
(find_attrs_to_cache): Same.
(write_test_expr): Same.
(walk_attr_value): Same.
(write_attr_get): Same.
(eliminate_known_true): Same.
(write_insn_cases): Same.
(write_attr_case): Same.
(write_attr_valueq): Same.
(write_attr_value): Same.
(write_dummy_eligible_delay): Same.
(next_comma_elt): Same.
(find_attr): Same.
(make_internal_attr): Same.
(copy_rtx_unchanging): Same.
(gen_insn_reserv): Same.
(check_tune_attr): Same.
(make_automaton_attrs): Same.
(handle_arg): Same.
* genextract.c (gen_insn): Same.
(VEC_char_to_string): Same.
* genmatch.c (print_operand): Same.
(lower): Same.
(parser::parse_operation): Same.
(parser::parse_capture): Same.
(parser::parse_c_expr): Same.
(parser::parse_simplify): Same.
(main): Same.
* genoutput.c (output_operand_data): Same.
(output_get_insn_name): Same.
(compare_operands): Same.
(place_operands): Same.
(process_template): Same.
(validate_insn_alternatives): Same.
(validate_insn_operands): Same.
(gen_expand): Same.
(note_constraint): Same.
* genpreds.c (write_one_predicate_function): Same.
(add_constraint): Same.
(process_define_register_constraint): Same.
(write_lookup_constraint_1): Same.
(write_lookup_constraint_array): Same.
(write_insn_constraint_len): Same.
(write_reg_class_for_constraint_1): Same.
(write_constraint_satisfied_p_array): Same.
* genrecog.c (optimize_subroutine_group): Same.
* gensupport.c (process_define_predicate): Same.
(queue_pattern): Same.
(remove_from_queue): Same.
(process_rtx): Same.
(is_predicable): Same.
(change_subst_attribute): Same.
(subst_pattern_match): Same.
(alter_constraints): Same.
(alter_attrs_for_insn): Same.
(shift_output_template): Same.
(alter_output_for_subst_insn): Same.
(process_one_cond_exec): Same.
(subst_dup): Same.
(process_define_cond_exec): Same.
(mnemonic_htab_callback): Same.
(gen_mnemonic_attr): Same.
(read_md_rtx): Same.
* ggc-page.c: Same.
* gimple-loop-interchange.cc (dump_reduction): Same.
(dump_induction): Same.
(loop_cand::~loop_cand): Same.
(free_data_refs_with_aux): Same.
(tree_loop_interchange::interchange_loops): Same.
(tree_loop_interchange::map_inductions_to_loop): Same.
(tree_loop_interchange::move_code_to_inner_loop): Same.
(compute_access_stride): Same.
(compute_access_strides): Same.
(proper_loop_form_for_interchange): Same.
(tree_loop_interchange_compute_ddrs): Same.
(prune_datarefs_not_in_loop): Same.
(prepare_data_references): Same.
(pass_linterchange::execute): Same.
* gimple-loop-jam.c (bb_prevents_fusion_p): Same.
(unroll_jam_possible_p): Same.
(fuse_loops): Same.
(adjust_unroll_factor): Same.
(tree_loop_unroll_and_jam): Same.
* gimple-loop-versioning.cc (loop_versioning::~loop_versioning): Same.
(loop_versioning::expensive_stmt_p): Same.
(loop_versioning::version_for_unity): Same.
(loop_versioning::dump_inner_likelihood): Same.
(loop_versioning::find_per_loop_multiplication): Same.
(loop_versioning::analyze_term_using_scevs): Same.
(loop_versioning::record_address_fragment): Same.
(loop_versioning::analyze_expr): Same.
(loop_versioning::analyze_blocks): Same.
(loop_versioning::prune_conditions): Same.
(loop_versioning::merge_loop_info): Same.
(loop_versioning::add_loop_to_queue): Same.
(loop_versioning::decide_whether_loop_is_versionable): Same.
(loop_versioning::make_versioning_decisions): Same.
(loop_versioning::implement_versioning_decisions): Same.
* gimple-ssa-evrp-analyze.c
(evrp_range_analyzer::record_ranges_from_phis): Same.
* gimple-ssa-store-merging.c (split_store::split_store): Same.
(count_multiple_uses): Same.
(split_group): Same.
(imm_store_chain_info::output_merged_store): Same.
(pass_store_merging::process_store): Same.
* gimple-ssa-strength-reduction.c (slsr_process_phi): Same.
* gimple-ssa-warn-alloca.c (adjusted_warn_limit): Same.
(is_max): Same.
(alloca_call_type): Same.
(pass_walloca::execute): Same.
* gimple-streamer-in.c (input_phi): Same.
(input_gimple_stmt): Same.
* gimple-streamer.h: Same.
* godump.c (go_force_record_alignment): Same.
(go_format_type): Same.
(go_output_type): Same.
(go_output_fndecl): Same.
(go_output_typedef): Same.
(keyword_hash_init): Same.
(find_dummy_types): Same.
* graph.c (draw_cfg_nodes_no_loops): Same.
(draw_cfg_nodes_for_loop): Same.
* hard-reg-set.h (hard_reg_set_iter_next): Same.
* hsa-brig.c: Same.
* hsa-common.h (hsa_internal_fn_hasher::equal): Same.
* hsa-dump.c (dump_hsa_cfun): Same.
* hsa-gen.c (gen_function_def_parameters): Same.
* hsa-regalloc.c (dump_hsa_cfun_regalloc): Same.
* input.c (dump_line_table_statistics): Same.
(test_lexer): Same.
* input.h: Same.
* internal-fn.c (get_multi_vector_move): Same.
(expand_load_lanes_optab_fn): Same.
(expand_GOMP_SIMT_ENTER_ALLOC): Same.
(expand_GOMP_SIMT_EXIT): Same.
(expand_GOMP_SIMT_LAST_LANE): Same.
(expand_GOMP_SIMT_ORDERED_PRED): Same.
(expand_GOMP_SIMT_VOTE_ANY): Same.
(expand_GOMP_SIMT_XCHG_BFLY): Same.
(expand_GOMP_SIMT_XCHG_IDX): Same.
(expand_addsub_overflow): Same.
(expand_neg_overflow): Same.
(expand_mul_overflow): Same.
(expand_call_mem_ref): Same.
(expand_mask_load_optab_fn): Same.
(expand_scatter_store_optab_fn): Same.
(expand_gather_load_optab_fn): Same.
* ipa-cp.c (ipa_get_parm_lattices): Same.
(print_all_lattices): Same.
(ignore_edge_p): Same.
(build_toporder_info): Same.
(free_toporder_info): Same.
(push_node_to_stack): Same.
(ipcp_lattice<valtype>::set_contains_variable): Same.
(set_agg_lats_to_bottom): Same.
(ipcp_bits_lattice::meet_with): Same.
(set_single_call_flag): Same.
(initialize_node_lattices): Same.
(ipa_get_jf_ancestor_result): Same.
(ipcp_verify_propagated_values): Same.
(propagate_scalar_across_jump_function): Same.
(propagate_context_across_jump_function): Same.
(propagate_bits_across_jump_function): Same.
(ipa_vr_operation_and_type_effects): Same.
(propagate_vr_across_jump_function): Same.
(set_check_aggs_by_ref): Same.
(set_chain_of_aglats_contains_variable): Same.
(merge_aggregate_lattices): Same.
(agg_pass_through_permissible_p): Same.
(propagate_aggs_across_jump_function): Same.
(call_passes_through_thunk_p): Same.
(propagate_constants_across_call): Same.
(devirtualization_time_bonus): Same.
(good_cloning_opportunity_p): Same.
(context_independent_aggregate_values): Same.
(gather_context_independent_values): Same.
(perform_estimation_of_a_value): Same.
(estimate_local_effects): Same.
(value_topo_info<valtype>::add_val): Same.
(add_all_node_vals_to_toposort): Same.
(value_topo_info<valtype>::propagate_effects): Same.
(ipcp_propagate_stage): Same.
(ipcp_discover_new_direct_edges): Same.
(same_node_or_its_all_contexts_clone_p): Same.
(cgraph_edge_brings_value_p): Same.
(gather_edges_for_value): Same.
(create_specialized_node): Same.
(find_more_scalar_values_for_callers_subset): Same.
(find_more_contexts_for_caller_subset): Same.
(copy_plats_to_inter): Same.
(intersect_aggregates_with_edge): Same.
(find_aggregate_values_for_callers_subset): Same.
(cgraph_edge_brings_all_agg_vals_for_node): Same.
(decide_about_value): Same.
(decide_whether_version_node): Same.
(spread_undeadness): Same.
(identify_dead_nodes): Same.
(ipcp_store_vr_results): Same.
* ipa-devirt.c (final_warning_record::grow_type_warnings): Same.
* ipa-fnsummary.c (ipa_fn_summary::account_size_time): Same.
(redirect_to_unreachable): Same.
(edge_set_predicate): Same.
(evaluate_conditions_for_known_args): Same.
(evaluate_properties_for_edge): Same.
(ipa_fn_summary_t::duplicate): Same.
(ipa_call_summary_t::duplicate): Same.
(dump_ipa_call_summary): Same.
(ipa_dump_fn_summary): Same.
(eliminated_by_inlining_prob): Same.
(set_cond_stmt_execution_predicate): Same.
(set_switch_stmt_execution_predicate): Same.
(compute_bb_predicates): Same.
(will_be_nonconstant_expr_predicate): Same.
(phi_result_unknown_predicate): Same.
(analyze_function_body): Same.
(compute_fn_summary): Same.
(estimate_edge_devirt_benefit): Same.
(estimate_edge_size_and_time): Same.
(estimate_calls_size_and_time): Same.
(estimate_node_size_and_time): Same.
(remap_edge_change_prob): Same.
(remap_edge_summaries): Same.
(ipa_merge_fn_summary_after_inlining): Same.
(ipa_fn_summary_generate): Same.
(inline_read_section): Same.
(ipa_fn_summary_read): Same.
(ipa_fn_summary_write): Same.
* ipa-fnsummary.h: Same.
* ipa-hsa.c (ipa_hsa_read_section): Same.
* ipa-icf-gimple.c (func_checker::compare_loops): Same.
* ipa-icf.c (sem_function::param_used_p): Same.
* ipa-inline-analysis.c (do_estimate_edge_time): Same.
* ipa-inline.c (edge_badness): Same.
(inline_small_functions): Same.
* ipa-polymorphic-call.c
(ipa_polymorphic_call_context::stream_out): Same.
* ipa-predicate.c (predicate::remap_after_duplication): Same.
(predicate::remap_after_inlining): Same.
(predicate::stream_out): Same.
* ipa-predicate.h: Same.
* ipa-profile.c (ipa_profile_read_summary): Same.
* ipa-prop.c (ipa_get_param_decl_index_1): Same.
(count_formal_params): Same.
(ipa_dump_param): Same.
(ipa_alloc_node_params): Same.
(ipa_print_node_jump_functions_for_edge): Same.
(ipa_print_node_jump_functions): Same.
(ipa_load_from_parm_agg): Same.
(get_ancestor_addr_info): Same.
(ipa_compute_jump_functions_for_edge): Same.
(ipa_analyze_virtual_call_uses): Same.
(ipa_analyze_stmt_uses): Same.
(ipa_analyze_params_uses_in_bb): Same.
(update_jump_functions_after_inlining): Same.
(try_decrement_rdesc_refcount): Same.
(ipa_impossible_devirt_target): Same.
(update_indirect_edges_after_inlining): Same.
(combine_controlled_uses_counters): Same.
(ipa_edge_args_sum_t::duplicate): Same.
(ipa_write_jump_function): Same.
(ipa_write_indirect_edge_info): Same.
(ipa_write_node_info): Same.
(ipa_read_edge_info): Same.
(ipa_prop_read_section): Same.
(read_replacements_section): Same.
* ipa-prop.h (ipa_get_param_count): Same.
(ipa_get_param): Same.
(ipa_get_type): Same.
(ipa_get_param_move_cost): Same.
(ipa_set_param_used): Same.
(ipa_get_controlled_uses): Same.
(ipa_set_controlled_uses): Same.
(ipa_get_cs_argument_count): Same.
* ipa-pure-const.c (analyze_function): Same.
(pure_const_read_summary): Same.
* ipa-ref.h: Same.
* ipa-reference.c (ipa_reference_read_optimization_summary): Same.
* ipa-split.c (test_nonssa_use): Same.
(dump_split_point): Same.
(dominated_by_forbidden): Same.
(split_part_set_ssa_name_p): Same.
(find_split_points): Same.
* ira-build.c (finish_loop_tree_nodes): Same.
(low_pressure_loop_node_p): Same.
* ira-color.c (ira_reuse_stack_slot): Same.
* ira-int.h: Same.
* ira.c (setup_reg_equiv): Same.
(print_insn_chain): Same.
(ira): Same.
* loop-doloop.c (doloop_condition_get): Same.
(add_test): Same.
(record_reg_sets): Same.
(doloop_optimize): Same.
* loop-init.c (loop_optimizer_init): Same.
(fix_loop_structure): Same.
* loop-invariant.c (merge_identical_invariants): Same.
(compute_always_reached): Same.
(find_exits): Same.
(may_assign_reg_p): Same.
(find_invariants_bb): Same.
(find_invariants_body): Same.
(replace_uses): Same.
(can_move_invariant_reg): Same.
(free_inv_motion_data): Same.
(move_single_loop_invariants): Same.
(change_pressure): Same.
(mark_ref_regs): Same.
(calculate_loop_reg_pressure): Same.
* loop-iv.c (biv_entry_hasher::equal): Same.
(iv_extend_to_rtx_code): Same.
(check_iv_ref_table_size): Same.
(clear_iv_info): Same.
(latch_dominating_def): Same.
(iv_get_reaching_def): Same.
(iv_constant): Same.
(iv_subreg): Same.
(iv_extend): Same.
(iv_neg): Same.
(iv_add): Same.
(iv_mult): Same.
(get_biv_step): Same.
(record_iv): Same.
(analyzed_for_bivness_p): Same.
(record_biv): Same.
(iv_analyze_biv): Same.
(iv_analyze_expr): Same.
(iv_analyze_def): Same.
(iv_analyze_op): Same.
(iv_analyze): Same.
(iv_analyze_result): Same.
(biv_p): Same.
(eliminate_implied_conditions): Same.
(simplify_using_initial_values): Same.
(shorten_into_mode): Same.
(canonicalize_iv_subregs): Same.
(determine_max_iter): Same.
(check_simple_exit): Same.
(find_simple_exit): Same.
(get_simple_loop_desc): Same.
* loop-unroll.c (report_unroll): Same.
(decide_unrolling): Same.
(unroll_loops): Same.
(loop_exit_at_end_p): Same.
(decide_unroll_constant_iterations): Same.
(unroll_loop_constant_iterations): Same.
(compare_and_jump_seq): Same.
(unroll_loop_runtime_iterations): Same.
(decide_unroll_stupid): Same.
(unroll_loop_stupid): Same.
(referenced_in_one_insn_in_loop_p): Same.
(reset_debug_uses_in_loop): Same.
(analyze_iv_to_split_insn): Same.
* lra-eliminations.c (lra_debug_elim_table): Same.
(setup_can_eliminate): Same.
(form_sum): Same.
(lra_get_elimination_hard_regno): Same.
(lra_eliminate_regs_1): Same.
(eliminate_regs_in_insn): Same.
(update_reg_eliminate): Same.
(init_elimination): Same.
(lra_eliminate): Same.
* lra-int.h: Same.
* lra-lives.c (initiate_live_solver): Same.
* lra-remat.c (create_remat_bb_data): Same.
* lra-spills.c (lra_spill): Same.
* lra.c (lra_set_insn_recog_data): Same.
(lra_set_used_insn_alternative_by_uid): Same.
(init_reg_info): Same.
(expand_reg_info): Same.
* lto-cgraph.c (output_symtab): Same.
(read_identifier): Same.
(get_alias_symbol): Same.
(input_node): Same.
(input_varpool_node): Same.
(input_ref): Same.
(input_edge): Same.
(input_cgraph_1): Same.
(input_refs): Same.
(input_symtab): Same.
(input_offload_tables): Same.
(output_cgraph_opt_summary): Same.
(input_edge_opt_summary): Same.
(input_cgraph_opt_section): Same.
* lto-section-in.c (lto_free_raw_section_data): Same.
(lto_create_simple_input_block): Same.
(lto_free_function_in_decl_state_for_node): Same.
* lto-streamer-in.c (lto_tag_check_set): Same.
(lto_location_cache::revert_location_cache): Same.
(lto_location_cache::input_location): Same.
(lto_input_location): Same.
(stream_input_location_now): Same.
(lto_input_tree_ref): Same.
(lto_input_eh_catch_list): Same.
(input_eh_region): Same.
(lto_init_eh): Same.
(make_new_block): Same.
(input_cfg): Same.
(fixup_call_stmt_edges): Same.
(input_struct_function_base): Same.
(input_function): Same.
(lto_read_body_or_constructor): Same.
(lto_read_tree_1): Same.
(lto_read_tree): Same.
(lto_input_scc): Same.
(lto_input_tree_1): Same.
(lto_input_toplevel_asms): Same.
(lto_input_mode_table): Same.
(lto_reader_init): Same.
(lto_data_in_create): Same.
* lto-streamer-out.c (output_cfg): Same.
* lto-streamer.h: Same.
* modulo-sched.c (duplicate_insns_of_cycles): Same.
(generate_prolog_epilog): Same.
(mark_loop_unsched): Same.
(dump_insn_location): Same.
(loop_canon_p): Same.
(sms_schedule): Same.
* omp-expand.c (expand_omp_for_ordered_loops): Same.
(expand_omp_for_generic): Same.
(expand_omp_for_static_nochunk): Same.
(expand_omp_for_static_chunk): Same.
(expand_omp_simd): Same.
(expand_omp_taskloop_for_inner): Same.
(expand_oacc_for): Same.
(expand_omp_atomic_pipeline): Same.
(mark_loops_in_oacc_kernels_region): Same.
* omp-offload.c (oacc_xform_loop): Same.
* omp-simd-clone.c (simd_clone_adjust): Same.
* optabs-query.c (get_traditional_extraction_insn): Same.
* optabs.c (expand_vector_broadcast): Same.
(expand_binop_directly): Same.
(expand_twoval_unop): Same.
(expand_twoval_binop): Same.
(expand_unop_direct): Same.
(emit_indirect_jump): Same.
(emit_conditional_move): Same.
(emit_conditional_neg_or_complement): Same.
(emit_conditional_add): Same.
(vector_compare_rtx): Same.
(expand_vec_perm_1): Same.
(expand_vec_perm_const): Same.
(expand_vec_cond_expr): Same.
(expand_vec_series_expr): Same.
(maybe_emit_atomic_exchange): Same.
(maybe_emit_sync_lock_test_and_set): Same.
(expand_atomic_compare_and_swap): Same.
(expand_atomic_load): Same.
(expand_atomic_store): Same.
(maybe_emit_op): Same.
(valid_multiword_target_p): Same.
(create_integer_operand): Same.
(maybe_legitimize_operand_same_code): Same.
(maybe_legitimize_operand): Same.
(create_convert_operand_from_type): Same.
(can_reuse_operands_p): Same.
(maybe_legitimize_operands): Same.
(maybe_gen_insn): Same.
(maybe_expand_insn): Same.
(maybe_expand_jump_insn): Same.
(expand_insn): Same.
* optabs.h (create_expand_operand): Same.
(create_fixed_operand): Same.
(create_output_operand): Same.
(create_input_operand): Same.
(create_convert_operand_to): Same.
(create_convert_operand_from): Same.
* optinfo.h: Same.
* poly-int.h: Same.
* predict.c (optimize_insn_for_speed_p): Same.
(optimize_loop_for_size_p): Same.
(optimize_loop_for_speed_p): Same.
(optimize_loop_nest_for_speed_p): Same.
(get_base_value): Same.
(predicted_by_loop_heuristics_p): Same.
(predict_extra_loop_exits): Same.
(predict_loops): Same.
(predict_paths_for_bb): Same.
(predict_paths_leading_to): Same.
(propagate_freq): Same.
(pass_profile::execute): Same.
* predict.h: Same.
* profile-count.c (profile_count::differs_from_p): Same.
(profile_probability::differs_lot_from_p): Same.
* profile-count.h: Same.
* profile.c (branch_prob): Same.
* regrename.c (free_chain_data): Same.
(mark_conflict): Same.
(create_new_chain): Same.
(merge_overlapping_regs): Same.
(init_rename_info): Same.
(merge_chains): Same.
(regrename_analyze): Same.
(regrename_do_replace): Same.
(scan_rtx_reg): Same.
(record_out_operands): Same.
(build_def_use): Same.
* regrename.h: Same.
* reload.h: Same.
* reload1.c (init_reload): Same.
(maybe_fix_stack_asms): Same.
(copy_reloads): Same.
(count_pseudo): Same.
(count_spilled_pseudo): Same.
(find_reg): Same.
(find_reload_regs): Same.
(select_reload_regs): Same.
(spill_hard_reg): Same.
(fixup_eh_region_note): Same.
(set_reload_reg): Same.
(allocate_reload_reg): Same.
(compute_reload_subreg_offset): Same.
(reload_adjust_reg_for_icode): Same.
(emit_input_reload_insns): Same.
(emit_output_reload_insns): Same.
(do_input_reload): Same.
(inherit_piecemeal_p): Same.
* rtl.h: Same.
* sanopt.c (maybe_get_dominating_check): Same.
(maybe_optimize_ubsan_ptr_ifn): Same.
(can_remove_asan_check): Same.
(maybe_optimize_asan_check_ifn): Same.
(sanopt_optimize_walker): Same.
* sched-deps.c (add_dependence_list): Same.
(chain_to_prev_insn): Same.
(add_insn_mem_dependence): Same.
(create_insn_reg_set): Same.
(maybe_extend_reg_info_p): Same.
(sched_analyze_reg): Same.
(sched_analyze_1): Same.
(get_implicit_reg_pending_clobbers): Same.
(chain_to_prev_insn_p): Same.
(deps_analyze_insn): Same.
(deps_start_bb): Same.
(sched_free_deps): Same.
(init_deps): Same.
(init_deps_reg_last): Same.
(free_deps): Same.
* sched-ebb.c: Same.
* sched-int.h: Same.
* sched-rgn.c (add_branch_dependences): Same.
(concat_insn_mem_list): Same.
(deps_join): Same.
(sched_rgn_compute_dependencies): Same.
* sel-sched-ir.c (reset_target_context): Same.
(copy_deps_context): Same.
(init_id_from_df): Same.
(has_dependence_p): Same.
(change_loops_latches): Same.
(bb_top_order_comparator): Same.
(make_region_from_loop_preheader): Same.
(sel_init_pipelining): Same.
(get_loop_nest_for_rgn): Same.
(make_regions_from_the_rest): Same.
(sel_is_loop_preheader_p): Same.
* sel-sched-ir.h (inner_loop_header_p): Same.
(get_all_loop_exits): Same.
* selftest.h: Same.
* sese.c (sese_build_liveouts): Same.
(sese_insert_phis_for_liveouts): Same.
* sese.h (defined_in_sese_p): Same.
* sreal.c (sreal::stream_out): Same.
* sreal.h: Same.
* streamer-hooks.h: Same.
* target-globals.c (save_target_globals): Same.
* target-globals.h: Same.
* target.def: Same.
* target.h: Same.
* targhooks.c (default_has_ifunc_p): Same.
(default_empty_mask_is_expensive): Same.
(default_init_cost): Same.
* targhooks.h: Same.
* toplev.c: Same.
* tree-affine.c (aff_combination_mult): Same.
(aff_combination_expand): Same.
(aff_combination_constant_multiple_p): Same.
* tree-affine.h: Same.
* tree-cfg.c (build_gimple_cfg): Same.
(replace_loop_annotate_in_block): Same.
(replace_uses_by): Same.
(remove_bb): Same.
(dump_cfg_stats): Same.
(gimple_duplicate_sese_region): Same.
(gimple_duplicate_sese_tail): Same.
(move_block_to_fn): Same.
(replace_block_vars_by_duplicates): Same.
(move_sese_region_to_fn): Same.
(print_loops_bb): Same.
(print_loop): Same.
(print_loops): Same.
(debug): Same.
(debug_loops): Same.
* tree-cfg.h: Same.
* tree-chrec.c (chrec_fold_plus_poly_poly): Same.
(chrec_fold_multiply_poly_poly): Same.
(chrec_evaluate): Same.
(chrec_component_in_loop_num): Same.
(reset_evolution_in_loop): Same.
(is_multivariate_chrec): Same.
(chrec_contains_symbols): Same.
(nb_vars_in_chrec): Same.
(chrec_convert_1): Same.
(chrec_convert_aggressive): Same.
* tree-chrec.h: Same.
* tree-core.h: Same.
* tree-data-ref.c (dump_data_dependence_relation): Same.
(canonicalize_base_object_address): Same.
(data_ref_compare_tree): Same.
(prune_runtime_alias_test_list): Same.
(get_segment_min_max): Same.
(create_intersect_range_checks): Same.
(conflict_fn_no_dependence): Same.
(object_address_invariant_in_loop_p): Same.
(analyze_ziv_subscript): Same.
(analyze_siv_subscript_cst_affine): Same.
(analyze_miv_subscript): Same.
(analyze_overlapping_iterations): Same.
(build_classic_dist_vector_1): Same.
(add_other_self_distances): Same.
(same_access_functions): Same.
(build_classic_dir_vector): Same.
(subscript_dependence_tester_1): Same.
(subscript_dependence_tester): Same.
(access_functions_are_affine_or_constant_p): Same.
(get_references_in_stmt): Same.
(loop_nest_has_data_refs): Same.
(graphite_find_data_references_in_stmt): Same.
(find_data_references_in_bb): Same.
(get_base_for_alignment): Same.
(find_loop_nest_1): Same.
(find_loop_nest): Same.
* tree-data-ref.h (dr_alignment): Same.
(ddr_dependence_level): Same.
* tree-if-conv.c (fold_build_cond_expr): Same.
(add_to_predicate_list): Same.
(add_to_dst_predicate_list): Same.
(phi_convertible_by_degenerating_args): Same.
(idx_within_array_bound): Same.
(all_preds_critical_p): Same.
(pred_blocks_visited_p): Same.
(predicate_bbs): Same.
(build_region): Same.
(if_convertible_loop_p_1): Same.
(is_cond_scalar_reduction): Same.
(predicate_scalar_phi): Same.
(remove_conditions_and_labels): Same.
(combine_blocks): Same.
(version_loop_for_if_conversion): Same.
(versionable_outer_loop_p): Same.
(ifcvt_local_dce): Same.
(tree_if_conversion): Same.
(pass_if_conversion::gate): Same.
* tree-if-conv.h: Same.
* tree-inline.c (maybe_move_debug_stmts_to_successors): Same.
* tree-loop-distribution.c (bb_top_order_cmp): Same.
(free_rdg): Same.
(stmt_has_scalar_dependences_outside_loop): Same.
(copy_loop_before): Same.
(create_bb_after_loop): Same.
(const_with_all_bytes_same): Same.
(generate_memset_builtin): Same.
(generate_memcpy_builtin): Same.
(destroy_loop): Same.
(build_rdg_partition_for_vertex): Same.
(compute_access_range): Same.
(data_ref_segment_size): Same.
(latch_dominated_by_data_ref): Same.
(compute_alias_check_pairs): Same.
(fuse_memset_builtins): Same.
(finalize_partitions): Same.
(find_seed_stmts_for_distribution): Same.
(prepare_perfect_loop_nest): Same.
* tree-parloops.c (lambda_transform_legal_p): Same.
(loop_parallel_p): Same.
(reduc_stmt_res): Same.
(add_field_for_name): Same.
(create_call_for_reduction_1): Same.
(replace_uses_in_bb_by): Same.
(transform_to_exit_first_loop_alt): Same.
(try_transform_to_exit_first_loop_alt): Same.
(transform_to_exit_first_loop): Same.
(num_phis): Same.
(gen_parallel_loop): Same.
(gather_scalar_reductions): Same.
(get_omp_data_i_param): Same.
(try_create_reduction_list): Same.
(oacc_entry_exit_single_gang): Same.
(parallelize_loops): Same.
* tree-pass.h: Same.
* tree-predcom.c (determine_offset): Same.
(last_always_executed_block): Same.
(split_data_refs_to_components): Same.
(suitable_component_p): Same.
(valid_initializer_p): Same.
(find_looparound_phi): Same.
(insert_looparound_copy): Same.
(add_looparound_copies): Same.
(determine_roots_comp): Same.
(predcom_tmp_var): Same.
(initialize_root_vars): Same.
(initialize_root_vars_store_elim_1): Same.
(initialize_root_vars_store_elim_2): Same.
(finalize_eliminated_stores): Same.
(initialize_root_vars_lm): Same.
(remove_stmt): Same.
(determine_unroll_factor): Same.
(execute_pred_commoning_cbck): Same.
(base_names_in_chain_on): Same.
(combine_chains): Same.
(pcom_stmt_dominates_stmt_p): Same.
(try_combine_chains): Same.
(prepare_initializers_chain_store_elim): Same.
(prepare_initializers_chain): Same.
(prepare_initializers): Same.
(prepare_finalizers_chain): Same.
(prepare_finalizers): Same.
(insert_init_seqs): Same.
* tree-scalar-evolution.c (loop_phi_node_p): Same.
(compute_overall_effect_of_inner_loop): Same.
(add_to_evolution_1): Same.
(add_to_evolution): Same.
(follow_ssa_edge_binary): Same.
(follow_ssa_edge_expr): Same.
(backedge_phi_arg_p): Same.
(follow_ssa_edge_in_condition_phi_branch): Same.
(follow_ssa_edge_in_condition_phi): Same.
(follow_ssa_edge_inner_loop_phi): Same.
(follow_ssa_edge): Same.
(analyze_evolution_in_loop): Same.
(analyze_initial_condition): Same.
(interpret_loop_phi): Same.
(interpret_condition_phi): Same.
(interpret_rhs_expr): Same.
(interpret_expr): Same.
(interpret_gimple_assign): Same.
(analyze_scalar_evolution_1): Same.
(analyze_scalar_evolution): Same.
(analyze_scalar_evolution_for_address_of): Same.
(get_instantiated_value_entry): Same.
(loop_closed_phi_def): Same.
(instantiate_scev_name): Same.
(instantiate_scev_poly): Same.
(instantiate_scev_binary): Same.
(instantiate_scev_convert): Same.
(instantiate_scev_not): Same.
(instantiate_scev_r): Same.
(instantiate_scev): Same.
(resolve_mixers): Same.
(initialize_scalar_evolutions_analyzer): Same.
(scev_reset_htab): Same.
(scev_reset): Same.
(derive_simple_iv_with_niters): Same.
(simple_iv_with_niters): Same.
(expression_expensive_p): Same.
(final_value_replacement_loop): Same.
* tree-scalar-evolution.h (block_before_loop): Same.
* tree-ssa-address.h: Same.
* tree-ssa-dce.c (find_obviously_necessary_stmts): Same.
* tree-ssa-dom.c (edge_info::record_simple_equiv): Same.
(record_edge_info): Same.
* tree-ssa-live.c (var_map_base_fini): Same.
(remove_unused_locals): Same.
* tree-ssa-live.h: Same.
* tree-ssa-loop-ch.c (should_duplicate_loop_header_p): Same.
(pass_ch_vect::execute): Same.
(pass_ch::process_loop_p): Same.
* tree-ssa-loop-im.c (mem_ref_hasher::hash): Same.
(movement_possibility): Same.
(outermost_invariant_loop): Same.
(stmt_cost): Same.
(determine_max_movement): Same.
(invariantness_dom_walker::before_dom_children): Same.
(move_computations): Same.
(may_move_till): Same.
(force_move_till_op): Same.
(force_move_till): Same.
(memref_free): Same.
(record_mem_ref_loc): Same.
(set_ref_stored_in_loop): Same.
(mark_ref_stored): Same.
(sort_bbs_in_loop_postorder_cmp): Same.
(sort_locs_in_loop_postorder_cmp): Same.
(analyze_memory_references): Same.
(mem_refs_may_alias_p): Same.
(find_ref_loc_in_loop_cmp): Same.
(rewrite_mem_ref_loc::operator): Same.
(first_mem_ref_loc_1::operator): Same.
(sm_set_flag_if_changed::operator): Same.
(execute_sm_if_changed_flag_set): Same.
(execute_sm): Same.
(hoist_memory_references): Same.
(ref_always_accessed::operator): Same.
(refs_independent_p): Same.
(record_dep_loop): Same.
(ref_indep_loop_p_1): Same.
(ref_indep_loop_p): Same.
(can_sm_ref_p): Same.
(find_refs_for_sm): Same.
(loop_suitable_for_sm): Same.
(store_motion_loop): Same.
(store_motion): Same.
(fill_always_executed_in): Same.
* tree-ssa-loop-ivcanon.c (constant_after_peeling): Same.
(estimated_unrolled_size): Same.
(loop_edge_to_cancel): Same.
(remove_exits_and_undefined_stmts): Same.
(remove_redundant_iv_tests): Same.
(unloop_loops): Same.
(estimated_peeled_sequence_size): Same.
(try_peel_loop): Same.
(canonicalize_loop_induction_variables): Same.
(canonicalize_induction_variables): Same.
* tree-ssa-loop-ivopts.c (iv_inv_expr_hasher::equal): Same.
(name_info): Same.
(stmt_after_inc_pos): Same.
(contains_abnormal_ssa_name_p): Same.
(niter_for_exit): Same.
(find_bivs): Same.
(mark_bivs): Same.
(find_givs_in_bb): Same.
(find_induction_variables): Same.
(find_interesting_uses_cond): Same.
(outermost_invariant_loop_for_expr): Same.
(idx_find_step): Same.
(add_candidate_1): Same.
(add_iv_candidate_derived_from_uses): Same.
(alloc_use_cost_map): Same.
(prepare_decl_rtl): Same.
(generic_predict_doloop_p): Same.
(computation_cost): Same.
(determine_common_wider_type): Same.
(get_computation_aff_1): Same.
(get_use_type): Same.
(determine_group_iv_cost_address): Same.
(iv_period): Same.
(difference_cannot_overflow_p): Same.
(may_eliminate_iv): Same.
(determine_set_costs): Same.
(cheaper_cost_pair): Same.
(compare_cost_pair): Same.
(iv_ca_cand_for_group): Same.
(iv_ca_recount_cost): Same.
(iv_ca_set_remove_invs): Same.
(iv_ca_set_no_cp): Same.
(iv_ca_set_add_invs): Same.
(iv_ca_set_cp): Same.
(iv_ca_add_group): Same.
(iv_ca_cost): Same.
(iv_ca_compare_deps): Same.
(iv_ca_delta_reverse): Same.
(iv_ca_delta_commit): Same.
(iv_ca_cand_used_p): Same.
(iv_ca_delta_free): Same.
(iv_ca_new): Same.
(iv_ca_free): Same.
(iv_ca_dump): Same.
(iv_ca_extend): Same.
(iv_ca_narrow): Same.
(iv_ca_prune): Same.
(cheaper_cost_with_cand): Same.
(iv_ca_replace): Same.
(try_add_cand_for): Same.
(get_initial_solution): Same.
(try_improve_iv_set): Same.
(find_optimal_iv_set_1): Same.
(create_new_iv): Same.
(rewrite_use_compare): Same.
(remove_unused_ivs): Same.
(determine_scaling_factor): Same.
* tree-ssa-loop-ivopts.h: Same.
* tree-ssa-loop-manip.c (create_iv): Same.
(compute_live_loop_exits): Same.
(add_exit_phi): Same.
(add_exit_phis): Same.
(find_uses_to_rename_use): Same.
(find_uses_to_rename_def): Same.
(find_uses_to_rename_in_loop): Same.
(rewrite_into_loop_closed_ssa): Same.
(check_loop_closed_ssa_bb): Same.
(split_loop_exit_edge): Same.
(ip_end_pos): Same.
(ip_normal_pos): Same.
(copy_phi_node_args): Same.
(gimple_duplicate_loop_to_header_edge): Same.
(can_unroll_loop_p): Same.
(determine_exit_conditions): Same.
(scale_dominated_blocks_in_loop): Same.
(niter_for_unrolled_loop): Same.
(tree_transform_and_unroll_loop): Same.
(rewrite_all_phi_nodes_with_iv): Same.
* tree-ssa-loop-manip.h: Same.
* tree-ssa-loop-niter.c (number_of_iterations_ne_max): Same.
(number_of_iterations_ne): Same.
(assert_no_overflow_lt): Same.
(assert_loop_rolls_lt): Same.
(number_of_iterations_lt): Same.
(adjust_cond_for_loop_until_wrap): Same.
(tree_simplify_using_condition): Same.
(simplify_using_initial_conditions): Same.
(simplify_using_outer_evolutions): Same.
(loop_only_exit_p): Same.
(ssa_defined_by_minus_one_stmt_p): Same.
(number_of_iterations_popcount): Same.
(number_of_iterations_exit): Same.
(find_loop_niter): Same.
(finite_loop_p): Same.
(chain_of_csts_start): Same.
(get_val_for): Same.
(loop_niter_by_eval): Same.
(derive_constant_upper_bound_ops): Same.
(do_warn_aggressive_loop_optimizations): Same.
(record_estimate): Same.
(get_cst_init_from_scev): Same.
(record_nonwrapping_iv): Same.
(idx_infer_loop_bounds): Same.
(infer_loop_bounds_from_ref): Same.
(infer_loop_bounds_from_array): Same.
(infer_loop_bounds_from_pointer_arith): Same.
(infer_loop_bounds_from_signedness): Same.
(bound_index): Same.
(discover_iteration_bound_by_body_walk): Same.
(maybe_lower_iteration_bound): Same.
(estimate_numbers_of_iterations): Same.
(estimated_loop_iterations): Same.
(estimated_loop_iterations_int): Same.
(max_loop_iterations): Same.
(max_loop_iterations_int): Same.
(likely_max_loop_iterations): Same.
(likely_max_loop_iterations_int): Same.
(estimated_stmt_executions_int): Same.
(max_stmt_executions): Same.
(likely_max_stmt_executions): Same.
(estimated_stmt_executions): Same.
(stmt_dominates_stmt_p): Same.
(nowrap_type_p): Same.
(loop_exits_before_overflow): Same.
(scev_var_range_cant_overflow): Same.
(scev_probably_wraps_p): Same.
(free_numbers_of_iterations_estimates): Same.
* tree-ssa-loop-niter.h: Same.
* tree-ssa-loop-prefetch.c (release_mem_refs): Same.
(idx_analyze_ref): Same.
(analyze_ref): Same.
(gather_memory_references_ref): Same.
(mark_nontemporal_store): Same.
(emit_mfence_after_loop): Same.
(may_use_storent_in_loop_p): Same.
(mark_nontemporal_stores): Same.
(should_unroll_loop_p): Same.
(volume_of_dist_vector): Same.
(add_subscript_strides): Same.
(self_reuse_distance): Same.
(insn_to_prefetch_ratio_too_small_p): Same.
* tree-ssa-loop-split.c (split_at_bb_p): Same.
(patch_loop_exit): Same.
(find_or_create_guard_phi): Same.
(easy_exit_values): Same.
(connect_loop_phis): Same.
(connect_loops): Same.
(compute_new_first_bound): Same.
(split_loop): Same.
(tree_ssa_split_loops): Same.
* tree-ssa-loop-unswitch.c (tree_ssa_unswitch_loops): Same.
(is_maybe_undefined): Same.
(tree_may_unswitch_on): Same.
(simplify_using_entry_checks): Same.
(tree_unswitch_single_loop): Same.
(tree_unswitch_loop): Same.
(tree_unswitch_outer_loop): Same.
(empty_bb_without_guard_p): Same.
(used_outside_loop_p): Same.
(get_vop_from_header): Same.
(hoist_guard): Same.
* tree-ssa-loop.c (gate_oacc_kernels): Same.
(get_lsm_tmp_name): Same.
* tree-ssa-loop.h: Same.
* tree-ssa-reassoc.c (add_repeat_to_ops_vec): Same.
(build_and_add_sum): Same.
(no_side_effect_bb): Same.
(get_ops): Same.
(linearize_expr): Same.
(should_break_up_subtract): Same.
(linearize_expr_tree): Same.
* tree-ssa-scopedtables.c: Same.
* tree-ssa-scopedtables.h: Same.
* tree-ssa-structalias.c (condense_visit): Same.
(label_visit): Same.
(dump_pred_graph): Same.
(perform_var_substitution): Same.
(move_complex_constraints): Same.
(remove_preds_and_fake_succs): Same.
* tree-ssa-threadupdate.c (dbds_continue_enumeration_p): Same.
(determine_bb_domination_status): Same.
(duplicate_thread_path): Same.
(thread_through_all_blocks): Same.
* tree-ssa-threadupdate.h: Same.
* tree-streamer-in.c (streamer_read_string_cst): Same.
(input_identifier): Same.
(unpack_ts_type_common_value_fields): Same.
(unpack_ts_block_value_fields): Same.
(unpack_ts_translation_unit_decl_value_fields): Same.
(unpack_ts_omp_clause_value_fields): Same.
(streamer_read_tree_bitfields): Same.
(streamer_alloc_tree): Same.
(lto_input_ts_common_tree_pointers): Same.
(lto_input_ts_vector_tree_pointers): Same.
(lto_input_ts_poly_tree_pointers): Same.
(lto_input_ts_complex_tree_pointers): Same.
(lto_input_ts_decl_minimal_tree_pointers): Same.
(lto_input_ts_decl_common_tree_pointers): Same.
(lto_input_ts_decl_non_common_tree_pointers): Same.
(lto_input_ts_decl_with_vis_tree_pointers): Same.
(lto_input_ts_field_decl_tree_pointers): Same.
(lto_input_ts_function_decl_tree_pointers): Same.
(lto_input_ts_type_common_tree_pointers): Same.
(lto_input_ts_type_non_common_tree_pointers): Same.
(lto_input_ts_list_tree_pointers): Same.
(lto_input_ts_vec_tree_pointers): Same.
(lto_input_ts_exp_tree_pointers): Same.
(lto_input_ts_block_tree_pointers): Same.
(lto_input_ts_binfo_tree_pointers): Same.
(lto_input_ts_constructor_tree_pointers): Same.
(lto_input_ts_omp_clause_tree_pointers): Same.
(streamer_read_tree_body): Same.
* tree-streamer.h: Same.
* tree-switch-conversion.c (bit_test_cluster::is_beneficial): Same.
* tree-vect-data-refs.c (vect_get_smallest_scalar_type): Same.
(vect_analyze_possibly_independent_ddr): Same.
(vect_analyze_data_ref_dependence): Same.
(vect_compute_data_ref_alignment): Same.
(vect_enhance_data_refs_alignment): Same.
(vect_analyze_data_ref_access): Same.
(vect_check_gather_scatter): Same.
(vect_find_stmt_data_reference): Same.
(vect_create_addr_base_for_vector_ref): Same.
(vect_setup_realignment): Same.
(vect_supportable_dr_alignment): Same.
* tree-vect-loop-manip.c (rename_variables_in_bb): Same.
(adjust_phi_and_debug_stmts): Same.
(vect_set_loop_mask): Same.
(add_preheader_seq): Same.
(vect_maybe_permute_loop_masks): Same.
(vect_set_loop_masks_directly): Same.
(vect_set_loop_condition_masked): Same.
(vect_set_loop_condition_unmasked): Same.
(slpeel_duplicate_current_defs_from_edges): Same.
(slpeel_add_loop_guard): Same.
(slpeel_can_duplicate_loop_p): Same.
(create_lcssa_for_virtual_phi): Same.
(iv_phi_p): Same.
(vect_update_ivs_after_vectorizer): Same.
(vect_gen_vector_loop_niters_mult_vf): Same.
(slpeel_update_phi_nodes_for_loops): Same.
(slpeel_update_phi_nodes_for_guard1): Same.
(find_guard_arg): Same.
(slpeel_update_phi_nodes_for_guard2): Same.
(slpeel_update_phi_nodes_for_lcssa): Same.
(vect_do_peeling): Same.
(vect_create_cond_for_alias_checks): Same.
(vect_loop_versioning): Same.
* tree-vect-loop.c (vect_determine_vf_for_stmt): Same.
(vect_inner_phi_in_double_reduction_p): Same.
(vect_analyze_scalar_cycles_1): Same.
(vect_fixup_scalar_cycles_with_patterns): Same.
(vect_get_loop_niters): Same.
(bb_in_loop_p): Same.
(vect_get_max_nscalars_per_iter): Same.
(vect_verify_full_masking): Same.
(vect_compute_single_scalar_iteration_cost): Same.
(vect_analyze_loop_form_1): Same.
(vect_analyze_loop_form): Same.
(vect_active_double_reduction_p): Same.
(vect_analyze_loop_operations): Same.
(neutral_op_for_slp_reduction): Same.
(vect_is_simple_reduction): Same.
(vect_model_reduction_cost): Same.
(get_initial_def_for_reduction): Same.
(get_initial_defs_for_reduction): Same.
(vect_create_epilog_for_reduction): Same.
(vectorize_fold_left_reduction): Same.
(vectorizable_reduction): Same.
(vectorizable_induction): Same.
(vectorizable_live_operation): Same.
(loop_niters_no_overflow): Same.
(vect_get_loop_mask): Same.
(vect_transform_loop_stmt): Same.
(vect_transform_loop): Same.
* tree-vect-patterns.c (vect_reassociating_reduction_p): Same.
(vect_determine_precisions): Same.
(vect_pattern_recog_1): Same.
* tree-vect-slp.c (vect_analyze_slp_instance): Same.
* tree-vect-stmts.c (stmt_vectype): Same.
(process_use): Same.
(vect_init_vector_1): Same.
(vect_truncate_gather_scatter_offset): Same.
(get_group_load_store_type): Same.
(vect_build_gather_load_calls): Same.
(vect_get_strided_load_store_ops): Same.
(vectorizable_simd_clone_call): Same.
(vectorizable_store): Same.
(permute_vec_elements): Same.
(vectorizable_load): Same.
(vect_transform_stmt): Same.
(supportable_widening_operation): Same.
* tree-vectorizer.c (vec_info::replace_stmt): Same.
(vec_info::free_stmt_vec_info): Same.
(vect_free_loop_info_assumptions): Same.
(vect_loop_vectorized_call): Same.
(set_uid_loop_bbs): Same.
(vectorize_loops): Same.
* tree-vectorizer.h (STMT_VINFO_BB_VINFO): Same.
* tree.c (add_tree_to_fld_list): Same.
(fld_type_variant_equal_p): Same.
(fld_decl_context): Same.
(fld_incomplete_type_of): Same.
(free_lang_data_in_binfo): Same.
(need_assembler_name_p): Same.
(find_decls_types_r): Same.
(get_eh_types_for_runtime): Same.
(find_decls_types_in_eh_region): Same.
(find_decls_types_in_node): Same.
(assign_assembler_name_if_needed): Same.
* value-prof.c (stream_out_histogram_value): Same.
* value-prof.h: Same.
* var-tracking.c (use_narrower_mode): Same.
(prepare_call_arguments): Same.
(vt_expand_loc_callback): Same.
(resolve_expansions_pending_recursion): Same.
(vt_expand_loc): Same.
* varasm.c (const_hash_1): Same.
(compare_constant): Same.
(tree_output_constant_def): Same.
(simplify_subtraction): Same.
(get_pool_constant): Same.
(output_constant_pool_2): Same.
(output_constant_pool_1): Same.
(mark_constants_in_pattern): Same.
(mark_constant_pool): Same.
(get_section_anchor): Same.
* vr-values.c (compare_range_with_value): Same.
(vr_values::extract_range_from_phi_node): Same.
* vr-values.h: Same.
* web.c (unionfind_union): Same.
* wide-int.h: Same.
From-SVN: r273311
|
|
tree-vect-loop.c:2415)
2019-07-09 Richard Biener <rguenther@suse.de>
PR tree-optimization/91114
* tree-vect-data-refs.c (vect_analyze_data_refs): Failure to
find a vector type isn't fatal.
* gcc.dg/vect/pr91114.c: New testcase.
From-SVN: r273306
|
|
tree-vect-loop.c:2416)
PR tree-optimization/91033
* tree-vectorizer.h (vect_mark_stmts_to_be_vectorized,
vect_analyze_data_refs): Add bool * arguments.
* tree-vect-data-refs.c (vect_analyze_data_refs): Add fatal argument,
if failure is due to scatter/gather, set *fatal to false if non-NULL.
* tree-vect-stmts.c (vect_mark_stmts_to_be_vectorized): Likewise.
* tree-vect-loop.c (vect_analyze_loop_2): Adjust
vect_mark_stmts_to_be_vectorized and vect_analyze_data_refs callers.
* tree-vect-slp.c (vect_slp_analyze_bb_1): Adjust
vect_analyze_data_refs caller.
* gcc.target/i386/pr91033.c: New test.
From-SVN: r272989
|
|
DR_OFFSET...
* tree-vect-data-refs.c (vect_find_stmt_data_reference): Handle
even zero DR_OFFSET, but DR_BASE_ADDRESS of POINTER_PLUS_EXPR
containing the offset as possible simd lane access. Look through
widening conversion. Move the
TREE_CODE (DR_INIT (newdr)) == INTEGER_CST test earlier and reindent.
* g++.dg/vect/simd-2.cc: Don't xfail, instead expect vectorization on
x86.
* g++.dg/vect/simd-5.cc: Likewise.
From-SVN: r272575
|
|
* omp-low.c (lower_rec_simd_input_clauses): Add rvar2 argument,
create another "omp scan inscan exclusive" array if
!ctx->scan_inclusive.
(lower_rec_input_clauses): Handle exclusive scan inscan reductions.
(lower_omp_scan): Likewise.
* tree-vectorizer.h (struct _stmt_vec_info): Use 3-bit instead of
2-bit bitfield for simd_lane_access_p member.
* tree-vect-data-refs.c (vect_analyze_data_refs): Also handle
aux == (void *)-4 as simd lane access.
* tree-vect-stmts.c (check_scan_store): Handle exclusive scan. Update
comment with permutations to show the canonical permutation order.
(vectorizable_scan_store): Handle exclusive scan.
(vectorizable_store): Call vectorizable_scan_store even for
STMT_VINFO_SIMD_LANE_ACCESS_P > 3.
* gcc.dg/vect/vect-simd-12.c: New test.
* gcc.dg/vect/vect-simd-13.c: New test.
* gcc.dg/vect/vect-simd-14.c: New test.
* gcc.dg/vect/vect-simd-15.c: New test.
* gcc.target/i386/sse2-vect-simd-12.c: New test.
* gcc.target/i386/sse2-vect-simd-13.c: New test.
* gcc.target/i386/sse2-vect-simd-14.c: New test.
* gcc.target/i386/sse2-vect-simd-15.c: New test.
* gcc.target/i386/avx2-vect-simd-12.c: New test.
* gcc.target/i386/avx2-vect-simd-13.c: New test.
* gcc.target/i386/avx2-vect-simd-14.c: New test.
* gcc.target/i386/avx2-vect-simd-15.c: New test.
* gcc.target/i386/avx512f-vect-simd-12.c: New test.
* gcc.target/i386/avx512f-vect-simd-13.c: New test.
* gcc.target/i386/avx512f-vect-simd-14.c: New test.
* gcc.target/i386/avx512bw-vect-simd-15.c: New test.
* g++.dg/vect/simd-6.cc: New test.
* g++.dg/vect/simd-7.cc: New test.
* g++.dg/vect/simd-8.cc: New test.
* g++.dg/vect/simd-9.cc: New test.
* c-c++-common/gomp/scan-2.c: Don't expect any diagnostics.
From-SVN: r272544
|
|
array" arrays with one byte elements.
* tree-vect-data-refs.c (vect_find_stmt_data_reference): Handle
"omp simd array" arrays with one byte elements.
* gcc.dg/vect/vect-simd-11.c: New test.
* gcc.target/i386/sse2-vect-simd-11.c: New test.
* gcc.target/i386/avx2-vect-simd-11.c: New test.
* gcc.target/i386/avx512bw-vect-simd-11.c: New test.
From-SVN: r272542
|
|
* omp-low.c (struct omp_context): Add scan_inclusive field.
(scan_omp_1_stmt) <case GIMPLE_OMP_SCAN>: Set ctx->scan_inclusive
if inclusive scan.
(struct omplow_simd_context): Add lastlane member.
(lower_rec_simd_input_clauses): Add rvar argument, handle inscan
reductions. Build 2 or 3 argument .GOMP_SIMD_LANE calls rather than
1 or 2 argument.
(lower_rec_input_clauses): Handle inscan reductions in simd contexts.
(lower_lastprivate_clauses): Set TREE_THIS_NOTRAP on the ARRAY_REF.
(lower_omp_scan): New function.
(lower_omp_1) <case GIMPLE_OMP_SCAN>: Use lower_omp_scan.
* tree-ssa-dce.c (eliminate_unnecessary_stmts): For IFN_GOMP_SIMD_LANE
check 3rd argument if present rather than 2nd.
* tree-vectorizer.h (struct _loop_vec_info): Add scan_map member.
(struct _stmt_vec_info): Change simd_lane_access_p from bool into
2-bit bitfield.
* tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Initialize
scan_map. For IFN_GOMP_SIMD_LANE check 3rd argument if present rather
than 2nd.
(_loop_vec_info::~_loop_vec_info): Delete scan_map.
* tree-vect-data-refs.c (vect_analyze_data_ref_accesses): Allow two
different STMT_VINFO_SIMD_LANE_ACCESS_P refs if they have the same
init.
(vect_find_stmt_data_reference): Encode in ->aux the 2nd
IFN_GOMP_SIMD_LANE argument.
(vect_analyze_data_refs): Set STMT_VINFO_SIMD_LANE_ACCESS_P from the
encoded ->aux value.
* tree-vect-stmts.c: Include attribs.h.
(vectorizable_call): Adjust comment about IFN_GOMP_SIMD_LANE.
(scan_operand_equal_p, check_scan_store, vectorizable_scan_store): New
functions.
(vectorizable_load): For STMT_VINFO_SIMD_LANE_ACCESS_P tests use != 0.
(vectorizable_store): Handle STMT_VINFO_SIMD_LANE_ACCESS_P > 1.
cp/
* semantics.c (finish_omp_clauses): For OMP_CLAUSE_REDUCTION_INSCAN
set need_copy_assignment.
testsuite/
* gcc.dg/vect/vect-simd-8.c: New test.
* gcc.dg/vect/vect-simd-9.c: New test.
* g++.dg/vect/simd-2.cc: New test.
* g++.dg/gomp/scan-1.C: New test.
From-SVN: r272399
|
|
Current vectoriser doesn't support masked loads for SLP. We should add that, to
allow things like:
void
f (int *restrict x, int *restrict y, int *restrict z, int n)
{
for (int i = 0; i < n; i += 2)
{
x[i] = y[i] ? z[i] : 1;
x[i + 1] = y[i + 1] ? z[i + 1] : 2;
}
}
to be vectorized using contiguous loads rather than LD2 and ST2.
This patch was motivated by SVE, but it is completely generic and should apply
to any architecture with masked loads.
From-SVN: r271704
|
|
2019-04-11 Richard Biener <rguenther@suse.de>
PR tree-optimization/90018
* tree-vect-data-refs.c (vect_preserves_scalar_order_p):
Test both SLP and interleaving variants.
* gcc.dg/vect/pr90018.c: New testcase.
From-SVN: r270273
|
|
2019-04-10 Richard Biener <rguenther@suse.de>
* tree-vectorizer.h (_stmt_vec_info): Remove same_dr_stmt
member.
(DR_GROUP_SAME_DR_STMT): Remove.
* tree-vect-stmts.c (vectorizable_load): Remove unreachable code.
* tree-vect-data-refs.c (vect_analyze_group_access_1): Likewise,
replace with assert.
(vect_analyze_data_ref_accesses): Fix INTEGER_CST comparison.
(vect_record_grouped_load_vectors): Remove unreachable code.
From-SVN: r270250
|
