diff options
| author | Tamar Christina <tamar.christina@arm.com> | 2023-11-21 13:14:29 +0000 |
|---|---|---|
| committer | Tamar Christina <tamar.christina@arm.com> | 2023-11-21 13:14:29 +0000 |
| commit | 5ff006bd3db5580fa70a05bd275b67067b8486d9 (patch) | |
| tree | ea20c7370d84942a760c0f8a1688d9f4977b570b /gcc | |
| parent | 0a033038cdd1360f96dff171d719ea49f951ce63 (diff) | |
| download | gcc-5ff006bd3db5580fa70a05bd275b67067b8486d9.zip gcc-5ff006bd3db5580fa70a05bd275b67067b8486d9.tar.gz gcc-5ff006bd3db5580fa70a05bd275b67067b8486d9.tar.bz2 | |
AArch64: only discount MLA for vector and scalar statements
In testcases gcc.dg/tree-ssa/slsr-19.c and gcc.dg/tree-ssa/slsr-20.c we have a
fairly simple computation. On the current generic costing we generate:
f:
add w0, w0, 2
madd w1, w0, w1, w1
lsl w0, w1, 1
ret
but on any other cost model but generic (including the new up coming generic)
we generate:
f:
adrp x2, .LC0
dup v31.2s, w0
fmov s30, w1
ldr d29, [x2, #:lo12:.LC0]
add v31.2s, v31.2s, v29.2s
mul v31.2s, v31.2s, v30.s[0]
addp v31.2s, v31.2s, v31.2s
fmov w0, s31
ret
.LC0:
.word 2
.word 4
This seems to be because the vectorizer thinks the vector transfers are free:
x1_4 + x2_6 1 times vector_stmt costs 0 in body
x1_4 + x2_6 1 times vec_to_scalar costs 0 in body
This happens because the stmt it's using to get the cost of register transfers
for the given type happens to be one feeding into a MUL. we incorrectly
discount the + for the register transfer.
This is fixed by guarding the check for aarch64_multiply_add_p with a kind
check and only do it for scalar_stmt and vector_stmt.
I'm sending this separate to my patch series but it's required for it.
It also seems to fix overvectorization cases in fotonik3d_r in SPECCPU 2017.
gcc/ChangeLog:
* config/aarch64/aarch64.cc (aarch64_adjust_stmt_cost): Guard mla.
(aarch64_vector_costs::count_ops): Likewise.
Diffstat (limited to 'gcc')
| -rw-r--r-- | gcc/config/aarch64/aarch64.cc | 21 |
1 files changed, 12 insertions, 9 deletions
diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc index 800a8b0..72e1350 100644 --- a/gcc/config/aarch64/aarch64.cc +++ b/gcc/config/aarch64/aarch64.cc @@ -17007,7 +17007,7 @@ aarch64_adjust_stmt_cost (vec_info *vinfo, vect_cost_for_stmt kind, } gassign *assign = dyn_cast<gassign *> (STMT_VINFO_STMT (stmt_info)); - if (assign) + if ((kind == scalar_stmt || kind == vector_stmt) && assign) { /* For MLA we need to reduce the cost since MLA is 1 instruction. */ if (!vect_is_reduction (stmt_info) @@ -17088,15 +17088,18 @@ aarch64_vector_costs::count_ops (unsigned int count, vect_cost_for_stmt kind, ops->reduction_latency = MAX (ops->reduction_latency, base); } - /* Assume that multiply-adds will become a single operation. */ - if (stmt_info && aarch64_multiply_add_p (m_vinfo, stmt_info, m_vec_flags)) - return; + if (stmt_info && (kind == scalar_stmt || kind == vector_stmt)) + { + /* Assume that multiply-adds will become a single operation. */ + if (aarch64_multiply_add_p (m_vinfo, stmt_info, m_vec_flags)) + return; + + /* Assume that bool AND with compare operands will become a single + operation. */ + if (aarch64_bool_compound_p (m_vinfo, stmt_info, m_vec_flags)) + return; + } - /* Assume that bool AND with compare operands will become a single - operation. */ - if (stmt_info - && aarch64_bool_compound_p (m_vinfo, stmt_info, m_vec_flags)) - return; /* Count the basic operation cost associated with KIND. */ switch (kind) |