diff options
| author | H.J. Lu <hjl.tools@gmail.com> | 2017-06-09 05:13:15 -0700 |
|---|---|---|
| committer | H.J. Lu <hjl.tools@gmail.com> | 2017-06-09 05:13:31 -0700 |
| commit | 2f5d20ac99b9434a634629282cbb46e2a8d56a1c (patch) | |
| tree | 6a9dcdd8346cd5ad6473b165a404c9961359c6ed /ChangeLog | |
| parent | 990c32b93a29d8b8d599e10ebca19a260f84cbba (diff) | |
| download | glibc-2f5d20ac99b9434a634629282cbb46e2a8d56a1c.zip glibc-2f5d20ac99b9434a634629282cbb46e2a8d56a1c.tar.gz glibc-2f5d20ac99b9434a634629282cbb46e2a8d56a1c.tar.bz2 | |
x86-64: Optimize memchr/rawmemchr/wmemchr with SSE2/AVX2
SSE2 memchr is extended to support wmemchr. AVX2 memchr/rawmemchr/wmemchr
are added to search 32 bytes with a single vector compare instruction.
AVX2 memchr/rawmemchr/wmemchr are as fast as SSE2 memchr/rawmemchr/wmemchr
for small sizes and up to 1.5X faster for larger sizes on Haswell and
Skylake. Select AVX2 memchr/rawmemchr/wmemchr on AVX2 machines where
vzeroupper is preferred and AVX unaligned load is fast.
NB: It uses TZCNT instead of BSF since TZCNT produces the same result
as BSF for non-zero input. TZCNT is faster than BSF and is executed
as BSF if machine doesn't support TZCNT.
* sysdeps/x86_64/memchr.S (MEMCHR): New. Depending on if
USE_AS_WMEMCHR is defined.
(PCMPEQ): Likewise.
(memchr): Renamed to ...
(MEMCHR): This. Support wmemchr if USE_AS_WMEMCHR is defined.
Replace pcmpeqb with PCMPEQ.
* sysdeps/x86_64/multiarch/Makefile (sysdep_routines): Add
memchr-sse2, rawmemchr-sse2, memchr-avx2, rawmemchr-avx2,
wmemchr-sse4_1, wmemchr-avx2 and wmemchr-c.
* sysdeps/x86_64/multiarch/ifunc-avx2.h: New file.
* sysdeps/x86_64/multiarch/memchr-avx2.S: Likewise.
* sysdeps/x86_64/multiarch/memchr-sse2.S: Likewise.
* sysdeps/x86_64/multiarch/memchr.c: Likewise.
* sysdeps/x86_64/multiarch/rawmemchr-avx2.S: Likewise.
* sysdeps/x86_64/multiarch/rawmemchr-sse2.S: Likewise.
* sysdeps/x86_64/multiarch/rawmemchr.c: Likewise.
* sysdeps/x86_64/multiarch/wmemchr-avx2.S: Likewise.
* sysdeps/x86_64/multiarch/wmemchr-sse2.S: Likewise.
* sysdeps/x86_64/multiarch/wmemchr.c: Likewise.
* sysdeps/x86_64/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Test __memchr_avx2, __memchr_sse2,
__rawmemchr_avx2, __rawmemchr_sse2, __wmemchr_avx2 and
__wmemchr_sse2.
Diffstat (limited to 'ChangeLog')
| -rw-r--r-- | ChangeLog | 26 |
1 files changed, 26 insertions, 0 deletions
@@ -1,3 +1,29 @@ +2017-06-09 H.J. Lu <hongjiu.lu@intel.com> + + * sysdeps/x86_64/memchr.S (MEMCHR): New. Depending on if + USE_AS_WMEMCHR is defined. + (PCMPEQ): Likewise. + (memchr): Renamed to ... + (MEMCHR): This. Support wmemchr if USE_AS_WMEMCHR is defined. + Replace pcmpeqb with PCMPEQ. + * sysdeps/x86_64/multiarch/Makefile (sysdep_routines): Add + memchr-sse2, rawmemchr-sse2, memchr-avx2, rawmemchr-avx2, + wmemchr-sse4_1, wmemchr-avx2 and wmemchr-c. + * sysdeps/x86_64/multiarch/ifunc-avx2.h: New file. + * sysdeps/x86_64/multiarch/memchr-avx2.S: Likewise. + * sysdeps/x86_64/multiarch/memchr-sse2.S: Likewise. + * sysdeps/x86_64/multiarch/memchr.c: Likewise. + * sysdeps/x86_64/multiarch/rawmemchr-avx2.S: Likewise. + * sysdeps/x86_64/multiarch/rawmemchr-sse2.S: Likewise. + * sysdeps/x86_64/multiarch/rawmemchr.c: Likewise. + * sysdeps/x86_64/multiarch/wmemchr-avx2.S: Likewise. + * sysdeps/x86_64/multiarch/wmemchr-sse2.S: Likewise. + * sysdeps/x86_64/multiarch/wmemchr.c: Likewise. + * sysdeps/x86_64/multiarch/ifunc-impl-list.c + (__libc_ifunc_impl_list): Test __memchr_avx2, __memchr_sse2, + __rawmemchr_avx2, __rawmemchr_sse2, __wmemchr_avx2 and + __wmemchr_sse2. + 2017-06-09 Florian Weimer <fweimer@redhat.com> * malloc/Makefile (test-srcs): Remove tst-dynarray, |