diff options
| author | Noah Goldstein <goldstein.w.n@gmail.com> | 2022-06-06 21:11:28 -0700 |
|---|---|---|
| committer | Noah Goldstein <goldstein.w.n@gmail.com> | 2022-06-07 13:08:28 -0700 |
| commit | dd5c483b2598f411428df4d8864c15c4b8a3cd68 (patch) | |
| tree | c859b1416e2e839f5d9903efe6358373b9f68b97 /sysdeps/x86_64 | |
| parent | 8a780a6b910023e71f3173f37f0793834c047554 (diff) | |
| download | glibc-dd5c483b2598f411428df4d8864c15c4b8a3cd68.zip glibc-dd5c483b2598f411428df4d8864c15c4b8a3cd68.tar.gz glibc-dd5c483b2598f411428df4d8864c15c4b8a3cd68.tar.bz2 | |
x86: Add COND_VZEROUPPER that can replace vzeroupper if no `ret`
The RTM vzeroupper mitigation has no way of replacing inline
vzeroupper not before a return.
This can be useful when hoisting a vzeroupper to save code size
for example:
```
L(foo):
cmpl %eax, %edx
jz L(bar)
tzcntl %eax, %eax
addq %rdi, %rax
VZEROUPPER_RETURN
L(bar):
xorl %eax, %eax
VZEROUPPER_RETURN
```
Can become:
```
L(foo):
COND_VZEROUPPER
cmpl %eax, %edx
jz L(bar)
tzcntl %eax, %eax
addq %rdi, %rax
ret
L(bar):
xorl %eax, %eax
ret
```
This code does not change any existing functionality.
There is no difference in the objdump of libc.so before and after this
patch.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Diffstat (limited to 'sysdeps/x86_64')
| -rw-r--r-- | sysdeps/x86_64/multiarch/avx-rtm-vecs.h | 1 | ||||
| -rw-r--r-- | sysdeps/x86_64/sysdep.h | 18 |
2 files changed, 19 insertions, 0 deletions
diff --git a/sysdeps/x86_64/multiarch/avx-rtm-vecs.h b/sysdeps/x86_64/multiarch/avx-rtm-vecs.h index 3f531dd..6ca9f5e 100644 --- a/sysdeps/x86_64/multiarch/avx-rtm-vecs.h +++ b/sysdeps/x86_64/multiarch/avx-rtm-vecs.h @@ -20,6 +20,7 @@ #ifndef _AVX_RTM_VECS_H #define _AVX_RTM_VECS_H 1 +#define COND_VZEROUPPER COND_VZEROUPPER_XTEST #define ZERO_UPPER_VEC_REGISTERS_RETURN \ ZERO_UPPER_VEC_REGISTERS_RETURN_XTEST diff --git a/sysdeps/x86_64/sysdep.h b/sysdeps/x86_64/sysdep.h index f14d507..4f512d5 100644 --- a/sysdeps/x86_64/sysdep.h +++ b/sysdeps/x86_64/sysdep.h @@ -106,6 +106,24 @@ lose: \ vzeroupper; \ ret +/* Can be used to replace vzeroupper that is not directly before a + return. This is useful when hoisting a vzeroupper from multiple + return paths to decrease the total number of vzerouppers and code + size. */ +#define COND_VZEROUPPER_XTEST \ + xtest; \ + jz 1f; \ + vzeroall; \ + jmp 2f; \ +1: \ + vzeroupper; \ +2: + +/* In RTM define this as COND_VZEROUPPER_XTEST. */ +#ifndef COND_VZEROUPPER +# define COND_VZEROUPPER vzeroupper +#endif + /* Zero upper vector registers and return. */ #ifndef ZERO_UPPER_VEC_REGISTERS_RETURN # define ZERO_UPPER_VEC_REGISTERS_RETURN \ |