[libunwind][AIX] Save/restore errno before/after system calls dlopen/dlsym/dlclose

Summary:
libunwind on AIX calls dlopen()/dlsym()/dlclose() to dynamically load libc++abi and get the personality for state table EH when it is running against the legacy xlcang++ compiler genereated applications. dlopen() sets errno to 0 when it is successful, which clobbers the value in errno from the user code. This seems to be an AIX bug that it should not set errno to 0 according to POSIX. We will open a bug report to AIX but in the mean time there won't be time line when AIX will have a fix and even AIX does fix it, it won't help earlier AIX releases in the field. This patch saves and restores errno before and after these calls so that user code can work as expected.

Reviewed by: compnerd, libunwind

Differential Revision: https://reviews.llvm.org/D131292

1 files changed, 6 insertions, 0 deletions

diff --git a/libunwind/src/UnwindCursor.hpp b/libunwind/src/UnwindCursor.hpp
index b8bd9bc..d325ed5 100644
--- a/libunwind/src/UnwindCursor.hpp
+++ b/libunwind/src/UnwindCursor.hpp
@@ -2106,6 +2106,11 @@ bool UnwindCursor<A, R>::getInfoFromTBTable(pint_t pc, R &registers) {
           // using dlopen().
           const char libcxxabi[] = "libc++abi.a(libc++abi.so.1)";
           void *libHandle;
+          // The AIX dlopen() sets errno to 0 when it is successful, which
+          // clobbers the value of errno from the user code. This is an AIX
+          // bug because according to POSIX it should not set errno to 0. To
+          // workaround before AIX fixes the bug, errno is saved and restored.
+          int saveErrno = errno;
           libHandle = dlopen(libcxxabi, RTLD_MEMBER | RTLD_NOW);
           if (libHandle == NULL) {
             _LIBUNWIND_TRACE_UNWINDING("dlopen() failed with errno=%d\n",
@@ -2119,6 +2124,7 @@ bool UnwindCursor<A, R>::getInfoFromTBTable(pint_t pc, R &registers) {
             assert(0 && "dlsym() failed");
           }
           dlclose(libHandle);
+          errno = saveErrno;
         }
       }
       xlcPersonalityV0InitLock.unlock();