Move malloc failure testing into OPENSSL_malloc

Rather than trying to override the actual malloc symbol, just intercept
OPENSSL_malloc and gate it on a build flag. (When we first wrote these,
OPENSSL_malloc was just an alias for malloc.)

This has several benefits:

- This is cross platform. We don't interfere with sanitizers or the
  libc, or have to mess with global symbols.

- This removes the reason bssl_shim and handshaker linked
  test_support_lib, so we can fix the tes_support_lib / gtest
  dependency.

- If we ever reduce the scope of fallible mallocs, we'll want to
  constrain the tests to only the ones that are fallible. An
  interception strategy like this can do it. Hopefully that will also
  take less time to run in the future.

Also fix the ssl malloc failure tests, as they haven't been working for
a while. (Malloc failure tests still take far too long to run to the end
though. My immediate motivation is less malloc failure and more to tidy
up the build.)

Bug: 563
Change-Id: I32165b8ecbebfdcfde26964e06a404762edd28e3
Reviewed-on: https://boringssl-review.googlesource.com/c/boringssl/+/56925
Commit-Queue: Bob Beck <bbe@google.com>
Reviewed-by: Bob Beck <bbe@google.com>
Auto-Submit: David Benjamin <davidben@google.com>

1 files changed, 4 insertions, 0 deletions

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 536d83e..e0808d7 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -364,6 +364,10 @@ if(CONSTANT_TIME_VALIDATION)
   add_definitions(-DNDEBUG)
 endif()
 
+if(MALLOC_FAILURE_TESTING)
+  add_definitions(-DBORINGSSL_MALLOC_FAILURE_TESTING)
+endif()
+
 function(go_executable dest package)
   set(godeps "${CMAKE_SOURCE_DIR}/util/godeps.go")
   if(NOT CMAKE_GENERATOR STREQUAL "Ninja")