tsan: fix unbounded memory consumption for large mallocs

This happens only in corner cases, but we observed this on a real app.
See the test for description of the exact scenario that lead to unbounded memory consumption.

llvm-svn: 240535