| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
fix typos
|
|
fix typos
|
|
The following assertion was being triggered:
```
assert.h assertion failed at llvm-project/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:237 in void fuzzer::TracePC::UpdateObservedPCs(): M.Size() == (size_t)(ModulePCTable[i].Stop - ModulePCTable[i].Start)
```
# The bug
When built with `-fsanitize=fuzzer`, each “module” (.so file, or the
binary itself) will be instrumented, and when loaded into the process
will make a call to these two functions:
- `__sanitizer_cov_8bit_counters_init`
- `__sanitizer_cov_pcs_init`
Each of these is called with start and end pointers defining an array.
In libFuzzer, these functions are implemented with
`HandleInline8bitCountersInit` and `HandlePCsInit`. Each of them pushes
back the provided pointers into a separate array, `Modules` and
`ModulePCTable` respectively. These arrays are meant to be kept in-sync;
index i into Modules should refer to the same `.so` as index i into
ModulePCTable. The assertion was triggering because these lists got
out-of-sync.
The problem is that the 8bit handler contains this line:
```
if (Start == Stop) return;
```
but the PC handler contains no such corresponding line. This meant that
if a module was ever instrumented but “empty” (its 8bit counter and PC
arrays were both of length 0), then its PC array would still be added
but its 8bit counter array would not.
# Why this issue was never seen before
The circumstances to trigger this issue are unusual:
- You need a compilation unit that doesn't contain any code (though it
may contain global variable declarations and similar). That doesn't
happen very often.
- That compilation unit must be dynamically linked, not statically
linked. If statically linked, it’ll be merged into a single “module”
with the main binary, and the arrays will be merged as well; you won’t
end up with length-0 arrays.
- To notice the issue, assertions must be enabled. If disabled,
libFuzzer will be buggy (it may have worse coverage), but it won't
crash, and "worse coverage" is extremely unlikely to be noticed.
# This change
This change solves the issue by adding the same `if (Start == Stop)
return;` check to `HandlePCsInit`. This prevents the arrays from getting
out-of-sync. This change also adds a test that identifies the previous
issue when compiled with assertions enabled, but now passes with the
fix.
|
|
(#159097)
Fix below buildbot failure.
```
/var/lib/buildbot/fuchsia-x86_64-linux/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:172:42: error: use of undeclared identifier 'uint'
172 | Printf("Flag: %s %u\n", Name, (uint)Val);
```
Replace uint with uint32_t cast.
This reverts commit 8062b166762b51f1c3a9168e7031babde3e330a8.
|
|
This reverts commit b44e6e01f7f778bbb569e07821f5ddfdb90c4d4d.
|
|
This pull request addresses fixes against violations happening under
subcategory 'cpp/wrong-type-format-argument' related to
dfsan,fuzzer,hwasan.
|
|
The declaration was static when it shouldn't be since it can be defined
in FuzzerUtilFuchsia.cpp
|
|
(#155271)
…(#155042)
This reverts commit 781a4db6b50bb660cb293d3e7e29957aeb4b02ac.
Relanded with the fix declaring StartRssThread.
|
|
Reverts llvm/llvm-project#154854 because it broke Clang toolchain
builders for Fuchsia:
https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8705803649235662417/overview
|
|
Every once in a couple hundred runs of a downstream fuzzer test, we see
a fuzzing test freeze while waiting for a thread to be suspended. The
main thread is frozen because it's waiting to suspend either the alarm
or rss thread which is stuck waiting for an exception they sent out to
be handled. Specifically, both threads send out a synthetic
`ZX_EXCP_THREAD_STARTING` exception to be handled by the crash handling
thread which sets up an exception channel on the whole process with
`ZX_EXCEPTION_CHANNEL_DEBUGGER`. This is the only channel type that
listens to thread stop/start exceptions. Normally, the exception would
be ignored and the alarm or rss thread would continue normally once the
crash handling thread closes the read exception. However, the memory
snapshot machinery can suspend this thread while its in the process of
waiting for or handling a `ZX_EXCP_THREAD_STARTING` sent by either the
rss or alarm thread. If this is suspended first, then we attempt to
suspend either the alarm or rss thread while they're still waiting for
the crash handling thread to handle its exception, we will freeze
waiting for those threads to give the suspend signal, which they won't
because they're blocked on waiting for the exception handler. This is
the deadlock.
Until there's a way for the memory snapshot machinery to suspend the
thread while it's stuck on an exception, then we can work around this in
the meantime by just ensuring the alarm and rss threads start normally
via signals on the initial startup path. I can assert locally the
freezing doesn't occur after 6000 runs where prior we would see it every
couple hundred runs. Note this type of issue can arise again if the
fuzzing test launches any dangling threads that happen to not start yet.
One of the recommendations for writing a fuzz test is that the test may
launch threads, but they should be joined by the end of the test
(https://llvm.org/docs/LibFuzzer.html#fuzz-target), so hopefully we
won't see this type of bug rise frequently from fuzz tests. More
broadly, this can also arise if any process launches its own debugger
via `ZX_EXCEPTION_CHANNEL_DEBUGGER`, but I would think in practice this
isn't very likely to happen.
More context in https://fxbug.dev/436923423.
---------
Co-authored-by: Petr Hosek <phosek@google.com>
|
|
Reverts llvm/llvm-project#147422
Seems to be causing problems with tracebacks. Probably the trackback
code doesn't know how to switch back to the regular stack after it gets
to the top of the signal stack.
|
|
SA_ONSTACK is required for certain runtimes that use small stacks, for
instance the Go runtime.
See https://github.com/golang/go/issues/49075
SA_ONSTACK is a no-op unless someone also calls sigaltstack.
|
|
Swift's FatalError raises a SIGTRAP, which currently causes the fuzzer
to exit without writing out the crashing input.
rdar://142975522
|
|
This is to fix modules build errors in chromium like
*
https://ci.chromium.org/ui/p/chromium/builders/try/linux-libfuzzer-asan-rel/2292144/overview
*
https://ci.chromium.org/ui/p/chromium/builders/try/linux-libfuzzer-asan-rel/2292444/overview
---------
Co-authored-by: Petr Hosek <phosek@google.com>
|
|
Fix for #144495 by 6f4add3 broke sanitizer-aarch64-linux buildbot.
compiler-rt/lib/fuzzer/tests build failed because the linker was
looking gcc_s without '-l' appended.
The CMake script was adding the library name without the required
'-l' prefix. This patch adds the -l prefix changing gcc_s to -lgcc_s
and gcc to -lgcc.
https://lab.llvm.org/buildbot/#/builders/51/builds/18170
|
|
compiler-rt/lib/fuzzer/tests build was failing on armv7, with undefined
references to unwinder symbols, such as __aeabi_unwind_cpp_pr0.
This occurs because the test is built with `-nostdlib++` but `libunwind`
is not explicitly linked to the final test executable.
This patch resolves the issue by adding CMake logic to explicitly link
the required unwinder to the fuzzer tests, inspired by the same solution
used to fix Scudo build failures by https://reviews.llvm.org/D142888.
|
|
|
|
|
|
(#115811)
Reverts llvm/llvm-project#110274
Buildbots broke
|
|
SetTimer() now uses setitimer() to sending SIGALRM every `
UnitTimeoutSec/2 + 1` s
Set UnitTimeoutSec with the `-timeout=` option
"POSIX.1-2008 marks getitimer() and setitimer() obsolete" and also has
some issues regarding accuracy of the timers under load . See
https://linux.die.net/man/2/setitimer.
I propose using timer_create() and sigaction() ,See
http://man7.org/linux/man-pages/man2/timer_create.2.html
# test result on my x86_64 linux
`make check-fuzzer`
|
|
The INSTALL_BYPRODUCTS ExternalProject_Add() argument was only added in
CMake 3.26 and the current minimum is 3.20. Work around this by using an
explicit ExternalProject_Add_Step() call for the install step with a
BYPRODUCTS argument. We can't keep using the `install` name here since that
is reserved by the CMake implementation and results in errors when used.
This commit should be reverted once LLVM depends on CMake 3.26.
Pull Request: https://github.com/llvm/llvm-project/pull/115677
|
|
Since b4130bee6bfd34d8045f02fc9f951bcb5db9d85c, we check for
_LIBCPP_HAS_THREAD_API_PTHREAD to decide between using
SetThreadDescription or pthread_setname_np for setting the thread name.
c6f3b7bcd0596d30f8dabecdfb9e44f9a07b6e4c changed how libcxx defines
their configuration macros - now they are always defined, but defined to
0 or 1, while they previously were either defined or undefined.
As these libcxx defines used to be defined to an empty string (rather
than expanding to 1) if enabled, we can't easily produce an expression
that works both with older and newer libcxx. Additionally, these defines
are libcxx internal config macros that aren't a detail that isn't
supported and isn't meant to be relied upon.
Simply skip trying to set thread name on MinGW as we can't easily know
which kind of thread native handle we have. Setting the thread name is
only a nice to have, quality of life improvement - things should work
the same even without it.
Additionally, libfuzzer isn't generally usable on MinGW targets yet
(Clang doesn't include it in the getSupportedSanitizers() method for the
MinGW target), so this shouldn't make any difference in practice anyway.
|
|
Using the build tree is somewhat fragile since the layout is not
guaranteed to be stable and means the tests are tightly coupled to the
libc++/libc++abi build tree layout. Instead update the ExternalProject
to install the library and headers and do not add the build tree to
the include/linker flags.
Pull Request: https://github.com/llvm/llvm-project/pull/115077
|
|
It was originally included transitively, but no longer is after recent
<vector> cleanups in libc++.
Similar to #113951.
|
|
The `check-fuzzer` runs fine with cl build llvm, but the following lit
tests fail with clang-cl build llvm
```
********************
Timed Out Tests (2):
libFuzzer-x86_64-default-Windows :: fork-ubsan.test
libFuzzer-x86_64-default-Windows :: fuzzer-oom.test
********************
Failed Tests (22):
libFuzzer-x86_64-default-Windows :: acquire-crash-state.test
libFuzzer-x86_64-default-Windows :: cross_over_copy.test
libFuzzer-x86_64-default-Windows :: cross_over_insert.test
libFuzzer-x86_64-default-Windows :: exit_on_src_pos.test
libFuzzer-x86_64-default-Windows :: fuzzer-alignment-assumption.test
libFuzzer-x86_64-default-Windows :: fuzzer-implicit-integer-sign-change.test
libFuzzer-x86_64-default-Windows :: fuzzer-implicit-signed-integer-truncation-or-sign-change.test
libFuzzer-x86_64-default-Windows :: fuzzer-implicit-signed-integer-truncation.test
libFuzzer-x86_64-default-Windows :: fuzzer-implicit-unsigned-integer-truncation.test
libFuzzer-x86_64-default-Windows :: fuzzer-printcovpcs.test
libFuzzer-x86_64-default-Windows :: fuzzer-timeout.test
libFuzzer-x86_64-default-Windows :: fuzzer-ubsan.test
libFuzzer-x86_64-default-Windows :: minimize_crash.test
libFuzzer-x86_64-default-Windows :: minimize_two_crashes.test
libFuzzer-x86_64-default-Windows :: null-deref-on-empty.test
libFuzzer-x86_64-default-Windows :: null-deref.test
libFuzzer-x86_64-default-Windows :: print-func.test
libFuzzer-x86_64-default-Windows :: stack-overflow-with-asan.test
libFuzzer-x86_64-default-Windows :: trace-malloc-2.test
libFuzzer-x86_64-default-Windows :: trace-malloc-unbalanced.test
libFuzzer-x86_64-default-Windows :: trace-malloc.test
```
The related commits are
https://github.com/llvm/llvm-project/commit/53a81d4d26f0409de8a0655d7af90f2bea222a12
and
https://github.com/llvm/llvm-project/commit/e31efd8f6fbc27000a4933f889e0deb922411006.
Following the change in
https://github.com/llvm/llvm-project/commit/e31efd8f6fbc27000a4933f889e0deb922411006
can fix these failures.
As for the issue mentioned in the comment that alternatename support in
clang not good enough(https://bugs.llvm.org/show_bug.cgi?id=40218). I
find that using `__builtin_function_start(func)` instead of directly
using `func` would make it work as intended.
|
|
Fixes https://github.com/llvm/llvm-project/issues/106871
|
|
(#106902)"
This reverts commit 7c4cffd9d8be424e9e9542be9aec3b5a6f69073e.
This commit broke compilation in environments that don't use
winpthreads.
|
|
|
|
|
|
Reverts llvm/llvm-project#99953
|
|
|
|
In C, it is UB to call `memcmp(NULL, NULL, 0)`, `memcpy(NULL, NULL, 0)`,
etc. Unfortunately, `(NULL, 0)` is the natural representation of an
empty sequence of objects and extremely common in real world code. As a
result, all C code, and C++ code which calls into C functions, must
carefully guard all calls to `memcpy`.
This is a serious, real world usability issue in C and should be fixed
in the language (see #49459). In the meantime, pay the cost of the extra
branch to avoid tripping UBSan in libFuzzer. Once the usability problem
in C has been fixed, these checks can be removed.
Fixes #96772
|
|
This fixes a few of these warnings, when building with Clang ToT on
Windows:
```
[622/7618] Building CXX object
projects\compiler-rt\lib\sanitizer_common\CMakeFiles\RTSanitizerCommonSymbolizer.x86_64.dir\sanitizer_symbolizer_win.cpp.obj
C:\src\git\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_symbolizer_win.cpp(74,3):
warning: cast from 'FARPROC' (aka 'long long (*)()') to
'decltype(::StackWalk64) *' (aka 'int (*)(unsigned long, void *, void *,
_tagSTACKFRAME64 *, void *, int (*)(void *, unsigned long long, void *,
unsigned long, unsigned long *), void *(*)(void *, unsigned long long),
unsigned long long (*)(void *, unsigned long long), unsigned long long
(*)(void *, void *, _tagADDRESS64 *))') converts to incompatible
function type [-Wcast-function-type-mismatch]
```
This is similar to https://github.com/llvm/llvm-project/pull/97905
|
|
Closes #82307.
I built LLVM with the changes and tested fuzzing in fork mode. The
coverage number was correct:
```
[ye@ye-arch ~]$ /home/ye/work/llvm-project/build/bin/clang++ -fsanitize=fuzzer test_fuzzer.cc
[ye@ye-arch ~]$ ./a.out corpus -fork=4
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3152497917
INFO: Loaded 1 modules (40 inline 8-bit counters): 40 [0x5aa6f7b310d0, 0x5aa6f7b310f8),
INFO: Loaded 1 PC tables (40 PCs): 40 [0x5aa6f7b310f8,0x5aa6f7b31378),
INFO: -fork=4: fuzzing in separate process(s)
INFO: -fork=4: 56 seed inputs, starting to fuzz in /tmp/libFuzzerTemp.FuzzWithFork54465.dir
#600649: cov: 36 ft: 224 corp: 56 exec/s: 300324 oom/timeout/crash: 0/0/0 time: 2s job: 1 dft_time: 0
#1548208: cov: 36 ft: 224 corp: 56 exec/s: 315853 oom/timeout/crash: 0/0/0 time: 3s job: 2 dft_time: 0
#2465991: cov: 36 ft: 224 corp: 56 exec/s: 229445 oom/timeout/crash: 0/0/0 time: 4s job: 3 dft_time: 0
#3887877: cov: 36 ft: 224 corp: 56 exec/s: 284377 oom/timeout/crash: 0/0/0 time: 5s job: 4 dft_time: 0
```
|
|
ToT targetting Windows"
This reverts commit 10e1b935e5d9017067207d62ababa733df088ecd.
|
|
targetting Windows
|
|
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
|
|
libfuzzer's -jobs option will, depending on the number of CPUs, spin up
a
WorkerThread and end up printing the log file using CopyFileToErr.
This leads to an MSan false positive. This patch disables the MSan
interceptor checks,
similarly to other instances in https://reviews.llvm.org/D48891
Side-note: this false positive issue first appeared when printf()
was replaced by puts() (90b4d1bcb20180c591385131b12fa90d2e4860b1).
The interceptor check was always present; however, MSan does not
check_printf by default.
|
|
libFuzzer uses std::clamp which was introduced in C++17.
|
|
All these unit tests already include ${COMPILER_RT_GTEST_SOURCE} as an
input source file and the target llvm_gtest does not exist for
standalone builds. Currently the DEPS argument is ignored for standalone
builds so the missing target is not a problem, but as part of fixing a
build race for standalone builds I am planning to include those
dependencies in COMPILER_RT_TEST_STANDALONE_BUILD_LIBS configurations.
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/83649
|
|
The actual solution relies on the premise /proc/self/smaps existence.
instead relying on native api like freebsd.
fixing fuzzer build too.
|
|
|
|
Following-up on GH-76761.
|
|
|
|
shuffling the order of its includes.
|
|
SetThreadDescription symbol needs to be dynamically loaded before usage.
Then using a wide string buffer, since we re using a null terminated
string, we can use MultiByteToWideChar -1 as 4th argument to finally set
the thread name.
Previously `SetThreadDescription` was called directly causing crash.
It was reverted in dd3aa26fc8e9de37a39611f7a6a602bcb4153784
|
|
This updates the signal handle thread coordinating to use a user signal
bit on the SignalHandlerEvent to coordinate shutdown instead of closing
the event handle. Closing the event handle is racy as the handle may be
closed before the signal handler thread resolves the handle value in
_zx_object_wait_many() and we would like to make this an explicit error.
Using the user signal bit 1 instead and then closing the event object
after the signal handler thread is joined cannot race as the wait will
terminate whether the signal is raised before or after the wait begins.
|
|
The Android LLVM build system builds the arm64 fuzzer lib without
HWASan, but then applications that enable HWASan can generated an object
file with a HWASan-ified version of some libc++ symbols (e.g.
`std::__1::piecewise_construct`). The linker can choose the HWASan-ified
definition, but then it cannot resolve the relocation from
libclang_rt.fuzzer-aarch64-android.a to this symbol because the high
bits of the address are unexpectedly set. This produces an error:
```
relocation R_AARCH64_ADR_PREL_PG_HI21 out of range
```
Fix this problem by linking a custom isolated libc++ into Android's
fuzzer library.
We need to pass through ANDROID_NATIVE_API_LEVEL so that the libc++ for
32-bit Android (API < 24) uses LLVM_FORCE_SMALLFILE_FOR_ANDROID.
|
|
|
|
Instead of a linear scan, use a bitset to track rarity of features. This
improves fuzzer throughput rather dramatically (close to 2x) in early
exploratory phases; in steady state this seems to improve fuzzing
throughput by ~15% according to perf.
The benchmarks are done on an executable with ~100k features, so the
results may change based on the executable that's being fuzzed.
kFeatureSetSize is 2M so the bitset is adding 256 KB to
sizeof(InputCorpus), but this should be fine since there's already three
arrays indexed by feature index for a total of 200 MB.
|
