diff options
Diffstat (limited to 'clang/docs')
| -rw-r--r-- | clang/docs/AllocToken.rst | 173 | ||||
| -rw-r--r-- | clang/docs/ClangLinkerWrapper.rst | 8 | ||||
| -rw-r--r-- | clang/docs/ClangOffloadPackager.rst | 193 | ||||
| -rw-r--r-- | clang/docs/OpenMPSupport.rst | 4 | ||||
| -rw-r--r-- | clang/docs/ReleaseNotes.rst | 8 | ||||
| -rw-r--r-- | clang/docs/UsersManual.rst | 18 | ||||
| -rw-r--r-- | clang/docs/index.rst | 2 |
7 files changed, 200 insertions, 206 deletions
diff --git a/clang/docs/AllocToken.rst b/clang/docs/AllocToken.rst new file mode 100644 index 0000000..fb5c060 --- /dev/null +++ b/clang/docs/AllocToken.rst @@ -0,0 +1,173 @@ +================= +Allocation Tokens +================= + +.. contents:: + :local: + +Introduction +============ + +Clang provides support for allocation tokens to enable allocator-level heap +organization strategies. Clang assigns mode-dependent token IDs to allocation +calls; the runtime behavior depends entirely on the implementation of a +compatible memory allocator. + +Possible allocator strategies include: + +* **Security Hardening**: Placing allocations into separate, isolated heap + partitions. For example, separating pointer-containing types from raw data + can mitigate exploits that rely on overflowing a primitive buffer to corrupt + object metadata. + +* **Memory Layout Optimization**: Grouping related allocations to improve data + locality and cache utilization. + +* **Custom Allocation Policies**: Applying different management strategies to + different partitions. + +Token Assignment Mode +===================== + +The default mode to calculate tokens is: + +* ``typehash``: This mode assigns a token ID based on the hash of the allocated + type's name. + +Other token ID assignment modes are supported, but they may be subject to +change or removal. These may (experimentally) be selected with ``-mllvm +-alloc-token-mode=<mode>``: + +* ``random``: This mode assigns a statically-determined random token ID to each + allocation site. + +* ``increment``: This mode assigns a simple, incrementally increasing token ID + to each allocation site. + +Allocation Token Instrumentation +================================ + +To enable instrumentation of allocation functions, code can be compiled with +the ``-fsanitize=alloc-token`` flag: + +.. code-block:: console + + % clang++ -fsanitize=alloc-token example.cc + +The instrumentation transforms allocation calls to include a token ID. For +example: + +.. code-block:: c + + // Original: + ptr = malloc(size); + + // Instrumented: + ptr = __alloc_token_malloc(size, <token id>); + +The following command-line options affect generated token IDs: + +* ``-falloc-token-max=<N>`` + Configures the maximum number of tokens. No max by default (tokens bounded + by ``SIZE_MAX``). + + .. code-block:: console + + % clang++ -fsanitize=alloc-token -falloc-token-max=512 example.cc + +Runtime Interface +----------------- + +A compatible runtime must be provided that implements the token-enabled +allocation functions. The instrumentation generates calls to functions that +take a final ``size_t token_id`` argument. + +.. code-block:: c + + // C standard library functions + void *__alloc_token_malloc(size_t size, size_t token_id); + void *__alloc_token_calloc(size_t count, size_t size, size_t token_id); + void *__alloc_token_realloc(void *ptr, size_t size, size_t token_id); + // ... + + // C++ operators (mangled names) + // operator new(size_t, size_t) + void *__alloc_token__Znwm(size_t size, size_t token_id); + // operator new[](size_t, size_t) + void *__alloc_token__Znam(size_t size, size_t token_id); + // ... other variants like nothrow, etc., are also instrumented. + +Fast ABI +-------- + +An alternative ABI can be enabled with ``-fsanitize-alloc-token-fast-abi``, +which encodes the token ID hint in the allocation function name. + +.. code-block:: c + + void *__alloc_token_0_malloc(size_t size); + void *__alloc_token_1_malloc(size_t size); + void *__alloc_token_2_malloc(size_t size); + ... + void *__alloc_token_0_Znwm(size_t size); + void *__alloc_token_1_Znwm(size_t size); + void *__alloc_token_2_Znwm(size_t size); + ... + +This ABI provides a more efficient alternative where +``-falloc-token-max`` is small. + +Disabling Instrumentation +------------------------- + +To exclude specific functions from instrumentation, you can use the +``no_sanitize("alloc-token")`` attribute: + +.. code-block:: c + + __attribute__((no_sanitize("alloc-token"))) + void* custom_allocator(size_t size) { + return malloc(size); // Uses original malloc + } + +Note: Independent of any given allocator support, the instrumentation aims to +remain performance neutral. As such, ``no_sanitize("alloc-token")`` +functions may be inlined into instrumented functions and vice-versa. If +correctness is affected, such functions should explicitly be marked +``noinline``. + +The ``__attribute__((disable_sanitizer_instrumentation))`` is also supported to +disable this and other sanitizer instrumentations. + +Suppressions File (Ignorelist) +------------------------------ + +AllocToken respects the ``src`` and ``fun`` entity types in the +:doc:`SanitizerSpecialCaseList`, which can be used to omit specified source +files or functions from instrumentation. + +.. code-block:: bash + + [alloc-token] + # Exclude specific source files + src:third_party/allocator.c + # Exclude function name patterns + fun:*custom_malloc* + fun:LowLevel::* + +.. code-block:: console + + % clang++ -fsanitize=alloc-token -fsanitize-ignorelist=my_ignorelist.txt example.cc + +Conditional Compilation with ``__SANITIZE_ALLOC_TOKEN__`` +----------------------------------------------------------- + +In some cases, one may need to execute different code depending on whether +AllocToken instrumentation is enabled. The ``__SANITIZE_ALLOC_TOKEN__`` macro +can be used for this purpose. + +.. code-block:: c + + #ifdef __SANITIZE_ALLOC_TOKEN__ + // Code specific to -fsanitize=alloc-token builds + #endif diff --git a/clang/docs/ClangLinkerWrapper.rst b/clang/docs/ClangLinkerWrapper.rst index eb38d2b..28f48fc 100644 --- a/clang/docs/ClangLinkerWrapper.rst +++ b/clang/docs/ClangLinkerWrapper.rst @@ -14,10 +14,10 @@ This tool works as a wrapper of the normal host linking job. This tool is used to create linked device images for offloading and the necessary runtime calls to register them. It works by first scanning the linker's input for embedded device offloading data stored at the ``.llvm.offloading`` section. This section -contains binary data created by the :doc:`ClangOffloadPackager`. The extracted -device files will then be linked. The linked modules will then be wrapped into a -new object file containing the code necessary to register it with the offloading -runtime. +contains binary data created by the ``llvm-offload-binary`` utility. The +extracted device files will then be linked. The linked modules will then be +wrapped into a new object file containing the code necessary to register it with +the offloading runtime. Usage ===== diff --git a/clang/docs/ClangOffloadPackager.rst b/clang/docs/ClangOffloadPackager.rst deleted file mode 100644 index 481069b..0000000 --- a/clang/docs/ClangOffloadPackager.rst +++ /dev/null @@ -1,193 +0,0 @@ -====================== -Clang Offload Packager -====================== - -.. contents:: - :local: - -.. _clang-offload-packager: - -Introduction -============ - -This tool bundles device files into a single image containing necessary -metadata. We use a custom binary format for bundling all the device images -together. The image format is a small header wrapping around a string map. This -tool creates bundled binaries so that they can be embedded into the host to -create a fat-binary. - -Binary Format -============= - -The binary format is marked by the ``0x10FF10AD`` magic bytes, followed by a -version. Each created binary contains its own magic bytes. This allows us to -locate all the embedded offloading sections even after they may have been merged -by the linker, such as when using relocatable linking. Conceptually, this binary -format is a serialization of a string map and an image buffer. The binary header -is described in the following :ref:`table<table-binary_header>`. - -.. table:: Offloading Binary Header - :name: table-binary_header - - +----------+--------------+----------------------------------------------------+ - | Type | Identifier | Description | - +==========+==============+====================================================+ - | uint8_t | magic | The magic bytes for the binary format (0x10FF10AD) | - +----------+--------------+----------------------------------------------------+ - | uint32_t | version | Version of this format (currently version 1) | - +----------+--------------+----------------------------------------------------+ - | uint64_t | size | Size of this binary in bytes | - +----------+--------------+----------------------------------------------------+ - | uint64_t | entry offset | Absolute offset of the offload entries in bytes | - +----------+--------------+----------------------------------------------------+ - | uint64_t | entry size | Size of the offload entries in bytes | - +----------+--------------+----------------------------------------------------+ - -Once identified through the magic bytes, we use the size field to take a slice -of the binary blob containing the information for a single offloading image. We -can then use the offset field to find the actual offloading entries containing -the image and metadata. The offload entry contains information about the device -image. It contains the fields shown in the following -:ref:`table<table-binary_entry>`. - -.. table:: Offloading Entry Table - :name: table-binary_entry - - +----------+---------------+----------------------------------------------------+ - | Type | Identifier | Description | - +==========+===============+====================================================+ - | uint16_t | image kind | The kind of the device image (e.g. bc, cubin) | - +----------+---------------+----------------------------------------------------+ - | uint16_t | offload kind | The producer of the image (e.g. openmp, cuda) | - +----------+---------------+----------------------------------------------------+ - | uint32_t | flags | Generic flags for the image | - +----------+---------------+----------------------------------------------------+ - | uint64_t | string offset | Absolute offset of the string metadata table | - +----------+---------------+----------------------------------------------------+ - | uint64_t | num strings | Number of string entries in the table | - +----------+---------------+----------------------------------------------------+ - | uint64_t | image offset | Absolute offset of the device image in bytes | - +----------+---------------+----------------------------------------------------+ - | uint64_t | image size | Size of the device image in bytes | - +----------+---------------+----------------------------------------------------+ - -This table contains the offsets of the string table and the device image itself -along with some other integer information. The image kind lets us easily -identify the type of image stored here without needing to inspect the binary. -The offloading kind is used to determine which registration code or linking -semantics are necessary for this image. These are stored as enumerations with -the following values for the :ref:`offload kind<table-offload_kind>` and the -:ref:`image kind<table-image_kind>`. - -.. table:: Image Kind - :name: table-image_kind - - +---------------+-------+---------------------------------------+ - | Name | Value | Description | - +===============+=======+=======================================+ - | IMG_None | 0x00 | No image information provided | - +---------------+-------+---------------------------------------+ - | IMG_Object | 0x01 | The image is a generic object file | - +---------------+-------+---------------------------------------+ - | IMG_Bitcode | 0x02 | The image is an LLVM-IR bitcode file | - +---------------+-------+---------------------------------------+ - | IMG_Cubin | 0x03 | The image is a CUDA object file | - +---------------+-------+---------------------------------------+ - | IMG_Fatbinary | 0x04 | The image is a CUDA fatbinary file | - +---------------+-------+---------------------------------------+ - | IMG_PTX | 0x05 | The image is a CUDA PTX file | - +---------------+-------+---------------------------------------+ - -.. table:: Offload Kind - :name: table-offload_kind - - +------------+-------+---------------------------------------+ - | Name | Value | Description | - +============+=======+=======================================+ - | OFK_None | 0x00 | No offloading information provided | - +------------+-------+---------------------------------------+ - | OFK_OpenMP | 0x01 | The producer was OpenMP offloading | - +------------+-------+---------------------------------------+ - | OFK_CUDA | 0x02 | The producer was CUDA | - +------------+-------+---------------------------------------+ - | OFK_HIP | 0x03 | The producer was HIP | - +------------+-------+---------------------------------------+ - | OFK_SYCL | 0x04 | The producer was SYCL | - +------------+-------+---------------------------------------+ - -The flags are used to signify certain conditions, such as the presence of -debugging information or whether or not LTO was used. The string entry table is -used to generically contain any arbitrary key-value pair. This is stored as an -array of the :ref:`string entry<table-binary_string>` format. - -.. table:: Offloading String Entry - :name: table-binary_string - - +----------+--------------+-------------------------------------------------------+ - | Type | Identifier | Description | - +==========+==============+=======================================================+ - | uint64_t | key offset | Absolute byte offset of the key in the string table | - +----------+--------------+-------------------------------------------------------+ - | uint64_t | value offset | Absolute byte offset of the value in the string table | - +----------+--------------+-------------------------------------------------------+ - -The string entries simply provide offsets to a key and value pair in the -binary images string table. The string table is simply a collection of null -terminated strings with defined offsets in the image. The string entry allows us -to create a key-value pair from this string table. This is used for passing -arbitrary arguments to the image, such as the triple and architecture. - -All of these structures are combined to form a single binary blob, the order -does not matter because of the use of absolute offsets. This makes it easier to -extend in the future. As mentioned previously, multiple offloading images are -bundled together by simply concatenating them in this format. Because we have -the magic bytes and size of each image, we can extract them as-needed. - -Usage -===== - -This tool can be used with the following arguments. Generally information is -passed as a key-value pair to the ``image=`` argument. The ``file`` and -``triple``, arguments are considered mandatory to make a valid image. -The ``arch`` argument is suggested. - -.. code-block:: console - - OVERVIEW: A utility for bundling several object files into a single binary. - The output binary can then be embedded into the host section table - to create a fatbinary containing offloading code. - - USAGE: clang-offload-packager [options] - - OPTIONS: - - Generic Options: - - --help - Display available options (--help-hidden for more) - --help-list - Display list of available options (--help-list-hidden for more) - --version - Display the version of this program - - clang-offload-packager options: - - --image=<<key>=<value>,...> - List of key and value arguments. Required - keywords are 'file' and 'triple'. - -o <file> - Write output to <file>. - -Example -======= - -This tool simply takes many input files from the ``image`` option and creates a -single output file with all the images combined. - -.. code-block:: console - - clang-offload-packager -o out.bin --image=file=input.o,triple=nvptx64,arch=sm_70 - -The inverse operation can be performed instead by passing the packaged binary as -input. In this mode the matching images will either be placed in the output -specified by the ``file`` option. If no ``file`` argument is provided a name -will be generated for each matching image. - -.. code-block:: console - - clang-offload-packager in.bin --image=file=output.o,triple=nvptx64,arch=sm_70 diff --git a/clang/docs/OpenMPSupport.rst b/clang/docs/OpenMPSupport.rst index cf89e31a..cdb3b33 100644 --- a/clang/docs/OpenMPSupport.rst +++ b/clang/docs/OpenMPSupport.rst @@ -548,9 +548,9 @@ implementation. +-------------------------------------------------------------+---------------------------+---------------------------+--------------------------------------------------------------------------+
| Increment between places for OMP_PLACES | :none:`unclaimed` | :none:`unclaimed` | |
+-------------------------------------------------------------+---------------------------+---------------------------+--------------------------------------------------------------------------+
-| OMP_AVAILABLE_DEVICES envirable | :none:`unclaimed` | :none:`unclaimed` | |
+| OMP_AVAILABLE_DEVICES envirable | :none:`unclaimed` | :none:`unclaimed` | (should wait for "Traits for default device envirable" being done) |
+-------------------------------------------------------------+---------------------------+---------------------------+--------------------------------------------------------------------------+
-| Traits for default device envirable | :none:`unclaimed` | :none:`unclaimed` | |
+| Traits for default device envirable | :part:`in progress` | :none:`unclaimed` | ro-i |
+-------------------------------------------------------------+---------------------------+---------------------------+--------------------------------------------------------------------------+
| Optionally omit array length expression | :good:`done` | :none:`unclaimed` | (Parse) https://github.com/llvm/llvm-project/pull/148048, |
| | | | (Sema) https://github.com/llvm/llvm-project/pull/152786 |
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst index 390e0fa..9a0d69c 100644 --- a/clang/docs/ReleaseNotes.rst +++ b/clang/docs/ReleaseNotes.rst @@ -126,6 +126,8 @@ AST Dumping Potentially Breaking Changes - Pretty-printing of templates with inherited (i.e. specified in a previous redeclaration) default arguments has been fixed. +- Default arguments of template template parameters are pretty-printed now. + Clang Frontend Potentially Breaking Changes ------------------------------------------- - Members of anonymous unions/structs are now injected as ``IndirectFieldDecl`` @@ -255,10 +257,16 @@ Non-comprehensive list of changes in this release - Fixed a crash when the second argument to ``__builtin_assume_aligned`` was not constant (#GH161314) +- Introduce support for :doc:`allocation tokens <AllocToken>` to enable + allocator-level heap organization strategies. A feature to instrument all + allocation functions with a token ID can be enabled via the + ``-fsanitize=alloc-token`` flag. + New Compiler Flags ------------------ - New option ``-fno-sanitize-debug-trap-reasons`` added to disable emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``). - New option ``-fsanitize-debug-trap-reasons=`` added to control emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``). +- New options for enabling allocation token instrumentation: ``-fsanitize=alloc-token``, ``-falloc-token-max=``, ``-fsanitize-alloc-token-fast-abi``, ``-fsanitize-alloc-token-extended``. Lanai Support diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst index a8bbf14..12c2ada 100644 --- a/clang/docs/UsersManual.rst +++ b/clang/docs/UsersManual.rst @@ -2155,13 +2155,11 @@ are listed below. .. option:: -f[no-]sanitize=check1,check2,... - Turn on runtime checks for various forms of undefined or suspicious - behavior. + Turn on runtime checks or mitigations for various forms of undefined or + suspicious behavior. These are disabled by default. - This option controls whether Clang adds runtime checks for various - forms of undefined or suspicious behavior, and is disabled by - default. If a check fails, a diagnostic message is produced at - runtime explaining the problem. The main checks are: + The following options enable runtime checks for various forms of undefined + or suspicious behavior: - .. _opt_fsanitize_address: @@ -2195,6 +2193,14 @@ are listed below. - ``-fsanitize=realtime``: :doc:`RealtimeSanitizer`, a real-time safety checker. + The following options enable runtime mitigations for various forms of + undefined or suspicious behavior: + + - ``-fsanitize=alloc-token``: Enables :doc:`allocation tokens <AllocToken>` + for allocator-level heap organization strategies, such as for security + hardening. It passes type-derived token IDs to a compatible memory + allocator. Requires linking against a token-aware allocator. + There are more fine-grained checks available: see the :ref:`list <ubsan-checks>` of specific kinds of undefined behavior that can be detected and the :ref:`list <cfi-schemes>` diff --git a/clang/docs/index.rst b/clang/docs/index.rst index be654af..272ae54 100644 --- a/clang/docs/index.rst +++ b/clang/docs/index.rst @@ -40,6 +40,7 @@ Using Clang as a Compiler SanitizerCoverage SanitizerStats SanitizerSpecialCaseList + AllocToken BoundsSafety BoundsSafetyAdoptionGuide BoundsSafetyImplPlans @@ -101,7 +102,6 @@ Using Clang Tools ClangLinkerWrapper ClangNVLinkWrapper ClangOffloadBundler - ClangOffloadPackager ClangRepl ClangSYCLLinker |