aboutsummaryrefslogtreecommitdiff
path: root/llvm/tools/llvm-profdata/llvm-profdata.cpp
AgeCommit message (Collapse)AuthorFilesLines
6 daysRevert "[NFC][profdata] Apply lints and other format fixes" (#149601)Florian Mayer1-58/+62
Reverts llvm/llvm-project#149433 This broke the hwasan buildbot: https://lab.llvm.org/buildbot/#/builders/55/builds/14455
6 days[profdata] Use --hot-func-list to show all hot functions (#149428)Ellis Hoag1-28/+22
The `--hot-func-list` flag is used for sample profiles to dump the list of hot functions. Add support to dump hot functions for IRPGO profiles as well. This also removes a `priority_queue` used for `--topn`. We can instead store all functions and sort at the end before dumping. Since we are storing `StringRef`s, I believe this won't consume too much memory.
6 days[NFC][profdata] Apply lints and other format fixes (#149433)Ellis Hoag1-62/+58
Apply lints and other format fixes to `llvm/tools/llvm-profdata/llvm-profdata.cpp`. This is intended to have no functional change.
2025-06-29[llvm-profdata] Resolve tilde for weighted input filenames (#146206)Yi Kong1-1/+4
When specifying a weighted input file, the shell does not automatically expand the tilde (`~`) character in the filename because the argument is passed as a single string in the format `<weight>,<filename>`. This commit fixes the issue by using `llvm::sys::fs::expand_tilde` to explicitly resolve the tilde in the filename, ensuring that paths like `~/path/to/file` are correctly handled.
2025-05-28[MemProf] Add basic summary section support (#141805)Teresa Johnson1-0/+13
This patch adds support for a basic MemProf summary section, which is built along with the indexed MemProf profile (e.g. when reading the raw or YAML profiles), and serialized through the indexed profile just after the header. Currently only 6 fields are written, specifically the number of contexts (total, cold, hot), and the max context size (cold, warm, hot). To support forwards and backwards compatibility for added fields in the indexed profile, the number of fields serialized first. The code is written to support forwards compatibility (reading newer profiles with additional summary fields), and comments indicate how to implement backwards compatibility (reading older profiles with fewer summary fields) as needed. Support is added to print the summary as YAML comments when displaying both the raw and indexed profiles via `llvm-profdata show`. Because they are YAML comments, the YAML reader ignores these (the summary is always recomputed when building the indexed profile as described above). This necessitated moving some options and a couple of interfaces out of Analysis/MemoryProfileInfo.cpp and into the new ProfileData/MemProfSummary.cpp file, as we need to classify context hotness earlier and also compute context ids to build the summary from older indexed profiles.
2025-05-24Re-apply "[StaticDataLayout][PGO]Implement reader and writer change for data ↵Mingming Liu1-0/+4
access profiles" (#141275) Re-apply https://github.com/llvm/llvm-project/pull/139997 after fixing the use-of-uninitialized-memory error (https://lab.llvm.org/buildbot/#/builders/94/builds/7373). Tested: The error is reproduced with https://github.com/llvm/llvm-zorg/blob/main/zorg/buildbot/builders/sanitizers/buildbot_bootstrap_msan.sh without the fix, and test pass with the fix. **Original commit message:** https://github.com/llvm/llvm-project/pull/138170 introduces classes to operate on data access profiles. This change supports the read and write of `DataAccessProfData` in indexed format of MemProf (v4) as well as its the text (yaml) format. For indexed format: * InstrProfWriter owns (by `std::unique_ptr<DataAccessProfData>`) the data access profiles, and gives a non-owned copy when it calls `writeMemProf`. * MemProf v4 header has a new `uint64_t` to record the byte offset of data access profiles. This `uint64_t` field is zero if data access profile is not set (nullptr). * MemProfReader reads the offset from v4 header and de-serializes in-memory bytes into class `DataAccessProfData`. For textual format: * MemProfYAML.h adds the mapping for DAP class, and make DAP optional for both read and write. 099a0fa (by @snehasish) introduces v4 which contains CalleeGuids in CallSiteInfo, and this change changes the v4 format in place with data access profiles. The current plan is to bump the version and enable v4 profiles with both features, assuming waiting for this change won't delay the callsite change too long. --------- Co-authored-by: Kazu Hirata <kazu@google.com>
2025-05-22Revert "[StaticDataLayout][PGO]Implement reader and writer change for data ↵Mingming Liu1-4/+0
access profiles" (#141157) Reverts llvm/llvm-project#139997 Sanitizer failures (https://lab.llvm.org/buildbot/#/builders/94/builds/7373) Will fix forward later.
2025-05-22[StaticDataLayout][PGO]Implement reader and writer change for data access ↵Mingming Liu1-0/+4
profiles (#139997) https://github.com/llvm/llvm-project/pull/138170 introduces classes to operate on data access profiles. This change supports the read and write of `DataAccessProfData` in indexed format of MemProf (v4) as well as its the text (yaml) format. For indexed format: * InstrProfWriter owns (by `std::unique_ptr<DataAccessProfData>`) the data access profiles, and gives a non-owned copy when it calls `writeMemProf`. * MemProf v4 header has a new `uint64_t` to record the byte offset of data access profiles. This `uint64_t` field is zero if data access profile is not set (nullptr). * MemProfReader reads the offset from v4 header and de-serializes in-memory bytes into class `DataAccessProfData`. For textual format: * MemProfYAML.h adds the mapping for DAP class, and make DAP optional for both read and write. 099a0fa (by @snehasish) introduces v4 which contains CalleeGuids in CallSiteInfo, and this change changes the v4 format in place with data access profiles. The current plan is to bump the version and enable v4 profiles with both features, assuming waiting for this change won't delay the callsite change too long. --------- Co-authored-by: Kazu Hirata <kazu@google.com>
2025-05-01[MemProf] Add v4 which contains CalleeGuids to CallSiteInfo. (#137394)Snehasish Kumar1-1/+2
This patch adds CalleeGuids to the serialized format and increments the version number to 4. The unit tests are updated to include a new test for v4 and the YAML format is also updated to be able to roundtrip the v4 format.
2025-03-08[llvm-profdata] Fix typo in llvm-profdata (#114675)Peter Jung1-1/+1
Signed-off-by: Peter Jung <admin@ptr1337.dev>
2025-03-03[NFC]Make file-local cl::opt global variables static (#126486)chrisPyr1-108/+113
#125983
2025-01-23[llvm-profdata] Add block percent to detailed summary (#105915)Ellis Hoag1-7/+2
2024-12-11[memprof] Improve the way we express Frames in YAML (#119629)Kazu Hirata1-1/+3
This patch does two things: - During deserialization, we accept a function name for Frame as an alternative to the usual GUID expressed as a hexadecimal number. - During serialization, we print a GUID of Frame as a 16-digit hexadecimal number prefixed with 0x in the usual way. (Without this patch, we print a decimal number, which is not customary.) The patch uses a machinery called "normalization" in YAML I/O, which lets us serialize and deserialize into an alternative data structure. For our use case, we have an alternative Frame data structure, which is identical to "struct Frame" except that Function is of type GUIDHex64 instead of GlobalValue::GUID. This alternative type supports the two bullet points above without modifying "struct Frame" at all.
2024-12-11[memprof] Move YAML support to MemProfYAML.h (NFC) (#119515)Kazu Hirata1-0/+1
The YAML support is increasing in size, so this patch moves it to a separate file.
2024-12-07[memprof] Add YAML read/write support to llvm-profdata (#118915)Kazu Hirata1-12/+67
This patch adds YAML read/write support to llvm-profdata. The primary intent is to accommodate MemProf profiles in test cases, thereby avoiding the binary format. The read support is via llvm-profdata merge. This is useful when we want to verify that the compiler does the right thing on a given .ll file and a MemProf profile in a test case. In the test case, we would convert the MemProf profile in YAML to an indexed profile and invoke the compiler on the .ll file along with the indexed profile. The write support is via llvm-profdata show --memory. This is useful when we wish to convert an indexed MemProf profile to YAML while writing tests. We would compile a test case in C++, run it for an indexed MemProf profile, and then convert it to the text format.
2024-12-04[PGO] Add option to always instrumenting loop entries (#116789)ronryvchin1-1/+3
This patch extends the PGO infrastructure with an option to prefer the instrumentation of loop entry blocks. This option is a generalization of https://github.com/llvm/llvm-project/commit/19fb5b467bb97f95eace1f3637d2d1041cebd3ce, and helps to cover cases where the loop exit is never executed. An example where this can occur are event handling loops. Note that change does NOT change the default behavior.
2024-11-22[memprof] Remove MemProf format Version 1 (#117357)Kazu Hirata1-2/+1
This patch removes MemProf format Version 1 now that Version 2 and 3 are working well.
2024-11-19[memprof] Add MemProfReader::takeMemProfData (#116769)Kazu Hirata1-27/+1
This patch adds MemProfReader::takeMemProfData, a function to return the complete MemProf profile from the reader. We can directly pass its return value to InstrProfWriter::addMemProfData without having to deal with the indivual components of the MemProf profile. The new function is named "take", but it doesn't do std::move yet because of type differences (DenseMap v.s. MapVector). The end state I'm trying to get to is roughly as follows: - MemProfReader accepts IndexedMemProfData as a parameter as opposed to the three individual components (frames, call stacks, and records). - MemProfReader keeps IndexedMemProfData as a class member without decomposing it into its individual components. - MemProfReader returns IndexedMemProfData like: IndexedMemProfData takeMemProfData() { return std::move(MemProfData); }
2024-11-20[llvm-profdata] fix typo (#116754)lifengxiang10251-1/+1
2024-11-15[memprof] Remove MemProf format Version 0 (#116442)Kazu Hirata1-2/+1
This patch removes MemProf format Version 0 now that version 2 and 3 seem to be working well. I'm not touching version 1 for now because some tests still rely on version 1. Note that Version 0 is identical to Version 1 except that the MemProf section of the indexed format has a MemProf version field.
2024-10-29[MemProf] Support for random hotness when writing profile (#113998)Teresa Johnson1-1/+11
Add support for generating random hotness in the memprof profile writer, to be used for testing. The random seed is printed to stderr, and an additional option enables providing a specific seed in order to reproduce a particular random profile.
2024-10-21[tools] Don't call StringRef::str() when calling StringMap::find (NFC) (#113119)Kazu Hirata1-2/+2
StringMap::find takes StringRef. We don't need to create an instance of std::string from StringRef only to convert it right back to StringRef.
2024-10-11[llvm-profdata] Default to MemProf version 3 (#108863)Kazu Hirata1-1/+1
It's very confusing to have support for Verion 3 but not default to it. This patch teaches llvm-profdata to use MemProf version 3 by default.
2024-10-08[llvm-profdata] Avoid repeated hash lookups (NFC) (#111629)Kazu Hirata1-5/+3
2024-09-30[llvm-profdata] Fix typo in usage (#110434)Corentin Kerisit1-2/+3
From `profata` to `profdata`
2024-09-06[InstrProf] Add debuginfod correlation support (#106606)gulfemsavrun1-9/+60
This patch adds debuginfod support into llvm-profdata to find the assosicated executable by a build id in a raw profile to correlate a profile with a provided correlation kind (debug-info or binary).
2024-08-28[llvm-profdata] Enabled functionality to write split-layout profile (#101795)William Junda Huang1-0/+13
Using the flag `-split_layout` in llvm-profdata merge, the output profile can write profiles with and without inlined function into two different extbinary sections (and their FuncOffsetTable too). The section without inlined functions are marked with `SecFlagFlat` and is skipped by ThinLTO because it provides no useful info. The split layout feature was already implemented in SampleProfWriter but previously there is no way to use it from llvm-profdata.
2024-07-09[NFC] Coding style: drop `k` in `kGlobalIdentifierDelimiter` (#98230)Mircea Trofin1-1/+1
2024-07-09[NFC] Coding style fixes: SampleProf (#98208)Mircea Trofin1-2/+4
Also some control flow simplifications. Notably, this doesn't address `sampleprof_error`. I *think* the style there tries to match `std::error_category`. Also left `hash_value` as-is, because it matches what we do in Hashing.h
2024-06-26[TypeProf][InstrFDO]Omit vtable symbols in indexed profiles by default (#96520)Mingming Liu1-4/+10
- The indexed iFDO profiles contains compressed vtable names for `llvm-profdata show --show-vtables` debugging usage. An optimized build doesn't need it and doesn't decompress the blob now [1], since optimized binary has the source code and IR to find vtable symbols. - The motivation is to avoid increasing profile size when it's not necessary. - This doesn't change the indexed profile format and thereby doesn't need a version change. [1] https://github.com/llvm/llvm-project/blob/eac925fb81f26342811ad1765e8f9919628e2254/llvm/include/llvm/ProfileData/InstrProfReader.h#L696-L699
2024-06-13[llvm-profdata] Clean up traverseAllValueSites (NFC) (#95467)Kazu Hirata1-15/+15
If NV == 0, nothing interesting happens after the "if" statement. We should just "continue" to the next value site. While I am at it, this patch migrates a use of getValueForSite to getValueArrayForSite.
2024-05-28[memprof] Add MemProf format Version 3 (#93608)Kazu Hirata1-1/+2
This patch adds Version 3 for development purposes. For now, this patch adds V3 as a copy of V2. For the most part, this patch adds "case Version3:" wherever "case Version2:" appears. One exception is writeMemProfV3, which is copied from writeMemProfV2 but updated to write out memprof::Version3 to the MemProf header. We'll incrementally modify writeMemProfV3 in subsequent patches.
2024-05-23[InstrProf] Evaluate function order using test traces (#92451)Ellis Hoag1-3/+40
The `llvm-profdata order` command is used to compute a function order using traces from the input profile. Add the `--num-test-traces` flag to keep aside N traces to evalute this order. These test traces are assumed to be the actual function execution order in some experiment. The output is a number that represents how many page faults we got. Lower is better. I tested on a large profile I already had. ``` llvm-profdata order default.profdata --num-test-traces=30 # Ordered 149103 functions # Total area under the page fault curve: 2.271827e+09 ... ``` I also improved `TemporalProfTraceTy::createBPFunctionNodes()` in a few ways: * Simplified how `UN`s are computed * Change how the initial `Node` order is computed * Filter out rare and common `UN`s * Output vector is an aliased argument instead of a return These changes slightly improved the evaluation in my test. ``` llvm-profdata order default.profdata --num-test-traces=30 # Ordered 149103 functions # Total area under the page fault curve: 2.268586e+09 ... ```
2024-05-20[llvm-profdata] Fix some style and clang-tidy issuesFangrui Song1-11/+10
Fix #92761 Fix #92762
2024-05-16[InstrProf] Remove unused argv in llvm-profdata.cpp (#92435)Ellis Hoag1-12/+12
https://github.com/llvm/llvm-project/pull/71328 refactored `llvm-profdata.cpp` to use subcommands (which is super nice), but left many unused `argv` variables. This opts to use `ProgName` where necessary, and removes `argv` otherwise.
2024-04-24[memprof] Reduce schema for Version2 (#89876)Kazu Hirata1-1/+5
Curently, the compiler only uses several fields of MemoryInfoBlock. Serializing all fields into the indexed MemProf file simply wastes storage. This patch limits the schema down to four fields for Version2 by default. It retains the old behavior of serializing all fields via: llvm-profdata merge --memprof-version=2 --memprof-full-schema This patch reduces the size of the indexed MemProf profile I have by 40% (1.6GB down to 1.0GB).
2024-04-18[memprof] Add Version2 of the indexed MemProf format (#89100)Kazu Hirata1-1/+14
This patch adds Version2 of the indexed MemProf format. The new format comes with a hash table from CallStackId to actual call stacks llvm::SmallVector<FrameId>. The rest of the format refers to call stacks with CallStackId. This "values + references" model effectively deduplicates call stacks. Without this patch, a large indexed memprof file of mine shrinks from 4.4GB to 1.6GB, a 64% reduction. This patch does not make Version2 generally available yet as I am planning to make a few more changes to the format.
2024-04-11[memprof] Use structured binding (NFC) (#88363)Kazu Hirata1-2/+2
2024-04-10[memprof] Rename RawMemProfReader.{cpp,h} to MemProfReader.{cpp,h} (NFC) ↵Kazu Hirata1-1/+1
(#88200) This patch renames RawMemProfReader.{cpp,h} to MemProfReader.{cpp,h}, respectively. Also, it re-creates RawMemProfReader.h just to include MemProfReader.h for compatibility with out-of-tree users.
2024-04-01[InstrFDO][TypeProf] Implement binary instrumentation and profile read/write ↵Mingming Liu1-0/+26
(#66825) (The profile format change is split into a standalone change into https://github.com/llvm/llvm-project/pull/81691) * For InstrFDO value profiling, implement instrumentation and lowering for virtual table address. * This is controlled by `-enable-vtable-value-profiling` and off by default. * When the option is on, raw profiles will carry serialized `VTableProfData` structs and compressed vtables as payloads. * Implement profile reader and writer support * Raw profile reader is used by `llvm-profdata` but not compiler. Raw profile reader will construct InstrProfSymtab with symbol names, and map profiled runtime address to vtable symbols. * Indexed profile reader is used by `llvm-profdata` and compiler. When initialized, the reader stores a pointer to the beginning of in-memory compressed vtable names and the length of string. When used in `llvm-profdata`, reader decompress the string to show symbols of a profiled site. When used in compiler, string decompression doesn't happen since IR is used to construct InstrProfSymtab. * Indexed profile writer collects the list of vtable names, and stores that to index profiles. * Text profile reader and writer support are added but mostly follow the implementation for indirect-call value type. * `llvm-profdata show -show-vtables <args> <profile>` is implemented. rfc in https://discourse.llvm.org/t/rfc-dynamic-type-profiling-and-optimizations-in-llvm/74600#pick-instrumentation-points-and-instrument-runtime-types-7
2024-03-28[memprof] Add MemProf version (#86414)Kazu Hirata1-1/+9
This patch adds a version field to the MemProf section of the indexed profile format, calling the new version "version 1". The existing version is called "version 0". The writer supports both versions via a command-line option: llvm-profdata merge --memprof-version=1 ... The reader supports both versions by automatically detecting the version from the header.
2024-03-08[PGO] Add support for writing previous indexed format (#84505)Teresa Johnson1-2/+11
Enable temporary support to ease use of new llvm-profdata with slightly older indexed profiles after 16e74fd48988ac95551d0f64e1b36f78a82a89a2, which bumped the indexed format for type profiling.
2024-03-05Rename llvm::ThreadPool -> llvm::DefaultThreadPool (NFC) (#83702)Mehdi Amini1-1/+1
The base class llvm::ThreadPoolInterface will be renamed llvm::ThreadPool in a subsequent commit. This is a breaking change: clients who use to create a ThreadPool must now create a DefaultThreadPool instead.
2024-02-07[NFC][InstrProf]Generalize getParsedIRPGOFuncName to getParsedIRPGOName (#81054)Mingming Liu1-1/+1
- Function getParsedIRPGOFuncName splits name by delimiter. The `[filename;]mangled-name` format could be generalized for non-function global values (e.g., vtables for type profiling). So rename the function. - Use kGlobalIdentifierDelimiter rather than semicolon directly for defragmentation.
2024-01-27[llvm-profdata] Simplify a string equality comparison (NFC)Kazu Hirata1-1/+1
2024-01-23Added feature in llvm-profdata merge to filter functions from the profile ↵William Junda Huang1-3/+70
(#78378) `--function=<regex>` Include functions matching regex in the output `--no-function=<regex>` Exclude functions matching regex from the output If both are specified, `--no-function` has a higher precedence if a function name matches both filters
2024-01-11[llvm-driver] Fix usage of `InitLLVM` on Windows (#76306)Alexandre Ganea1-2/+0
Previously, some tools such as `clang` or `lld` which require strict order for certain command-line options, such as `clang -cc1` or `lld -flavor`, would not longer work on Windows, when these tools were linked as part of `llvm-driver`. This was caused by `InitLLVM` which was part of the `*_main()` function of these tools, which in turn calls `windows::GetCommandLineArguments`. That function completly replaces argc/argv by new UTF-8 contents, so any ajustements to argc/argv made by `llvm-driver` prior to calling these tools was reset. `InitLLVM` is now called by the `llvm-driver`. Any tool that participates in (or is part of) the `llvm-driver` doesn't call `InitLLVM` anymore.
2024-01-04[InstrProf] No linkage prefixes in IRPGO names (#76994)Ellis Hoag1-1/+5
Change the format of IRPGO counter names to `[<filepath>;]<mangled-name>` which is computed by `GlobalValue::getGlobalIdentifier()` to fix #74565. In fe051934cbb0aaf25d960d7d45305135635d650b (https://reviews.llvm.org/D156569) the format of IRPGO counter names was changed to be `[<filepath>;]<linkage-name>` where `<linkage-name>` is basically `F.getName()` with some prefix, e.g., `_` or `l_` on Mach-O (yes, it is confusing that `<linkage-name>` is computed with `Mangler().getNameWithPrefix()` while `<mangled-name>` is just `F.getName()`). We discovered in #74565 that this causes some missed import issues on some targets and #74008 is a partial fix. Since `<mangled-name>` may not match the `<linkage-name>` on some targets like Mach-O, we will need to post-process the output of `llvm-profdata order` before passing to the linker via `-order_file`. Profiles generated after fe051934cbb0aaf25d960d7d45305135635d650b will become stale after this diff, but I think this is acceptable since that patch landed after the LLVM 18 cut which hasn't been released yet.
2024-01-04[llvm-profdata] Use semicolon as the delimiter for supplementary profiles. ↵Mingming Liu1-8/+9
(#75080) When merging instrFDO profiles with afdo profile as supplementary, instrFDO counters for static functions are stored with function's PGO name (with filename.cpp; prefix). - This pull request fixes the delimiter used when a PGO function name is 'normalized' for AFDO look-up.
2023-12-21[llvm-profdata] Modernize FuncSampleStats, ValueSitesStats, and HotFuncInfo ↵Kazu Hirata1-16/+13
(NFC)