diff options
author | Mingming Liu <mingmingl@google.com> | 2025-09-12 15:58:16 -0700 |
---|---|---|
committer | GitHub <noreply@github.com> | 2025-09-12 15:58:16 -0700 |
commit | 52c583b3f95a0e666ab837e39a5db900b66adf15 (patch) | |
tree | 1da839e57d9c62fa64056c366143fb4f095f7f59 /llvm/lib/ProfileData/SampleProf.cpp | |
parent | 1131e44ed3f5fadb2d22ff155d4e47f69757d02f (diff) | |
download | llvm-52c583b3f95a0e666ab837e39a5db900b66adf15.zip llvm-52c583b3f95a0e666ab837e39a5db900b66adf15.tar.gz llvm-52c583b3f95a0e666ab837e39a5db900b66adf15.tar.bz2 |
[SampleFDO][TypeProf]Support vtable type profiling for ext-binary and text format (#148002)
This change extends SampleFDO ext-binary and text format to record the
vtable symbols and their counts for virtual calls inside a function. The
vtable profiles will allow the compiler to annotate vtable types on IR
instructions and perform vtable-based indirect call promotion. An RFC is
in
https://discourse.llvm.org/t/rfc-vtable-type-profiling-for-samplefdo/87283
Given a function below, the before vs after of a function's profile is
illustrated in text format in the table:
```
__attribute__((noinline)) int loop_func(int i, int a, int b) {
Base *ptr = createType(i);
int sum = ptr->func(a, b);
delete ptr;
return sum;
}
```
| before | after |
| --- | --- |
| Samples collected in the function's body { <br> 0: 636241 <br> 1:
681458, calls: _Z10createTypei:681458 <br> 3: 543499, calls:
_ZN12_GLOBAL__N_18Derived24funcEii:410621 _ZN8Derived14funcEii:132878
<br> 5.1: 602201, calls: _ZN12_GLOBAL__N_18Derived2D0Ev:454635
_ZN8Derived1D0Ev:147566 <br> 7: 511057 <br> } | Samples collected in the
function's body { <br> 0: 636241 <br> 1: 681458, calls:
_Z10createTypei:681458 <br> 3: 543499, calls:
_ZN12_GLOBAL__N_18Derived24funcEii:410621 _ZN8Derived14funcEii:132878
<br> 3: vtables: _ZTV8Derived1:1377 _ZTVN12_GLOBAL__N_18Derived2E:4250
<br> 5.1: 602201, calls: _ZN12_GLOBAL__N_18Derived2D0Ev:454635
_ZN8Derived1D0Ev:147566 <br> 5.1: vtables: _ZTV8Derived1:227
_ZTVN12_GLOBAL__N_18Derived2E:765 <br> 7: 511057 <br> } |
Key points for this change:
1. In-memory representation of vtable profiles
* A field of type `map<LineLocation, map<FunctionId, uint64_t>>` is
introduced in a function's in-memory representation
[FunctionSamples](https://github.com/llvm/llvm-project/blob/ccc416312ed72e92a885425d9cb9c01f9afa58eb/llvm/include/llvm/ProfileData/SampleProf.h#L749-L754).
2. The vtable counters for one LineLocation represents the relative
frequency among vtables for this LineLocation. They are not required to
be comparable across LineLocations.
3. For backward compatibility of ext-binary format, we take one bit from
ProfSummaryFlag as illustrated in the enum class `SecProfSummaryFlags`.
The ext-binary profile reader parses the integer type flag and reads
this bit. If it's set, the profile reader will parse vtable profiles.
4. The vtable profiles are optional in ext-binary format, and not
serialized out by default, we introduce an LLVM boolean option (named
`-extbinary-write-vtable-type-prof`). The ext-binary profile writer
reads the boolean option and decide whether to set the section flag bit
and serialize the in-memory class members corresponding to vtables.
5. This change doesn't implement `llvm-profdata overlap --sample` for
the vtable profiles. A subsequent change will do it to keep this one
focused on the profile format change.
We don't plan to add the vtable support to non-extensible format mainly
because of the maintenance cost to keep backward compatibility for prior
versions of profile data.
* Currently, the [non-extensible binary
format](https://github.com/llvm/llvm-project/blob/5c28af409978c19a35021855a29dcaa65e95da00/llvm/lib/ProfileData/SampleProfWriter.cpp#L899-L900)
does not have feature parity with extensible binary format today, for
instance, the former doesn't support [profile symbol
list](https://github.com/llvm/llvm-project/blob/41e22aa31b1905aa3e9d83c0343a96ec0d5187ec/llvm/include/llvm/ProfileData/SampleProf.h#L1518-L1522)
or context-sensitive PGO, both of which give measurable performance
boost. Presumably the non-extensible format is not in wide use.
---------
Co-authored-by: Paschalis Mpeis <paschalis.mpeis@arm.com>
Diffstat (limited to 'llvm/lib/ProfileData/SampleProf.cpp')
-rw-r--r-- | llvm/lib/ProfileData/SampleProf.cpp | 40 |
1 files changed, 40 insertions, 0 deletions
diff --git a/llvm/lib/ProfileData/SampleProf.cpp b/llvm/lib/ProfileData/SampleProf.cpp index 60c1393..ac7513e 100644 --- a/llvm/lib/ProfileData/SampleProf.cpp +++ b/llvm/lib/ProfileData/SampleProf.cpp @@ -47,6 +47,24 @@ bool FunctionSamples::ProfileIsPreInlined = false; bool FunctionSamples::UseMD5 = false; bool FunctionSamples::HasUniqSuffix = true; bool FunctionSamples::ProfileIsFS = false; + +std::error_code +serializeTypeMap(const TypeCountMap &Map, + const MapVector<FunctionId, uint32_t> &NameTable, + raw_ostream &OS) { + encodeULEB128(Map.size(), OS); + for (const auto &[TypeName, SampleCount] : Map) { + if (auto NameIndexIter = NameTable.find(TypeName); + NameIndexIter != NameTable.end()) { + encodeULEB128(NameIndexIter->second, OS); + } else { + // If the type is not in the name table, we cannot serialize it. + return sampleprof_error::truncated_name_table; + } + encodeULEB128(SampleCount, OS); + } + return sampleprof_error::success; +} } // namespace sampleprof } // namespace llvm @@ -178,6 +196,17 @@ raw_ostream &llvm::sampleprof::operator<<(raw_ostream &OS, return OS; } +static void printTypeCountMap(raw_ostream &OS, LineLocation Loc, + const TypeCountMap &TypeCountMap) { + if (TypeCountMap.empty()) { + return; + } + OS << Loc << ": vtables: "; + for (const auto &[Type, Count] : TypeCountMap) + OS << Type << ":" << Count << " "; + OS << "\n"; +} + /// Print the samples collected for a function on stream \p OS. void FunctionSamples::print(raw_ostream &OS, unsigned Indent) const { if (getFunctionHash()) @@ -192,7 +221,13 @@ void FunctionSamples::print(raw_ostream &OS, unsigned Indent) const { SampleSorter<LineLocation, SampleRecord> SortedBodySamples(BodySamples); for (const auto &SI : SortedBodySamples.get()) { OS.indent(Indent + 2); + const auto &Loc = SI->first; OS << SI->first << ": " << SI->second; + if (const TypeCountMap *TypeCountMap = + this->findCallsiteTypeSamplesAt(Loc)) { + OS.indent(Indent + 2); + printTypeCountMap(OS, Loc, *TypeCountMap); + } } OS.indent(Indent); OS << "}\n"; @@ -214,6 +249,11 @@ void FunctionSamples::print(raw_ostream &OS, unsigned Indent) const { OS << Loc << ": inlined callee: " << FuncSample.getFunction() << ": "; FuncSample.print(OS, Indent + 4); } + auto TypeSamplesIter = VirtualCallsiteTypeCounts.find(Loc); + if (TypeSamplesIter != VirtualCallsiteTypeCounts.end()) { + OS.indent(Indent + 2); + printTypeCountMap(OS, Loc, TypeSamplesIter->second); + } } OS.indent(Indent); OS << "}\n"; |