aboutsummaryrefslogtreecommitdiff
path: root/llvm/tools/llvm-link/llvm-link.cpp
AgeCommit message (Collapse)AuthorFilesLines
2025-06-12[DebugInfo][RemoveDIs] Delete debug-info-format flag (#143746)Jeremy Morse1-7/+1
This flag was used to let us incrementally introduce debug records into LLVM, however everything is now using records. It serves no purpose now, so delete it.
2025-06-09[DebugInfo][RemoveDIs] Rip out the UseNewDbgInfoFormat flag (#143207)Jeremy Morse1-4/+2
Start removing debug intrinsics support -- starting with the flag that controls production of their replacement, debug records. This patch removes the command-line-flag and with it the ability to switch back to intrinsics. The module / function / block level "IsNewDbgInfoFormat" flags get hardcoded to true, I'll to incrementally remove things that depend on those flags.
2025-04-09[DebugInfo][RemoveDIs] Eliminate another debug-info variation flag (#133917)Jeremy Morse1-6/+0
The "preserve input debug-info format" flag allowed some tooling to opt into not seeing the new debug records yet, and to not autoupgrade. This was good at the time, but un-necessary now that we'll be ditching intrinsics shortly. It also hides errors now: verify-uselistorder was hardcoding this flag to on, and as a result it hasn't seen debug records before. Thus, we missed a uselistorder variation: constant-expressions such as GEPs can be contained within debug records and completely isolated from the value hierachy, see the metadata-use-uselistorder.ll test. These Values didn't get ordered, but were legitimate uses of constants like "i64 0", and we now run into difficulty handling that. The patch to AsmWriter seeks Values to order even through debug-info now. Finally there are a few intrinsics-tests relying on this flag that we can just delete, such as one in llvm-reduce and another few in the LocalTest unit tests. For the fast-isel test, it was added in https://reviews.llvm.org/D67703 explicitly for checking the size of blocks without debug-info and in 1525abb9c94 the codepath it tests moved towards being sunsetted. It'll be totally redundant once RemoveDIs is on permanently. Note that there's now no explicit test for the textual-IR autoupgrade path. I submit that we can rely on the thousands of .ll files where we've only been bothered to update the outputs, not the inputs, to debug records.
2025-04-01[DebugInfo][RemoveDIs] Remove debug-intrinsic printing cmdline options (#131855)Jeremy Morse1-10/+2
During the transition from debug intrinsics to debug records, we used several different command line options to customise handling: the printing of debug records to bitcode and textual could be independent of how the debug-info was represented inside a module, whether the autoupgrader ran could be customised. This was all valuable during development, but now that totally removing debug intrinsics is coming up, this patch removes those options in favour of a single flag (experimental-debuginfo-iterators), which enables autoupgrade, in-memory debug records, and debug record printing to bitcode and textual IR. We need to do this ahead of removing the experimental-debuginfo-iterators flag, to reduce the amount of test-juggling that happens at that time. There are quite a number of weird test behaviours related to this -- some of which I simply delete in this commit. Things like print-non-instruction-debug-info.ll , the test suite now checks for debug records in all tests, and we don't want to check we can print as intrinsics. Or the update_test_checks tests -- these are duplicated with write-experimental-debuginfo=false to ensure file writing for intrinsics is correct, but that's something we're imminently going to delete. A short survey of curious test changes: * free-intrinsics.ll: we don't need to test that debug-info is a zero cost intrinsic, because we won't be using intrinsics in the future. * undef-dbg-val.ll: apparently we pinned this to non-RemoveDIs in-memory mode while we sorted something out; it works now either way. * salvage-cast-debug-info.ll: was testing intrinsics-in-memory get salvaged, isn't necessary now * localize-constexpr-debuginfo.ll: was producing "dead metadata" intrinsics for optimised-out variable values, dbg-records takes the (correct) representation of poison/undef as an operand. Looks like we didn't update this in the past to avoid spurious test differences. * Transforms/Scalarizer/dbginfo.ll: this test was explicitly testing that debug-info affected codegen, and we deferred updating the tests until now. This is just one of those silent gnochange issues that get fixed by RemoveDIs. Finally: I've added a bitcode test, dbg-intrinsics-autoupgrade.ll.bc, that checks we can autoupgrade debug intrinsics that are in bitcode into the new debug records.
2025-03-14[RemoveDIs] Remove "try-debuginfo-iterators..." test flags (#130298)Jeremy Morse1-5/+0
These date back to when the non-intrinsic format of variable locations was still being tested and was behind a compile-time flag, so not all builds / bots would correctly run them. The solution at the time, to get at least some test coverage, was to have tests opt-in to non-intrinsic debug-info if it was built into LLVM. Nowadays, non-intrinsic format is the default and has been on for more than a year, there's no need for this flag to exist. (I've downgraded the flag from "try" to explicitly requesting non-intrinsic format in some places, so that we can deal with tests that are explicitly about non-intrinsic format in their own commit).
2025-01-06[nfc][thinlto] remove unnecessary return from `renameModuleForThinLTO` (#121851)Mircea Trofin1-3/+2
Same goes for `FunctionImportGlobalProcessing::run`. The return value was used, but it was always `false`.
2024-09-25[SystemZ][z/OS] Open text files in text mode (#109972)Abhina Sree1-3/+3
This patch continues the work that was started here https://reviews.llvm.org/D99426 to correctly open text files in text mode.
2024-09-01[LTO] Reduce memory usage for import lists (#106772)Kazu Hirata1-1/+2
This patch reduces the memory usage for import lists by employing memory-efficient data structures. With this patch, an import list for a given destination module is basically DenseSet<uint32_t> with each element indexing into the deduplication table containing tuples of: {SourceModule, GUID, Definition/Declaration} In one of our large applications, the peak memory usage goes down by 9.2% from 6.120GB to 5.555GB during the LTO indexing step. This patch addresses several sources of space inefficiency associated with std::unordered_map: - std::unordered_map<GUID, ImportKind> takes up 16 bytes because of padding even though ImportKind only carries one bit of information. - std::unordered_map uses pointers to elements, both in the hash table proper and for collision chains. - We allocate an instance of std::unordered_map for each {Destination Module, Source Module} pair for which we have at least one import. Most import lists have less than 10 imports, so the metadata like the size of std::unordered_map and the pointer to the hash table costs a lot relative to the actual contents.
2024-08-22[LTO] Turn ImportMapTy into a proper class (NFC) (#105748)Kazu Hirata1-3/+2
This patch turns type alias ImportMapTy into a proper class to provide a more intuitive interface like: ImportList.addDefinition(...) as opposed to: FunctionImporter::addDefinition(ImportList, ...) Also, this patch requires all non-const accesses to go through addDefinition, maybeAddDeclaration, and addGUID while providing const accesses via: const ImportMapTyImpl &getImportMap() const { return ImportMap; } I realize ImportMapTy may not be the best name as a class (maybe OK as a type alias). I am not renaming ImportMapTy in this patch at least because there are 47 mentions of ImportMapTy under llvm/.
2024-08-22[LTO] Introduce helper functions to add GUIDs to ImportList (NFC) (#105555)Kazu Hirata1-3/+3
The new helper functions make the intent clearer while hiding implementation details, including how we handle previously added entries. Note that: - If we are adding a GUID as a GlobalValueSummary::Definition, then we override a previously added GlobalValueSummary::Declaration entry for the same GUID. - If we are adding a GUID as a GlobalValueSummary::Declaration, then a previously added GlobalValueSummary::Definition entry for the same GUID takes precedence, and no change is made.
2024-07-06[llvm] Remove redundant calls to std::unique_ptr<T>::get (NFC) (#97778)Kazu Hirata1-2/+2
2024-06-20Reland "[ThinLTO] Populate declaration import status except for distributed ↵Mingming Liu1-1/+5
ThinLTO under a default-off new option" (#95482) Make `FunctionsToImportTy` an `unordered_map` rather than `DenseMap`. Credit goes to jvoung@ for the 'DenseMap -> unordered_map' change. This is a reland of https://github.com/llvm/llvm-project/pull/92718 * `DenseMap` allocates space for a large number of key/value pairs and wastes space when the number of elements are small. * While init bucket size is zero [1], it quickly allocates buckets for 64 elements [2] when the number of elements is small (for example, 3 or 4 elements). The programmer manual [3] also mentions it could waste space. * Experiments show `FunctionsToImportTy.size()` is smaller than 4 for multiple binaries with high indexing ram usage. `unordered_map` grows factor is at most 2 in llvm libc [4] for insert operations. With this change, `ComputeCrossModuleImport` ram increase is smaller than 0.5G on a couple of binaries with high indexing ram usage. A wider range of (pre-release) tests pass. [1] https://github.com/llvm/llvm-project/blob/ad79a14c9e5ec4a369eed4adf567c22cc029863f/llvm/include/llvm/ADT/DenseMap.h#L431-L432 [2] https://github.com/llvm/llvm-project/blob/ad79a14c9e5ec4a369eed4adf567c22cc029863f/llvm/include/llvm/ADT/DenseMap.h#L849 [3] https://llvm.org/docs/ProgrammersManual.html#llvm-adt-densemap-h [4] https://github.com/llvm/llvm-project/blob/ad79a14c9e5ec4a369eed4adf567c22cc029863f/libcxx/include/__hash_table#L1525-L1526 **Original commit message** The goal is to populate `declaration` import status if a new flag `-import-declaration` is on. * For in-process ThinLTO, the `declaration` status is visible to backend `function-import` pass, so `FunctionImporter::importFunctions` should read the import status and be no-op for declaration summaries. Basically, the postlink pipeline is updated to keep its current behavior (import definitions), but not updated to handle `declaration` summaries. Two use cases ([better call-graph sort](https://discourse.llvm.org/t/rfc-for-better-call-graph-sort-build-a-more-complete-call-graph-by-adding-more-indirect-call-edges/74029#support-cross-module-function-declaration-import-5) or [cross-module auto-init](https://github.com/llvm/llvm-project/pull/87597#discussion_r1556067195)) would use this bit differently. * For distributed ThinLTO, the `declaration` status is not serialized to bitcode. As discussed, https://github.com/llvm/llvm-project/pull/87600 will do this.
2024-06-14Reapply#4 "[RemoveDIs] Load into new debug info format by default in LLVM ↵Stephen Tozer1-7/+1
(#89799)" Reapplies commit c5aeca73 (and its followup commit 21396be8), which were reverted due to missing functionality in MLIR and Flang regarding printing debug records. This has now been added in commit 08aa511, along with support for printing debug records in flang. This reverts commit 2dc2290860355dd2bac3b655eea895fe30fde257.
2024-06-11Revert new debug info format commits:Stephen Tozer1-1/+7
"[Flang] Update test to not check for tail calls on debug intrinsics" & "Reapply#3 "[RemoveDIs] Load into new debug info format by default in LLVM (#89799)" Recent updates to flang have added debug info generation via MLIR, a path which currently does not support debug records. The patch that enables debug records by default (and a small followup patch) are thus being reverted until the MLIR path has been fixed. This reverts commits: 21396be865b4640abf6afa0b05de6708a1a996e0 c5aeca732d1ff6769b0659efebd1cfb5f60487e4
2024-06-10Reapply#3 "[RemoveDIs] Load into new debug info format by default in LLVM ↵Stephen Tozer1-7/+1
(#89799)" Reapplies commit 91446e2, which was reverted due to a downstream error, discussed on the pull request. The error could not be reproduced upstream, and cannot be reproduced downstream as-of current main, so until the error can be confirmed to still exist this patch should return. This reverts commit 23f8fac745bdde70ed4f9c585d19c4913734f1b8.
2024-06-05Revert "Reland "[ThinLTO] Populate declaration import status except for ↵Mingming Liu1-5/+1
distributed ThinLTO under a default-off new option" (#92718) (#94503) This reverts commit e33db249b53fb70dce62db3ebd82d42239bd1d9d. The change from *set to *map increases memory usage, and caused indexing OOM in some applications. Need to profile offline to bring the memory usage down.
2024-05-20Reland "[ThinLTO] Populate declaration import status except for distributed ↵Mingming Liu1-1/+5
ThinLTO under a default-off new option" (#92718) The original PR is reviewed in https://github.com/llvm/llvm-project/pull/88024, and this PR adds one line (https://github.com/llvm/llvm-project/pull/92718/commits/b9f04d199dec4f3c221d981dcb91e55298d0693f) to fix test Limit to one thread for in-process ThinLTO to test `LLVM_DEBUG` log. - This should fix build bot failure like https://lab.llvm.org/buildbot/#/builders/259/builds/4727 and https://lab.llvm.org/buildbot/#/builders/9/builds/43876 - I could repro the failure and see interleaved log messages by using `-thinlto-threads=all` **Original Commit Message:** The goal is to populate `declaration` import status if a new flag `-import-declaration` is on. * For in-process ThinLTO, the `declaration` status is visible to backend `function-import` pass, so `FunctionImporter::importFunctions` should read the import status and be no-op for declaration summaries. Basically, the postlink pipeline is updated to keep its current behavior (import definitions), but not updated to handle `declaration` summaries. Two use cases ([better call-graph sort](https://discourse.llvm.org/t/rfc-for-better-call-graph-sort-build-a-more-complete-call-graph-by-adding-more-indirect-call-edges/74029#support-cross-module-function-declaration-import-5) or [cross-module auto-init](https://github.com/llvm/llvm-project/pull/87597#discussion_r1556067195)) would use this bit differently. * For distributed ThinLTO, the `declaration` status is not serialized to bitcode. As discussed, https://github.com/llvm/llvm-project/pull/87600 will do this.
2024-05-19Revert "[ThinLTO] Populate declaration import status except for distributed ↵Mingming Liu1-5/+1
ThinLTO under a default-off new option" (#92715) Reverts llvm/llvm-project#88024 Build bot failures (https://lab.llvm.org/buildbot/#/builders/259/builds/4727 and https://lab.llvm.org/buildbot/#/builders/9/builds/43876)
2024-05-19[ThinLTO] Populate declaration import status except for distributed ThinLTO ↵Mingming Liu1-1/+5
under a default-off new option (#88024) The goal is to populate `declaration` import status if a new flag`-import-declaration` is on. * For in-process ThinLTO, the `declaration` status is visible to backend `function-import` pass, so `FunctionImporter::importFunctions` should read the import status and be no-op for declaration summaries. Basically, the postlink pipeline is updated to keep its current behavior (import definitions), but not updated to handle `declaration` summaries. Two use cases (better call-graph sort and cross-module auto-init) would use this bit differently. * For distributed ThinLTO, the `declaration` status is not serialized to bitcode. As discussed, https://github.com/llvm/llvm-project/pull/87600 will do this. [1] https://discourse.llvm.org/t/rfc-for-better-call-graph-sort-build-a-more-complete-call-graph-by-adding-more-indirect-call-edges/74029#support-cross-module-function-declaration-import-5 [2] https://github.com/llvm/llvm-project/pull/87597#discussion_r1556067195
2024-05-13Revert "Repply#2 "[RemoveDIs] Load into new debug info format by default in ↵Fangrui Song1-1/+7
LLVM (#89799)"" This reverts commit 91446e2aa687ec57ad88dc0df793d0c6e694a7c9 and a unittest followup 1530f319311908b06fe935c89fca692d3e53184f (#90476). In a stage-2 -flto=thin -gsplit-dwarf -g -fdebug-info-for-profiling -fprofile-sample-use= build of clang, a ThinLTO backend compile has assertion failures: Global is external, but doesn't have external or weak linkage! ptr @_ZN5clang12ast_matchers8internal18makeAllOfCompositeINS_8QualTypeEEENS1_15BindableMatcherIT_EEN4llvm8ArrayRefIPKNS1_7MatcherIS5_EEEE function declaration may only have a unique !dbg attachment ptr @_ZN5clang12ast_matchers8internal18makeAllOfCompositeINS_8QualTypeEEENS1_15BindableMatcherIT_EEN4llvm8ArrayRefIPKNS1_7MatcherIS5_EEEE The failures somehow go away if -fprofile-sample-use= is removed.
2024-05-03Repply#2 "[RemoveDIs] Load into new debug info format by default in LLVM ↵Stephen Tozer1-7/+1
(#89799)" Reapplies the original commit: 2f01fd99eb8c8ab3db9aba72c4f00e31e9e60a05 The previous application of this patch failed due to some missing DbgVariableRecord support in clang, which has been added now by commit 8805465e. This will probably break some downstream tools that don't already handle debug records. If your downstream code breaks as a result of this change, the simplest fix is to convert the module in question to the old debug format before you process it, using `Module::convertFromNewDbgValues()`. For more information about how to handle debug records or about what has changed, see the migration document: https://llvm.org/docs/RemoveDIsDebugInfo.html This reverts commit 4fd319ae273ed6c252f2067909c1abd9f6d97efa.
2024-05-02Revert#2 "[RemoveDIs] Load into new debug info format by default in LLVM ↵Stephen Tozer1-1/+7
(#89799)" Reverted following probably-causing failures on some clang buildbots: https://lab.llvm.org/buildbot/#/builders/245/builds/24037 This reverts commit a12622543de15df45fb9ad64e8ab723289d55169.
2024-05-02Reapply "[RemoveDIs] Load into new debug info format by default in LLVM ↵Stephen Tozer1-7/+1
(#89799)" Fixes the broken tests in the original commit: 2f01fd99eb8c8ab3db9aba72c4f00e31e9e60a05 This will probably break some downstream tools that don't already handle debug records. If your downstream code breaks as a result of this change, the simplest fix is to convert the module in question to the old debug format before you process it, using `Module::convertFromNewDbgValues()`. For more information about how to handle debug records or about what has changed, see the migration document: https://llvm.org/docs/RemoveDIsDebugInfo.html This reverts commit 00821fed09969305b0003d3313c44d1e761a7131.
2024-05-01Revert "[RemoveDIs] Load into new debug info format by default in LLVM (#89799)"Stephen Tozer1-1/+7
A unit test was broken by the above commit: https://lab.llvm.org/buildbot/#/builders/139/builds/64627 This reverts commit 2f01fd99eb8c8ab3db9aba72c4f00e31e9e60a05.
2024-05-01[RemoveDIs] Load into new debug info format by default in LLVM (#89799)Stephen Tozer1-7/+1
This patch enables parsing and creating modules directly into the new debug info format. Prior to this patch, all modules were constructed with the old debug info format by default, and would be converted into the new format just before running LLVM passes. This is an important milestone, in that this means that every tool will now be exposed to debug records, rather than those that run LLVM passes. As far as I've tested, all LLVM tools/projects now either handle debug records, or convert them to the old intrinsic format. There are a few unit tests that need updating for this patch; these are either cases of tests that previously needed to set the debug info format to function, or tests that depend on the old debug info format in some way. There should be no visible change in the output of any LLVM tool as a result of this patch, although the likelihood of this patch breaking downstream code means an NFC tag might be a little misleading, if not technically incorrect: This will probably break some downstream tools that don't already handle debug records. If your downstream code breaks as a result of this change, the simplest fix is to convert the module in question to the old debug format before you process it, using `Module::convertFromNewDbgValues()`. For more information about how to handle debug records or about what has changed, see the migration document: https://llvm.org/docs/RemoveDIsDebugInfo.html
2024-04-22[RemoveDIs] Make verify-uselistorder preserve the input debug info format ↵Stephen Tozer1-1/+11
(#87789) Verify-uselistorder wants to take some input IR and verify that the uselist order is stable after roundtripping to bitcode and assembly. This is disrupted if the file is converted between the new and old debug info formats after parsing - while there's no functional difference, the change to the in-memory representation of the IR modifies the uselist. This patch changes verify-uselistorder to not convert input files between debug info formats by default, preventing changes from being made to the file being checked. In addition, this patch makes it so that when we _do_ print IR in the new debug info format to bitcode or assembly, we delete any lingering debug intrinsic declarations, ensuring that we don't write uselist entries for them.
2024-04-05[RemoveDIs] Add flag to preserve the debug info format of input IR (#87379)Stephen Tozer1-0/+5
This patch adds a new flag: `--preserve-input-debuginfo-format` This flag instructs the tool to not convert the debug info format (intrinsics/records) of input IR, but to instead determine the format of the input IR and overwrite the other format-determining flags so that we process and output the file in the same format that we received it in. This flag is turned off by llvm-link, llvm-lto, and llvm-lto2, and should be turned off by any other tool that expects to parse multiple IR modules and have their debug info formats match. The motivation for this flag is to allow tools to not convert the debug info format - verify-uselistorder and llvm-reduce, and any downstream tools that seek to test or mutate IR as-is, without applying extraneous modifications to the input. This is a necessary step to using debug records by default in all (other) LLVM tools.
2024-03-25[RemoveDIs] Load into new debug info format by default in llvm-link (#86274)Orlando Cazalet-Hyams1-0/+6
Directly load all bitcode into the new debug info format in llvm-link. This means that new-mode bitcode no longer round-trips back to old-mode after parsing, and that old-mode bitcode gets auto-upgraded to new-mode debug info (which is the current in-memory default in LLVM).
2024-02-23[llvm-link] Improve missing file error message (#82514)Michael Halkenhäuser1-1/+9
Add error messages showing the missing filenames. Currently, we only get 'No such file or directory' without any(!) further info. This patch will (only upon ENOENT error) iterate over all requested files and print which ones are actually missing.
2024-02-08[NFC][RemoveDIs] Remove conditional compilation for RemoveDIs (#81149)Jeremy Morse1-6/+2
A colleague observes that switching the default value of LLVM_EXPERIMENTAL_DEBUGINFO_ITERATORS to "On" hasn't flipped the value in their CMakeCache.txt. This probably means that everyone with an existing build tree is going to not have support built in, meaning everyone in LLVM would need to clean+rebuild their worktree when we flip the switch on... which doesn't sound good. So instead, just delete the flag and everything it does, making everyone build and run ~400 lit tests in RemoveDIs mode. None of the buildbots have had trouble with this, so it Should Be Fine (TM). (Sending for review as this is changing various comments, and touches several different areas -- I don't want to get too punchy).
2024-01-25Reapply 215b8f1e252, reverted in c3f7fb1421eJeremy Morse1-0/+18
Turns out I was using DbgMarker::getDbgValueRange rather than the helper utility in Instruction::getDbgValueRange, which checks for null-ness. Original commit message follows. [DebugInfo][RemoveDIs] Convert debug-info modes when loading bitcode (#78967) As part of eliminating debug-intrinsics in LLVM, we'll shortly be pushing the conversion from "old" dbg.value mode to "new" DPValue mode out from when the pass manager runs, to when modules are loaded. This patch adds that conversion process and some (temporary) options to llvm-lto{,2} to help test it. Specifically: now whenever we load a bitcode module, consider a flag of whether to "upgrade" it into the new debug-info mode, and if we're lazily materializing functions then do that lazily too. Doing this exposes an error in the IRLinker/materializer handling of DPValues, where we need to transfer the debug-info format flag correctly, and in ValueMapper we need to remap the Values that DPValues point at. I've added some test coverage in the modified tests; these will be exercised by our llvm-new-debug-iterators buildbot. This upgrading of debug-info won't be happening for the llvm18 release, instead we'll turn it on after the branch date, thenbe push the boundary of where "new" debug-info starts and ends down into the existing debug-info upgrade path over the course of the next release.
2024-01-25Revert "[DebugInfo][RemoveDIs] Convert debug-info modes when loading bitcode ↵Jeremy Morse1-18/+0
(#78967)" This reverts commit 215b8f1e252b4f30cf1b734faa370c0ac4b88659. Numerous builders exploded from this X_X, for example https://lab.llvm.org/buildbot/#/builders/46/builds/62657
2024-01-25[DebugInfo][RemoveDIs] Convert debug-info modes when loading bitcode (#78967)Jeremy Morse1-0/+18
As part of eliminating debug-intrinsics in LLVM, we'll shortly be pushing the conversion from "old" dbg.value mode to "new" DPValue mode out from when the pass manager runs, to when modules are loaded. This patch adds that conversion process and some (temporary) options to llvm-lto{,2} to help test it. Specifically: now whenever we load a bitcode module, consider a flag of whether to "upgrade" it into the new debug-info mode, and if we're lazily materializing functions then do that lazily too. Doing this exposes an error in the IRLinker/materializer handling of DPValues, where we need to transfer the debug-info format flag correctly, and in ValueMapper we need to remap the Values that DPValues point at. I've added some test coverage in the modified tests; these will be exercised by our llvm-new-debug-iterators buildbot. This upgrading of debug-info won't be happening for the llvm18 release, instead we'll turn it on after the branch date, thenbe push the boundary of where "new" debug-info starts and ends down into the existing debug-info upgrade path over the course of the next release.
2023-08-04[FunctionImport] Reduce string duplication (NFC)Teresa Johnson1-1/+7
The import/export maps, and the ModuleToDefinedGVSummaries map, are all indexed by module paths, which are StringRef obtained from the module summary index, which already has a data structure than owns these strings (the ModulePathStringTable). Because these other maps are also StringMap, which makes a copy of the string key, we were keeping multiple extra copies of the module paths, leading to memory overhead. Change these to DenseMap keyed by StringRef, and document that the strings are owned by the index. The only exception is the llvm-link tool which synthesizes an import list from command line options, and I have added a string cache to maintain ownership there. I measured around 5% memory reduction in the thin link of a large binary. Differential Revision: https://reviews.llvm.org/D156580
2022-11-21[llvm-link] Fix options of llvm-linkStephen Neuendorffer1-2/+3
This tool only parsed options after creating the LLVMContext. Unfortunately, this means that some options, such as --opaque-pointers, which are read when the LLVMContext is created are impossible to set from the command line. This patch moves the LLVMContext creation after the option parsing.
2022-06-28Avoid ar_to_bc.sh script in symbolizer build; add -ignore-non-bitcode flag ↵Matthias Braun1-0/+7
to llvm-link Remove the `ar_to_bc.sh` helper script in the compiler-rt symbolizer build. Instead use `llvm-link` directly as D81109 introduced the capability to read archives. I just needed to add a new flag `-ignore-non-bitcode` to avoid `llvm-link` aborting when it found a non-bitcode file in the archive. Differential Revision: https://reviews.llvm.org/D128729
2022-06-05Remove unneeded cl::ZeroOrMore for cl::opt/cl::list optionsFangrui Song1-1/+1
2022-06-04Remove unneeded cl::ZeroOrMore for cl::list optionsFangrui Song1-1/+1
2022-05-16[AIX] llvm-link support big archive.zhijian1-4/+9
Summary: Use object::Archive::create so that the returned archive object has a dynamic type of either Archive or BigArchive. Reviewers: James Henderson,Fangrui Song Differential Revision: https://reviews.llvm.org/D124940
2021-07-21[llvm][tools] Hide more unrelated LLVM tool optionsTimm Bäder1-21/+35
Differential Revision: https://reviews.llvm.org/D106366
2021-04-28NFC: Run clang-format over llvm-link.Nick Lewycky1-34/+30
2021-04-06[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag ↵Abhina Sreeskantharajan1-1/+2
instead of OF_Text Problem: On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable. Solution: This patch adds two new flags - OF_CRLF which indicates that CRLF translation is used. - OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation. Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF. So this is the behaviour per platform with my patch: z/OS: OF_None: open in binary mode OF_Text : open in text mode OF_TextWithCRLF: open in text mode Windows: OF_None: open file with no carriage return OF_Text: open file with no carriage return OF_TextWithCRLF: open file with carriage return The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set. ``` if (Flags & OF_CRLF) CrtOpenFlags |= _O_TEXT; ``` These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows. ./llvm/lib/Support/raw_ostream.cpp ./llvm/lib/TableGen/Main.cpp ./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp ./llvm/unittests/Support/Path.cpp ./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp ./clang/lib/Frontend/CompilerInstance.cpp ./clang/lib/Driver/Driver.cpp ./clang/lib/Driver/ToolChains/Clang.cpp Reviewed By: MaskRay Differential Revision: https://reviews.llvm.org/D99426
2021-03-30Add -disable-verify flag to llvm-link.Nick Lewycky1-3/+6
This flag allows the developer to see the result of linking even if it fails the verifier, as a step in debugging cases where the linked module fails the verifier. Differential Revision: https://reviews.llvm.org/D99382
2021-03-19[llvm] Add assertions for the smart pointers with the possibility to be null ↵Ella Ma1-1/+3
in ModuleLazyLoaderCache::operator() Split from D91844. The return value of function `ModuleLazyLoaderCache::operator()` in file llvm/tools/llvm-link/llvm-link.cpp. According to the bug report of my static analyzer, the std::function variable `ModuleLazyLoaderCache::createLazyModule` points to function `loadFile`, which may return `nullptr` when error. And the pointer is dereferenced without a check. Reviewed By: tejohnson Differential Revision: https://reviews.llvm.org/D97258
2021-01-19[llvm-link] Improve link time for bitcode archives [NFC]Sergey Dmitriev1-2/+8
Linking large bitcode archives currently takes a lot of time with llvm-link, this patch adds couple improvements which reduce link time for archives - Use one Linker instance for archive instead of recreating it for each member - Lazy load archive members Reviewed By: tra, jdoerfert Differential Revision: https://reviews.llvm.org/D94643
2021-01-11[tools] Mark output of tools as text if it is really textAbhina Sreeskantharajan1-1/+2
This is a continuation of https://reviews.llvm.org/D67696. The following tools also need to set the OF_Text flag correctly. - llvm-profdata - llvm-link Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D94313
2021-01-05[llvm-link] fix linker behavior when linking archives with --only-needed optionSergey Dmitriev1-7/+5
This patch fixes linker behavior when archive is linked with other inputs as a library (i.e. when --only-needed option is specified). In this case library is expected to be normally linked first into a separate module and only after that linker should import required symbols from the linked library module. Reviewed By: tra Differential Revision: https://reviews.llvm.org/D92535
2020-12-09[llvm-link][NFC] Minor cleanupSergey Dmitriev1-8/+6
llvm::Linker::linkModules() is a static member, so there is no need to pass reference to llvm::Linker instance to loadArFile() function. Reviewed By: tra Differential Revision: https://reviews.llvm.org/D92918
2020-12-02[llvm-link] use file magic when deciding if input should be loaded as archiveSergey Dmitriev1-14/+20
llvm-link should not rely on the '.a' file extension when deciding if input file should be loaded as archive. Archives may have other extensions (f.e. .lib) or no extensions at all. This patch changes llvm-link to use llvm::file_magic to check if input file is an archive. Reviewed By: RaviNarayanaswamy Differential Revision: https://reviews.llvm.org/D92376
2020-12-02Revert "[llvm-link] use file magic when deciding if input should be loaded ↵Sergey Dmitriev1-20/+14
as archive" This reverts commit 55f8c2fdfbc5eda1be946e97ecffa2dea44a883e.