| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This is a temporary measure to alleviate Linux pre-commit CI waiting
times that started snowballing
[recently](https://discourse.llvm.org/t/long-wait-for-linux-presubmit-testing/79547/5).
My [initial
estimate](https://github.com/llvm/llvm-project/pull/94208#issuecomment-2155972973)
of 4 additional minutes spent per built seems to be in the right
ballpark, but looks like that was the last straw to break camel's back.
It seems that CI load got past the tipping point, and now it's not able
to burn through the queue over the night on workdays.
I don't intend to overthrow the consensus we reached in #94208, but it
shouldn't come at the expense of the whole LLVM community. I'll enable
this back as soon as we have news that we got more capacity for Linux
pre-commit CI.
|
|
This patch removes LLDB from a list of projects that are excluded from
building and testing on pre-merge CI on Linux.
Windows environment needs to be prepared in order to test LLDB
(https://github.com/llvm/llvm-project/pull/94208#issuecomment-2146256857),
but we don't have enough maintenance resources to do that at the moment.
Because LLDB has been in the list of projects that need to be tested on
Clang changes, this PR make this happen on Linux. This seems to be the
consensus in the discussion of this PR.
|
|
|
|
Windows Only (#93729)
Flang triggers some OOM on Windows CI right now. This is disruptive to
MLIR and LLVM changes that don't touch Flang, as such we disable
building Flang on Windows only for these PR that don't touch flang. The
testing on Linux is unchanged, and the post-merge Windows testing is
still fully covering here.
|
|
This is an experiment to see if we can prevent some of the compiler OOMs
happening without unduly impacting the Windows build latency.
|
|
Following the discussion in
https://github.com/llvm/llvm-project/pull/93233#issuecomment-2127920882,
this patch merges `clang-ci` pipeline into main `GitHub Pull Requests`
pipeline. `clang-ci` enables additional test coverage for Clang by
compiling it, and then using it to compile and test libc++, libc++abi,
and libunwind in C++03, C++26, and Clang Modules modes.
Additional work we skip and total time savings we should see:
1. Checking out the repo to generate the clang-ci pipeline (2 minutes)
2. Building Clang (3.5 minutes)
3. Uploading the artifacts once, then downloading them 3 times and
unpacking 3 times (0.5 minutes)
Note that because previously-split jobs for each mode are now under a
single Linux job, it now takes around 8 minutes more see the Linux CI
results despite total time savings.
The primary goal of this patch is to reduce the load of CI by removing
duplicated work. I consider this goal achieved. I could keep the job
parallelism we had (3 libc++ jobs depending on a main Linux job), but I
don't consider it worth the effort and opportunity cost, because
parallelism is not helping once the pool of builders is fully
subscribed.
|
|
This patch aims to rectify the Windows CI situation by decoupling Clang
changes from Flang test suite, which is causing Windows CI to "pause"
for 20 minutes (details can be found
[here](https://discourse.llvm.org/t/flang-tests-are-extremely-slow-on-windows/78591/11)).
This even seems desirable in the long run, because it was highlighted
that the only part of Clang that Flang depends on is Driver ([Discourse
post](https://discourse.llvm.org/t/flang-tests-are-extremely-slow-on-windows/78591/14)).
Importantly, this patch leaves the question of _entirely_ disabling
Flang tests on Windows CI out of scope.
|
|
|
|
|
|
We need clang and llvm to build in-tree.
|
|
My patch #86877 contains a mistake.
Should have read the comment.
Recent buildkite runs fail because of this, so it is a bit urgent.
|
|
This fixes all shellcheck warnings we have in `monolithic-linux.sh` and
`monolithic-windows.sh`.
All of them have to do with
[SC2086](https://www.shellcheck.net/wiki/SC2086) - Double quote to
prevent globbing and word splitting.
|
|
Clang has a custom separate pipeline integrated with libc++ that only
runs in release mode. It means that changes which touches only clang
won't run the clang tests in the configuration used by LLVM premerge and
will break it unknowingly.
|
|
#84828 added `-k 0` to pre-merge CI so that if one job fails the others
would continue building. This pull request fixes the location of `-k 0`
in the ninja command line.
Resolves #84842 and #83371
|
|
The -k option allows to continue the build after failures as much as
possible. This is useful here because when we run
> ninja check-llvm check-clang
we would like the clang tests to run even if there is a failure in a
llvm tests.
The downside is that a build failure in one file that would prevent from
running any test does not prevent from building more targets, wasting
build resources potentially.
Fixes #83371
|
|
|
|
The failure rate is too high.
See
https://discourse.llvm.org/t/rfc-future-of-windows-pre-commit-ci/76840
|
|
Otherwise, when the base branch is not something that the CI runner
has checked out, that reference to e.g. release/18.x is ambiguous.
|
|
We should diff against the base branch, not always against `main`. This
allows the BuildKite pre-commit CI to work properly when we target other
branches, such as `release/18.x`.
|
|
The "scheduled build" pipeline on BuildKite had been disabled for months
and doesn't exist anymore, so this script is effectively dead code. When
we set up a cron-activated build again, we should do it using Github
actions (which could trigger a BK pipeline if needed).
Keeping this script around just creates additional confusion about
what's used and what's not used for doing CI.
|
|
|
|
Fixing
https://buildkite.com/llvm-project/github-pull-requests/builds/30321#018d1a4a-bf72-449e-a70a-444ded875255
Co-authored-by: XinWang10 <108658776+XinWang10@users.noreply.github.com>
|
|
There are builds like
https://buildkite.com/llvm-project/github-pull-requests/builds/24894
It looks like a deadlock in a test, but we can't see which one.
`--timeout=` will make lit kill and report such tests.
`--time-tests` produces nice report about slowest test, so we can tune
them over time.
The same build as above with new flags
https://buildkite.com/llvm-project/github-pull-requests/builds/24961
|
|
PR for: https://discourse.llvm.org/t/add-check-mlir-python-to-the-mlir-pre-commit-tests/74041
It’s easy to forget about the Python bindings and not build/test them
locally. It’s also easy to change something that’ll break the python
binding tests and not find out till after you’ve committed your change.
These tests seem to run quickly and don’t require much extra setup, so
let's add them to the general MLIR pre-merge tests.
|
|
Basically a reland of ec9d80ec43f5761a34c4a785c67d9e7d21ec8bda but now
with fetching of main before that to get a correct merge base.
|
|
we tried to generate a full diff against main in
ec9d80e but it resulted in wrong diffs.
It seems that the issue was that 'main' was not
updated after agent restart and diff main...HEAD kept growing.
Not enabling diff main...HEAD just yet and will check logs for new PRs
first.
|
|
pre-commit CI (#67743)"
This reverts commit ec9d80ec43f5, since we are currently seeing tons of
CI jobs being triggered spuriously.
|
|
pre-commit CI (#67743)
Since we moved to Github PRs, the workflow has changed a bit and folks
often merge `main` back into their PR branch. This is fine, except the
previous way of determining modified files for pre-commit CI would use
the content modified just in the latest commit, whatever it is. This
means that in case someone merged main back into their PR branch, we'd
think that the files in the merge commit were modified by the PR, and
we'd spuriously trigger a CI run. This should fix this issue.
The downside is that the merge target is hardcoded to `main`, which
might not always be what we want. I still think this is an improvement
over the status quo.
|
|
ninja takes every target as a separate argument
|
|
That fixes issues like https://buildkite.com/llvm-project/github-pull-requests/builds/1350#018a8848-f70e-47f4-8aec-0bb48c49a7ca
|
|
- fixed build for linux (clang was missing)
- removed /monolithic-.. from build directory - it does not add anything
and makes path longer for windows which is not great;
- added env-based configuration to control cache and agent targeting;
- print (s)ccache stats to file not to pullute normal log.
|
|
|
|
- Add control over "modified files" set and agent targeting rules. Mostly for testing purposes.
- Moved sections that outputs "steps" after functions
- Added step that report back to Phabricator
That will allow to replace step generation from invocation of script https://github.com/google/llvm-premerge-checks/blob/main/scripts/pipeline_premerge.py to plain call to ".ci/generate-buildkite-pipeline-premerge".
Differential Revision: https://reviews.llvm.org/D159280
|
|
This basically inlines the logic that was previously located in
https://github.com/google/llvm-premerge-checks so it is part of
the monorepo. This has the benefit of making it extremely easy
for individual projects to understand and modify this logic for
their own needs, unlike the current model where this logic lives
in a separate non-LLVM repository.
It also allows testing changes to the CI configuration using a simple
Phabricator review, since the code that defines the CI pipeline is taken
from the patch under review.
This (or something equivalent) is necessary if we want to retain the
current monolithic pre-commit CI throughout the GitHub PR transition.
Since triggering the monolithic CI is currently attached to the system
we use for triggering CI pipelines from Phabricator, we will lose that
part of the CI when we move to GitHub PRs if we don't do anything.
I've decided to rewrite the code as a shell script because the logic
was fairly simple and it seemed a lot easier than figuring out how to
pull only the relevant parts of llvm-premerge-checks into the monorepo.
Furthermore, I think we should strive to move away from the monolithic
CI altogether since sub-projects should understand, own and maintain the
tests that are relevant for them to run in the CI (with LLVM providing
the infrastructure). Hence, this is somewhat of a temporary solution
until monolithic CI is removed entirely.
Differential Revision: https://reviews.llvm.org/D158863
|
|
This simply moves existing CI-related scripts for Clang outside of libc++
to make the two projects independent with respect to CI.
The reason for making this change is that libc++'s CI pipeline history
was full of Clang runs, making it a lot harder to figure out information
about libc++'s own jobs. I created a pipeline for Clang and this patch
will run pre-commit CI for Clang inside that pipeline instead.
Differential Revision: https://reviews.llvm.org/D155078
|
