aboutsummaryrefslogtreecommitdiff
path: root/tools/buildman
AgeCommit message (Collapse)AuthorFilesLines
2020-11-05patman: Rename functions in patchstreamSimon Glass1-3/+3
Rename these functions to lower case as per PEP8. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-09-22buildman: Use git worktrees instead of git clones when possibleAlper Nebi Yasak2-8/+42
This patch makes buildman create linked working trees instead of clones of the source repository, but keeps updating the older clones of the repository that might already exist. These worktrees share "everything except working directory specific files such as HEAD, index, etc." with the source repository. See the git-worktree(1) manual page for more information. If git-worktree isn't available, silently falls back to cloning the repository. Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org> Tested-by: Simon Glass <sjg@chromium.org>
2020-09-22tools: Drop unnecessary use of __file__Simon Glass1-3/+0
There are few places where the path of the current modules is calculated but not used. Drop them. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-28buildman: Show the build rate at the endSimon Glass2-1/+15
It is interesting to note the number of builds completed per second to track machine performance and build speed. Add a 'rate' value at the end of the build to show this. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-28buildman: Correct the testOutputDir() unit testSimon Glass1-2/+1
This current fails with an error. Fix it. Signed-off-by: Simon Glass <sjg@chromium.org> Fixes: 7664b03ffc5 ("buildman: Remove _of_#_ from results directory paths")
2020-07-28buildman: Allow using older versions of genboardscfg.pySimon Glass1-2/+8
Older versions of this script don't support the -q flag. Since buildman runs this script from when it starts, we may get the old version. Fix this in two ways: 1. Use the version from the same tree as buildman is run from, if available 2. Failing that, allow the -q flag to be missing Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-01kconfiglib: Update to the 14.1.0 releaseTom Rini1-242/+372
A large number of changes have happened upstream since our last sync in commit 65e05ddc1ae2 ("kconfiglib: Update to the 12.14.0 release"). The big motivation for this sync is support for user defined macros within Kconfig. Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Tom Rini <trini@konsulko.com>
2020-06-11buildman: Remove _of_#_ from results directory pathsOvidiu Panait2-9/+9
Currently, the following scenario will rebuild the first commit even though it is not really necessary - the commit sha or the position in the patchset did not change: $ git am <local-patch-0001> $ tools/buildman/buildman -P -E -W -b master mx6 <do some more development work> $ git am <local-patch-0002> $ tools/buildman/buildman -P -E -W -b master mx6 <- will rebuild the first commit as well, even though nothing has changed about it. This is due to the fact that previous results directories get removed when the number of commits change. By removing the _of_#_ part of the directory path, the commits will be rebuilt only if the commit sha or the position in the patchset changes. Also, update the testcase to reflect this change. Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
2020-04-26patman: Tidy up sys.path changesSimon Glass2-3/+1
Now that we are using absolute paths we can remove some of the sys.path mangling that appears in the tools. We only need to add the path to 'tools/' so that everything can find modules relative to that directory. The special paths for finding pylibfdt remain. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26patman: Move to absolute importsSimon Glass7-27/+26
At present patman sets the python path on startup so that it can access the libraries it needs. If we convert to use absolute imports this is not necessary. Move patman to use absolute imports. This requires changes in tools which use the patman libraries (which is most of them). Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26buildman: Move to absolute importsSimon Glass6-29/+30
At present buildman sets the python path on startup so that it can access the libraries it needs. If we convert to use absolute imports this is not necessary. Move buildman to use absolute imports. Also adjust moveconfig.py too since it uses some buildman modules and cannot work without this. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26buildman: Rename the main moduleSimon Glass2-1/+1
Python does not like the module name being the same as the module directory. To allow buildman modules to be used from other tools, rename it. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26patman: Drop references to __future__Simon Glass1-2/+0
We don't need these now that the tools using Python 3. Drop them. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26buildman: Write output files when using -wSimon Glass3-14/+26
At present buildman does not write its own output files (err, done, the environment) when using -w. However this is useful for when the build is run with -s to check it. In fact ProduceResultSummary() reads the result from those files rather than using the 'result' info directly. So ProcessResult() does not work with -w at present. It does not print any output. Fix this by writing output files even when -w is used. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26buildman: Use out-env for environment outputSimon Glass2-1/+8
At present the environment used by U-Boot is written to the 'env' directory. This is fine when the output directory is not the same as the source directory, but when it is (as with -w) it conflicts with the source directory of the same name. Rename 'env' to 'out-env' to fix this. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26buildman: Make sure that -o is given with -wSimon Glass4-3/+13
It is a bad idea to use the default output directory ('..') with -w since it does a build in that directory and writes various files these. Require that -o is given to avoid this problem. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26buildman: Correct operation of -A flagSimon Glass1-1/+1
This was broken when -a was removed and unfortunately there are no tests for this. Fix it. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26buildman: Fix test for new 9.2 kernelSimon Glass1-1/+1
The naming is slightly different on kernel.org now. Update the regex so that the test still passes. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Change the exit codesSimon Glass3-9/+9
The current exit codes of 128 and 129 are useful in that they do not conflict with those returned by tools, but they are not actually valid. It seems better to pick some codes which work with 'bit bisect run'. Update them to 100 (for errors) and 101 (for warnings). Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Update the TODO itemsSimon Glass1-6/+11
A few of these have been done. Drop those and add some new ideas. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Add an option to ignore migration warningsSimon Glass5-35/+107
These are becoming more common now. They cause boards to show warnings which can be mistaking for compiler warnings. Add a buildman option to ignore them. This option works only with the summary option (-s). It does not affect the build process. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Add an option to ignore device-tree warningsSimon Glass5-23/+52
Unfortunately the plague of device-tree warnings has not lifted. These warnings infiltrate almost every build, adding noise and confusion. Add a buildman option to ignore them. This option works only with the summary option (-s). It does not affect the build process. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Make -I the defaultSimon Glass6-24/+41
At present buildman defaults to running 'mrproper' on every thread before it starts building commits for each board. This can add a delay of about 5 seconds to the start of the process, since the tools and other invariants must be rebuilt. In particular, a build without '-b', to build current source, runs much slower without -I, since any existing build is removed, thus losing the possibility of an incremental build. Partly this behaviour was to avoid strange build-system problems caused by running 'make defconfig' for one board and then one with a different architecture. But these problems were fixed quite a while ago. The -I option (which disabled mrproper) was introduced four years ago and does not seem to cause any problems with builds. So make -I the default and deprecate the option. To allow use of 'mrproper', add a new -m flag. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Update workflow documentation with more detailSimon Glass1-17/+36
Make a few additions and change some wording in the workflow documentation. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Add the abbreviation for --boardsSimon Glass1-3/+3
This option may be frequency used, so mention that it can be abbreviated to --bo Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Update the 'theory of operation' a littleSimon Glass1-30/+39
Make a few updates to this important section of the documentation, to make things clearer. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Show a summary of the build resultSimon Glass2-3/+20
When buildman finishes it leaves the last summary line visible, which shows the number of successful builds, builds with warnings and builds with errors. It is useful also to see how many builds were done in total along with the time taken. Show these on a separate line before buildman finishes. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Limit the length of progress messagesSimon Glass1-1/+1
If a progress message is longer than the terminal line it will scroll the terminal. Limit the messages to the terminal width. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Show the number of builds remainingSimon Glass1-4/+9
It is nice to see the actual number of builds remaining to complete. Add this in the progress message, using a different colour. Drop the unnecessary 'name' variable while we are here. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Drop unused output codeSimon Glass1-5/+0
The commit counter is a hangover from when buildman processed each board for a commit. Now buildman processes each commit for a board, so this output is never triggered. Delete it. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Show a message when fetching a repoSimon Glass1-1/+4
Fetching updated versions of a repo can take time. At present buildman gives no indication that it is doing this. Add a message to explain the delay. Tidy up a few other messages while we are here. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Drop the line-clearing code in BuilderSimon Glass1-25/+3
The new feature in terminal can be used by buildman. Update the Builder class accordingly. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Use spaces in the board listSimon Glass2-4/+4
At present the board names shown with -l are separated by commas. This makes it hard to double-click to select a particular board. Also it is not possible to select all boards and paste them as arguments to a subsequent buildman run, since buildman requires spaces to separate the list on the command line, not commas. Change the output format to use spaces instead of commas. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Show the list of boards in magentaSimon Glass2-41/+44
It is quite hard to see the list of board for each error line since the colour is the same as the actual error line. Show the board list in magenta so that it is easier to distinguish them. There is no point in checking the colour of the overall line, since there are now multiple colours. So drop those tests. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Use an object to hold error linesSimon Glass1-17/+52
At present the string for each error line is created in _CalcErrorDelta() and used to create the summary output. This is inflexible since all the information (error/warning character, error line, list of boards with that error line) is munged together in a string. Create an object to hold this information and only convert it to a string when printing the actual output. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Use yellow consistently for warning linesSimon Glass2-4/+4
At present warnings are shown in yellow in the summary (-s) but magenta in the detail listing (-e). Use yellow in both. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Test the output with --list-error-boardsSimon Glass1-14/+32
Add a test to cover this flag, which adds the name of each board to each error/warning line. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Add a test helper for creating a line prefixSimon Glass1-20/+31
The split/join code is repeated in a lot of places. Add a function to handle this. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Split out testOutput() into separate functionsSimon Glass1-6/+28
We want to add a few more tests similar to testOutput(). Split its logic into a function which runs buildman to get the output and another which checks the output. This will make it easier to reuse the code. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Create temp directory in test setupSimon Glass1-16/+10
Rather than having a few tests handle this themselves, create the temporary directory in the setUp() method and remove it in tearDown(). This will make it easier to add more tests. Only testOutput and testGit() actually need it, but it doesn't add to the test time noticeably to do this for all tests in this file. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Use an iterator to check test outputSimon Glass1-42/+52
Rather than using the absolute array index, use an interator to work through the expected output lines. This is easier to follow. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Add test coverage for error/warning colourSimon Glass1-0/+7
Buildman should output the right colours for each error/warning line. Some of these checks are missing. Add them. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-21buildman: Refactor error-line output int a functionSimon Glass1-13/+19
Reduce the amount of repeated code by creating an _OutputErrLines() function to hold this code. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-10buildman: Drop the -a optionSimon Glass3-15/+8
There is no point in setting the ARCH environment variable since the U-Boot build system no-longer uses it. It seems safe to drop this feature since it was only recently added. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Tom Rini <trini@konsulko.com>
2020-04-10buildman: Allow building within a subdir of the current dirSimon Glass3-33/+2
This is useful in some situations, in particular with -w and when building in-tree. Now that we are more careful about what we remove in _PrepareOutputSpace(), it should be safe to relax this restriction. Update the progress information also so it is clear what buildman is doing. Remove files can take a long time. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-10buildman: Be more selective about which directories to removeSimon Glass2-5/+42
At present buildman removes any directory it doesn't intend to write output into. This is overly expansive since if the output directory happens to be somewhere with existing files, they may be removed. Using an existing directory for buildman is not a good practice, but since the result might be catastrophic, it is best to guard against it. A previous commit[1] fixed this by refusing to write to a subdirectory of the current directory, assumed to have U-Boot source code. But we can do better by only removing directories that look like the ones buildman creates. Update the code to do this and add a test. Signed-off-by: Simon Glass <sjg@chromium.org> [1] 409fc029c40 tools: buildman: Don't use the working dir as build dir
2020-04-10buildman: Allow ignoring warnings in the return codeSimon Glass3-3/+21
Sometimes we don't want buildman to return failure if it seems warnings. Add a -W option to support this. If buildman detects warnings (and no errors) it will return an exit code of 0 (success). Note that the definition of 'warnings' includes the migration warnings produced by U-Boot, such as: ===================== WARNING ====================== This board does not use CONFIG_DM_MMC. Please update ... ==================================================== Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Tom Rini <trini@konsulko.com>
2020-04-10buildman: Update help for -dSimon Glass2-4/+4
This help is a bit ambiguous. It only does anything if asked to show size changes with -S. Update the help and the function comments. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-10bulidman: Add support for a simple buildSimon Glass6-15/+90
It is useful to run a simple build and put all the output in a single directory. Add a -w option to support this. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-10buildman: Document the members of BuilderJobSimon Glass1-1/+5
This class has a few more members now. Add documentation for them and fix a nit in the 'commits' comment. Signed-off-by: Simon Glass <sjg@chromium.org>