switch to fully parallel mode

This switches "make check" to fully parallel mode.

One primary issue facing full parallelization is the overhead of
"runtest".  On my machine, if I "touch gdb.base/empty.exp", making a
new file, and then "time runtest.exp", it takes 0.08 seconds.

Multiply this by the 1008 (in my configuration) tests and you get ~80
seconds.  This is the overhead that would theoretically be present if
all tests were run in parallel.

However, the problem isn't nearly as bad as this, for two reasons.

First, you must divide by the number of jobs, assuming perfect
parallelization -- reasonably true for small -j numbers, based on the
results I see.

Second, the current test suite parallelization approach bundles the
tests, largely by directory, but also splitting up gdb.base into two
halves.

I was curious to see how the current bundling played out in practice,
so I ran "make -j1 check RUNTEST='/bin/time runtest'".  This invokes
the parallel mode (thus the bundling) and then shows the time taken by
each invocation of runtest.

Then, I ran "/bin/time make -j3 check".  (See below about -j2.)

The time for the entire -j3 test run was the same as the time for
"gdb.base1".  What this means is that gdb.base1 is currently the
time-limiting run, preventing further parallelization gains.

So, I reason, whatever overhead we see from full parallelization will
only be seen by "-j1" and "-j2".

I then tried a -j2 test run.  This does take longer than a -j3 build,
meaning that the gdb.base1 job finishes and then proceeds to other
runtest invocations.

Finally I tried a -j2 test run with the appended patch.
This was 9% slower than the -j2 run without the patch.

I think that is a reasonable slowdown for what is probably a rare
case.  I believe this patch will yield faster test results for all -j
values greater than 2.  For -j3 on my machine, the test suite is a few
seconds faster; I didn't try any larger -j values.

For -j1, I went ahead and changed the Makefile so that, if no -j
option is given, then the "check-single" mode is used.  You can still
use "make -j1 check" to get single-job parallel-mode, though of course
there's no good reason to do so.

This change is likely to speed up the plain "make check" scenario a
little as we will now bypass dg-extract-results.sh.

One drawback of this change is that "make -jN check" is now much more
verbose.  I generally only look at the .sum and .log files, but
perhaps this will bother some.

Another interesting question is scalability of the result.  The
slowest test, which limits the scalability, took 80.78 seconds.  The
mean of the remaining tests is 1.08 seconds.  (Note that this is just
a rough estimate, since there are still outliers.)

This means we can run 80.78 / 1.08 =~ 74 tests in the time available.
And, in this data set (slightly older than the above, but materially
the same) there were 948 tests.  So, I think the current test suite
should scale ok up to about -j12.

We could improve this number if need be by breaking up the biggest
tests.

2013-11-04  Tom Tromey  <tromey@redhat.com>

	* Makefile.in (TEST_DIRS): Remove.
	(TEST_TARGETS, check-parallel): Rewrite.
	(check-gdb.%, BASE1_FILES, BASE2_FILES, check-gdb.base%)
	(subdir_do, subdirs): Remove.
	(do-check-parallel, check/%): New targets.
	(clean): Remove outputs, temp, and cache directories.
	(saw_dash_j): New variable.
	(CHECK_TARGET): Use it.
	(check): Depend on all, site.exp.  Rewrite.
	(check-single): Remove dependencies.
	(slow_tests, all_tests, reordered_tests): New variables.

0 files changed, 0 insertions, 0 deletions