libstdc++: Document simd testsuite

Signed-off-by: Matthias Kretz <m.kretz@gsi.de>

libstdc++-v3/ChangeLog:

	* testsuite/experimental/simd/README.md: New file.

1 files changed, 257 insertions, 0 deletions

diff --git a/libstdc++-v3/testsuite/experimental/simd/README.md b/libstdc++-v3/testsuite/experimental/simd/README.md
new file mode 100644
index 0000000..db0d71f
--- /dev/null
+++ b/libstdc++-v3/testsuite/experimental/simd/README.md
@@ -0,0 +1,257 @@
+# SIMD Tests
+
+To execute the simd testsuite, call `make check-simd`, typically with `-j N` 
+argument.
+
+For more control over verbosity, compiler flags, and use of a simulator, use 
+the environment variables documented below.
+
+## Environment variables
+
+### `target_list`
+
+Similar to dejagnu target lists: E.g. 
+`target_list="unix{-march=sandybridge,-march=native/-ffast-math,-march=native/-ffinite-math-only}" 
+would create three subdirs in `testsuite/simd/` to run the complete simd 
+testsuite first with `-march=sandybridge`, then with `-march=native 
+-ffast-math`, and finally with `-march=native -ffinite-math-only`.
+
+
+### `CHECK_SIMD_CONFIG`
+
+This variable can be set to a path to a file which is equivalent to a dejagnu 
+board. The file needs to be a valid `sh` script since it is sourced from the 
+`scripts/check_simd` script. It's purpose is to set the `target_list` variable 
+depending on `$target_triplet` (or whatever else makes sense for you). Example:
+
+```sh
+case "$target_triplet" in
+x86_64-*)
+  target_list="unix{-march=sandybridge,-march=skylake-avx512,-march=native/-ffast-math,-march=athlon64,-march=core2,-march=nehalem,-march=skylake,-march=native/-ffinite-math-only,-march=knl}"
+  ;;
+
+powerpc64le-*)
+  define_target power7 "-mcpu=power7 -static" "$HOME/bin/run_on_gccfarm gcc112"
+  define_target power8 "-mcpu=power8 -static" "$HOME/bin/run_on_gccfarm gcc112"
+  define_target power9 "-mcpu=power9 -static" "$HOME/bin/run_on_gccfarm gcc135"
+  target_list="power7 power8 power9{,-ffast-math}"
+  ;;
+
+powerpc64-*)
+  define_target power7 "-mcpu=power7 -static" "$HOME/bin/run_on_gccfarm gcc110"
+  define_target power8 "-mcpu=power8 -static" "$HOME/bin/run_on_gccfarm gcc110"
+  target_list="power7 power8{,-ffast-math}"
+  ;;
+esac
+```
+
+The `unix` target is pre-defined to have no initial flags and no simulator. Use 
+the `define_target(name, flags, sim)` function to define your own targets for 
+the `target_list` variable. In the example above `define_target power7 
+"-mcpu=power7 -static" "$HOME/bin/run_on_gccfarm gcc112"` defines the target 
+`power7` which always uses the flags `-mcpu=power7` and `-static` when 
+compiling tests and prepends `$HOME/bin/run_on_gccfarm gcc112` to test 
+executables. In `target_list` you can now use the name `power7`. E.g. 
+`target_list="power7 power7/-ffast-math"` or it's shorthand 
+`target_list="power7{,-ffast-math}"`.
+
+
+### `DRIVEROPTS`
+
+This variable affects the `Makefile`s generated per target (as defined above). 
+It's a string of flags that are prepended to the `driver.sh` invocation which 
+builds and runs the tests. You `cd` into a simd test subdir and use `make help` 
+to see possible options and a list of all valid targets.
+
+```
+use DRIVEROPTS=<options> to pass the following options:
+-q, --quiet         Disable same-line progress output (default if stdout is
+                    not a tty).
+-p, --percentage    Add percentage to default same-line progress output.
+-v, --verbose       Print one line per test and minimal extra information on
+                    failure.
+-vv                 Print all compiler and test output.
+-k, --keep-failed   Keep executables of failed tests.
+--sim <executable>  Path to an executable that is prepended to the test
+                    execution binary (default: the value of
+                    GCC_TEST_SIMULATOR).
+--timeout-factor <x>
+                    Multiply the default timeout with x.
+-x, --run-expensive Compile and run tests marked as expensive (default:
+                    true if GCC_TEST_RUN_EXPENSIVE is set, false otherwise).
+-o <pattern>, --only <pattern>
+                    Compile and run only tests matching the given pattern.
+```
+
+
+### `TESTFLAGS`
+
+This variable also affects the `Makefile`s generated per target. It's a list of 
+compiler flags that are appended to `CXXFLAGS`.
+
+
+### `GCC_TEST_SIMULATOR`
+
+If `--sim` is not passed via `DRIVEROPTS`, then this variable is prepended to 
+test invocations. If a simulator was defined via the `CHECK_SIMD_CONFIG` 
+script, then then generated `Makefile` sets the `GCC_TEST_SIMULATOR` variable.
+
+
+### `GCC_TEST_RUN_EXPENSIVE`
+
+If set to any non-empty string, run tests marked as expensive, otherwise treat 
+these tests as `UNSUPPORTED`.
+
+
+## Writing new tests
+
+A test starts with the copyright header, directly followed by directives 
+influencing the set of tests to generate and whether the test driver should 
+expect a failure.
+
+Then the test must at least `#include "bits/verify.h", which provides `main` 
+and declares a `template <typename V> void test()` function, which the test has 
+to define. The template parameter is set to `simd<T, Abi>` type where `T` and 
+`Abi` are determined by the type and ABI subset dimensions.
+
+The `test()` functions are typically implemented using the `COMPARE(x, 
+reference)`, `VERIFY(boolean)`, and `ULP_COMPARE(x, reference, 
+allowed_distance)` macros.
+
+### Directives
+
+* `// skip: <type pattern> <ABI subset pattern> <target triplet pattern> 
+  <CXXFLAGS pattern>`
+  If all patterns match, the test is silently skipped.
+
+* `// only: <type pattern> <ABI subset pattern> <target triplet pattern> 
+  <CXXFLAGS pattern>`
+  If any pattern doesn't match, the test is silently skipped.
+
+* `// expensive: <type pattern> <ABI subset pattern> <target triplet pattern>
+  <CXXFLAGS pattern>`
+  If all patterns match, the test is `UNSUPPORTED` unless expensive tests are 
+  enabled.
+
+* `// xfail: run|compile <type pattern> <ABI subset pattern> <target triplet 
+  pattern> <CXXFLAGS pattern>`
+  If all patterns match, test compilation or execution is expected to fail. The 
+  test then shows as "XFAIL: ...". If the test passes, the test shows "XPASS: 
+  ...".
+
+All patterns are matched via
+```sh
+case '<test context>' in
+  <pattern>)
+  # treat as match
+  ;;
+esac
+```
+The `<CXXFLAGS pattern>` is implicitly adds a `*` wildcard before and after the 
+pattern. Thus, the `CXXFLAGS` pattern matches a substring and all other 
+patterns require a full match.
+
+Examples:
+```cpp
+// The test is only valid for floating-point types:
+// only: float|double|ldouble * * *
+
+// Skip the test for long double for all powerpc64* targets:
+// skip: ldouble * powerpc64* *
+
+// The test is expected to unconditionally fail on execution:
+// xfail: run * * * *
+
+// ABI subsets 1-9 are considered expensive:
+// expensive: * [1-9] * *
+```
+
+
+## Implementation sketch
+
+* `scripts/create_testsuite_files` collects all `*.c` and `*.cc` files with 
+  `simd/tests/` in their path into the file `testsuite_file_simd` (and at the 
+  same time removes them from `testsuite_files`.
+
+* The `check-simd` target in `testsuite/Makefile.am` calls 
+  `scripts/check_simd`. This script calls 
+  `testsuite/experimental/simd/generate_makefile.sh` to generate `Makefile`s in 
+  all requested subdirectories. The subdirectories are communicated back to the 
+  make target via a `stdout` pipe. The `check-simd` rule then spawns sub-make 
+  in these subdirectories. Finally it collects all summaries 
+  (`simd_testsuite.sum`) to present them at the end of the rule.
+
+* The generated Makefiles define targets for each file in `testsuite_file_simd` 
+  (you can edit this file after it was generated, though that's not 
+  recommended) while adding two test dimensions: type and ABI subset. The type 
+  is a list of all arithmetic types, potentially reduced via `only` and/or 
+  `skip` directives in the test's source file. The ABI subset is a number 
+  between 0 and 9 (inclusive) mapping to a set of `simd_abi`s in
+  `testsuite/experimental/simd/tests/bits/verify.h` (`iterate_abis()`). The 
+  tests are thus potentially compiled 170 (17 arithmetic types * 10 ABI 
+  subsets) times. This is necessary to limit the memory usage of GCC to 
+  reasonable numbers and keep the compile time below 1 minute (per compiler 
+  invocation).
+
+* When `make` executes in the generated subdir, the `all` target depends on 
+  building and running all tests via `testsuite/experimental/simd/driver.sh` 
+  and collecting their logs into a `simd_testsuite.log` and then extracting 
+  `simd_testsuite.sum` from it.
+
+* The `driver.sh` script builds and runs the test, parses the compiler and test 
+  output, and prints progress information to the terminal.
+
+## Appendix
+
+### `run_on_gccfarm` script
+
+```sh
+#!/bin/sh
+usage() {
+  cat <<EOF
+Usage $0 <hostname> <executable> [arguments]
+
+Copies <executable> to $host, executes it and cleans up again.
+EOF
+}
+
+[ $# -lt 2 ] && usage && exit 1
+case "$1" in
+  -h|--help)
+    usage
+    exit
+    ;;
+esac
+
+host="$1"
+exe="$2"
+shift 2
+
+# Copy executable locally to strip it before scp to remote host
+local_tmpdir=$(mktemp -d)
+cp "$exe" $local_tmpdir
+cd $local_tmpdir
+exe="${exe##*/}"
+powerpc64le-linux-gnu-strip "$exe"
+
+ssh_controlpath=~/.local/run_on_gccfarm/$host
+if [ ! -S $ssh_controlpath ]; then
+  mkdir -p ~/.local/run_on_gccfarm
+  (
+    flock -n 9
+    if [ ! -S $ssh_controlpath ]; then
+      ssh -o ControlMaster=yes -o ControlPath=$ssh_controlpath -o ControlPersist=10m $host.fsffrance.org true
+    fi
+  ) 9> ~/.local/run_on_gccfarm/lockfile
+fi
+opts="-o ControlPath=$ssh_controlpath"
+
+remote_tmpdir=$(ssh $opts $host.fsffrance.org mktemp -d -p .)
+scp $opts -C -q "$exe" $host.fsffrance.org:$remote_tmpdir/
+cd
+rm -r "$local_tmpdir" &
+ssh $opts $host.fsffrance.org $remote_tmpdir/$exe "$@"
+ret=$?
+ssh $opts $host.fsffrance.org rm -r $remote_tmpdir &
+exit $ret
+```