math: Refactor how to use libm-test-ulps

The current approach tracks math maximum supported errors by explicitly
setting them per function and architecture. On newer implementations or
new compiler versions, the file is updated with newer values if it
shows higher results. The idea is to track the maximum known error, to
update the manual with the obtained values.

The constant libm-test-ulps shows little value, where it is usually a
mechanical change done by the maintainer, for past releases it is
usually ignored whether the ulp change resulted from a compiler
regression, and the math tests already have a maximum ulp error that
triggers a regression.

It was shown by a recent update after the new acosf [1] implementation
that is correctly rounded, where the libm-test-ulps was indeed from a
compiler issue.

This patch removes all arch-specific libm-test-ulps, adds system generic
libm-test-ulps where applicable, and changes its semantics. The generic
files now track specific implementation constraints, like if it is
expected to be correctly rounded, or if the system-specific has
different error expectations.

Now multiple libm-test-ulps can be defined, and system-specific
overrides generic implementation.  This is for the case where
arch-specific implementation might show worse precision than generic
implementation, for instance, the cbrtf on i686.

Regressions are only reported if the implementation shows larger errors
than 9 ulps (13 for IBM long double) unless it is overridden by
libm-test-ulps and the maximum error is not printed at the end of tests.
The regen-ulps rule is also removed since it does not make sense to
update the libm-test-ulps automatically.

The manual error table is also removed, Paul Zimmermann and others have
been tracking libm precision with a more comprehensive analysis for some
releases; so link to his work instead.

[1] https://sourceware.org/git/?p=glibc.git;a=commit;h=9cc9f8e11e8fb8f54f1e84d9f024917634a78201

1 files changed, 38 insertions, 40 deletions

diff --git a/math/README.libm-test b/math/README.libm-test
index f32c629..84b2979 100644
--- a/math/README.libm-test
+++ b/math/README.libm-test
@@ -30,8 +30,8 @@ libm.  These files, along with generated files named
 cases for each math function that are specific to the target platform
 but still independent of the real floating type.  The results of the
 processing are "libm-test-<func>.c" and a file "libm-test-ulps.h" with
-platform specific deltas by which the actual math function results may
-deviate from the expected results and still be considered correct.
+specific math results that can be either generic for the floating
+type or platform specific.
 
 The test drivers "test-double-<func>.c", "test-float-<func>.c", and
 "test-ldouble-<func>.c", generated by the Makefile, test the normal
@@ -42,13 +42,11 @@ of macros just before including the generic "libm-test.c" file. Each
 driver is compiled into a single executable test program with the
 corresponding name.
 
-As mentioned above, the "gen-libm-test.py" script looks for a file
-named "libm-test-ulps" in the platform specific sysdep directory (or
-its fpu or nofpu subdirectory) and for each variant (real floating
-type and rounding mode) of every tested function reads from it the
-maximum difference expressed as Units of Least Precision (ULP) the
-actual result of the function may deviate from the expected result
-before it's considered incorrect.
+The math tests do not report up to 9 Units of Least Precision (ULP)
+(13 for IBM long double format) difference between the obtained
+result and the expected one as a regression.  The "gen-libm-test.py"
+script looks for files named "libm-test-ulps" in the sysdep directories
+to generate the "libm-test-ulps.h" file.
 
 The "auto-libm-test-out-<func>" files contain sets of test cases to
 exercise, the conditions under which to exercise each, and the
@@ -57,29 +55,39 @@ expected results.  The files are generated by the
 the comments in gen-auto-libm-tests.c for details about the content
 and format of the -in and -out files.
 
-How can I generate "libm-test-ulps"?
+How can I use "libm-test-ulps"?
 ====================================
 
-To automatically generate a new "libm-test-ulps" run "make regen-ulps".
-This generates the file "math/NewUlps" in the build directory.  The file
-contains the sorted results of all the tests.  You can use the "NewUlps"
-file as the machine's updated "libm-test-ulps" file.  Copy "NewUlps" to
-"libm-test-ulps" in the appropriate machine sysdep directory.  Verify
-the changes, post your patch, and check it in after review.
-
-To manually generate a new "libm-test-ulps" file, first remove "ULPs"
-file in the current directory, then you can execute for example:
-    ./testrun.sh math/test-double -u --ignore-max-ulp=yes
-This generates a file "ULPs" with all double ULPs in it, ignoring any
-previously calculated ULPs, and running with the newly built dynamic
-loader and math library (assumes you didn't install your build).  Now
-generate the ULPs for all other formats, the tests will be appending the
-data to the "ULPs" file.  As final step run "gen-libm-test.py" with the
-file as input and ask to generate a pretty printed output in the file
-"NewUlps":
-  gen-libm-test.py -u ULPs -n NewUlps
-Copy "NewUlps" to "libm-test-ulps" in the appropriate machine sysdep
-directory.
+A "libm-test-ulps" is required only to test for extra constraints in
+the math tests.  The file contains lines for maximal errors of single
+functions, like:
+
+Function "yn":
+float: 2
+double: 6
+
+It means that if the "yn" shows error larger than 2 ULP for float
+or 6 ULP for double, the related test for "symbol" will fail.  It can
+be useful to check for correctly rounded implementation, where the
+expected ULP is 0.
+
+The function is tested with default FE_TONEAREST rounding mode.  To
+check with a different one, the function definition name should be
+prepended with an underline plus the rounding mode 'downward' (FE_DOWNWARD),
+'towardzero' (FE_TOWARDZERO), or 'upward' (FE_UPWARD).  For instance,
+
+Function "yn_downward":
+float: 3
+double: 7
+
+It means that 'yn' will be checked with FE_DOWNWARD rounding mode
+and any error larger than 3 ULPs for float or 7 ULPs for double will be
+reported as a regression.
+
+The keywords are float, double, ldouble, and float128.
+
+Also, multiple "libm-test-ulps" can be added, "gen-libm-test.py" will
+merge the input in only one table.
 
 Note that the test drivers have an option "-u" to output an unsorted
 list of all epsilons that the functions have.  The output can be read
@@ -87,16 +95,6 @@ in directly but it's better to pretty print it first.
 "gen-libm-test.py" has an option to generate a pretty-printed and
 sorted new ULPs file from the output of the test drivers.
 
-Contents of libm-test-ulps
-==========================
-
-Since libm-test-ulps can be generated automatically, just a few notes.
-The file contains lines for maximal errors of single functions, like:
-
-Function "yn":
-double: 6
-
-The keywords are float, double, and ldouble.
 
 Adding tests to libm-test-<func>.inc
 ====================================