libstdc++: Add support for running tests with multiple -std options

This copies the code from the compiler's gcc/testsuite/lib/g++-dg.exp so
that each test can be run multiple times, with different -std options.
This means that we can remove most { dg-options "-std=gnu++20" }
directives from tests, because the testsuite will automatically select
a set of -std options that includes that version.

Tests that should only run for a specific standard (e.g. ones that use
something like { dg-do run { target c++11_only } }) should still specify
that standard with { dg-options "-std=gnu++11" }, which overrides the
automatic selection. But a dg-options that selects a newer standard than
the default can be removed, because that standard will be selected
automatically based on a selector like { target c++20 } in the dg-do
directive. This will allow those tests to be run for more than just the
one they currently hardcode, so that e.g. std::format tests can be run
for all of C++20, C++23 and C++26. Currently that has to be done by
adding a second test file that uses a different dg-options line.

By default most tests will continue to run with only the default dialect
(currently -std=gnu++17) so that the time to run the entire testsuite is
not increased. We can revisit this later if increasing the testsuite
time (and coverage) is acceptable. Libstdc++ developers can easily
override the defaults to run for multiple versions. To test all
versions, either add 'set v3_std_list { 98 11 14 17 20 23 26 }' to
~/.dejagnurc or define GLIBCXX_TESTSUITE_STDS="98,11,14,17,20,23,26" in
the environment.

This should be more efficient than the current way to test with multple
standards, i.e. --target_board=unix{-std=c++14,-std=c++17,-std=c++20},
because today all tests with an explicit -std option hardcoded in them
get run for each target board variation but using the exact same
hardcoded -std every time. With the new approach you can just use the
default --target_board=unix and set GLIBCXX_TESTSUITE_STDS="14,17,20"
and now a test that has { target c++20 } will only run once (and be
UNSUPPORTED twice), instead of running with identical options three
times.

In order to support ~/.dejagnurc and $DEJAGNU files that need to work
with versions of GCC without this change, a new variable is added to
site.tmp to detect whether v3_std_list is supported. That allows e.g.

if { [info exists v3-use-std-list] } {
  set v3_std_list { 11 17 23 }
  set target_list { "unix{,-m32}" }
} else {
  set target_list { "unix{,-std=gnu++2b,-std=gnu++11,-m32}" }
}

libstdc++-v3/ChangeLog:

	* doc/xml/manual/test.xml: Update documentation on running and
	writing tests.
	* doc/html/manual/test.html: Regenerate.
	* testsuite/Makefile.am: Add v3-use-std-list to site.tmp
	* testsuite/Makefile.in: Regenerate.
	* testsuite/lib/dg-options.exp (add_options_for_strict_std): New
	proc.
	* testsuite/lib/libstdc++.exp (search_for): New utility proc.
	(v3-dg-runtest): New proc to replace dg-runtest.
	* testsuite/libstdc++-dg/conformance.exp: Use v3-dg-runtest.

2 files changed, 68 insertions, 30 deletions

diff --git a/libstdc++-v3/doc/html/manual/test.html b/libstdc++-v3/doc/html/manual/test.html
index fadadf2..b29c2eb 100644
--- a/libstdc++-v3/doc/html/manual/test.html
+++ b/libstdc++-v3/doc/html/manual/test.html
@@ -351,6 +351,16 @@ cat 27_io/objects/char/3_xin.in | a.out</pre></dd><dt><span class="term"><code c
       the third variation would use the default for <code class="option">-std</code>
       (which is <code class="option">-std=gnu++14</code> as of GCC 6).
     </p><p>
+      Since GCC 14, the libstdc++ testsuite has built-in support for running
+      tests with more than one <code class="option">-std</code>, similar to the G++ tests.
+      Adding <code class="code">set v3_std_list { 11 17 23 }</code> to
+      <code class="filename">~/.dejagnurc</code> or a file named by the
+      <code class="envar">DEJAGNU</code> environment variable will cause every test to
+      be run three times, using a different <code class="option">-std</code> each time.
+      Alternatively, a list of standard versions to test with can be specified
+      as a comma-separated list in the <span style="color: red">&lt;envvar&gt;GLIBCXX_TESTSUITE_STDS&lt;/envvar&gt;</span>
+      environment variable.
+    </p><p>
       To run the libstdc++ test suite under the
       <a class="link" href="debug_mode.html" title="Chapter 17. Debug Mode">debug mode</a>, use
       <strong class="userinput"><code>make check-debug</code></strong>. Alternatively, edit
@@ -467,11 +477,12 @@ cat 27_io/objects/char/3_xin.in | a.out</pre></dd><dt><span class="term"><code c
     It is possible to indicate that a test should <span class="emphasis"><em>only</em></span>
     be run for a specific standard (and not later standards) using an
     effective target like <code class="literal">c++11_only</code>. However, this means
-    the test will be skipped by default (because the default mode is
-    <code class="literal">gnu++14</code>), and so will only run when
-    <code class="option">-std=gnu++11</code> or <code class="option">-std=c++11</code> is used
-    explicitly. For tests that require a specific standard it is better to
-    use a <code class="literal">dg-options</code> directive:
+    the test will be skipped by default unless <code class="option">-std=gnu++11</code>
+    or <code class="option">-std=c++11</code> is explicitly specified, either via a
+    target board, the <code class="varname">v3_std_list</code> dejagnu variable,
+    or the <span style="color: red">&lt;envvar&gt;GLIBCXX_TESTSUITE_STDS&lt;/envvar&gt;</span> environment variable.
+    For tests that require a specific standard it is useful to also add a
+    <code class="literal">dg-options</code> directive:
 </p><pre class="programlisting">    // { dg-options "-std=gnu++11" }</pre><p>
     This means the test will not get skipped by default, and will always use
     the specific standard dialect that the test requires. This isn't needed
@@ -479,16 +490,21 @@ cat 27_io/objects/char/3_xin.in | a.out</pre></dd><dt><span class="term"><code c
     minimum standard instead, to allow them to be tested for all
     possible variations.
   </p><p>
-    Similarly, tests which depend on a newer standard than the default
-    must use <code class="literal">dg-options</code> instead of (or in addition to)
-    an effective target, so that they are not skipped by default.
-    For example, tests for C++17 features should use
-</p><pre class="programlisting">    // { dg-options "-std=gnu++17" }</pre><p>
-    before any <code class="literal">dg-do</code> such as:
-</p><pre class="programlisting">    // { dg-do run "c++17" }</pre><p>
-    The <code class="literal">dg-options</code> directive must come first, so that
-    the <code class="literal">-std</code> flag has already been added to the options
-    before checking the <code class="literal">c++17</code> target.
+    N.B. when a <code class="literal">dg-options</code> directive is used, it must come
+    first so dejagnu will include those options when checking against any
+    effective targets in <code class="literal">dg-do</code> and
+    <code class="literal">dg-require-effective-target</code> directives.
+  </p><p>
+    Since GCC 14, tests which depend on a newer standard than the default
+    do not need to specify that standard in a <code class="literal">dg-options</code>
+    directive. The testsuite will detect when a test requires a newer standard
+    and will automatically add a suitable <code class="option">-std</code> flag.
+  </p><p>
+    If a testcase requires the use of a strict language dialect, e.g.
+    <code class="option">-std=c++11</code> rather than <code class="option">-std=gnu++11</code>,
+    the following directive will cause that to be used when the testsuite
+    decides which <code class="option">-std</code> options to use for the test:
+</p><pre class="programlisting">    // { dg-add-options strict_std }</pre><p>
   </p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="tests.dg.examples"></a>Examples of Test Directives</h4></div></div></div><p>
 Example 1: Testing compilation only:
 </p><pre class="programlisting">
diff --git a/libstdc++-v3/doc/xml/manual/test.xml b/libstdc++-v3/doc/xml/manual/test.xml
index 964c53d..936f974 100644
--- a/libstdc++-v3/doc/xml/manual/test.xml
+++ b/libstdc++-v3/doc/xml/manual/test.xml
@@ -599,6 +599,18 @@ cat 27_io/objects/char/3_xin.in | a.out</programlisting>
     </para>
 
     <para>
+      Since GCC 14, the libstdc++ testsuite has built-in support for running
+      tests with more than one <option>-std</option>, similar to the G++ tests.
+      Adding <code>set v3_std_list { 11 17 23 }</code> to
+      <filename>~/.dejagnurc</filename> or a file named by the
+      <envar>DEJAGNU</envar> environment variable will cause every test to
+      be run three times, using a different <option>-std</option> each time.
+      Alternatively, a list of standard versions to test with can be specified
+      as a comma-separated list in the <envvar>GLIBCXX_TESTSUITE_STDS</envvar>
+      environment variable.
+    </para>
+
+    <para>
       To run the libstdc++ test suite under the
       <link linkend="manual.ext.debug_mode">debug mode</link>, use
       <userinput>make check-debug</userinput>. Alternatively, edit
@@ -766,11 +778,12 @@ cat 27_io/objects/char/3_xin.in | a.out</programlisting>
     It is possible to indicate that a test should <emphasis>only</emphasis>
     be run for a specific standard (and not later standards) using an
     effective target like <literal>c++11_only</literal>. However, this means
-    the test will be skipped by default (because the default mode is
-    <literal>gnu++14</literal>), and so will only run when
-    <option>-std=gnu++11</option> or <option>-std=c++11</option> is used
-    explicitly. For tests that require a specific standard it is better to
-    use a <literal>dg-options</literal> directive:
+    the test will be skipped by default unless <option>-std=gnu++11</option>
+    or <option>-std=c++11</option> is explicitly specified, either via a
+    target board, the <varname>v3_std_list</varname> dejagnu variable,
+    or the <envvar>GLIBCXX_TESTSUITE_STDS</envvar> environment variable.
+    For tests that require a specific standard it is useful to also add a
+    <literal>dg-options</literal> directive:
 <programlisting>    // { dg-options "-std=gnu++11" }</programlisting>
     This means the test will not get skipped by default, and will always use
     the specific standard dialect that the test requires. This isn't needed
@@ -780,16 +793,25 @@ cat 27_io/objects/char/3_xin.in | a.out</programlisting>
   </para>
 
   <para>
-    Similarly, tests which depend on a newer standard than the default
-    must use <literal>dg-options</literal> instead of (or in addition to)
-    an effective target, so that they are not skipped by default.
-    For example, tests for C++17 features should use
-<programlisting>    // { dg-options "-std=gnu++17" }</programlisting>
-    before any <literal>dg-do</literal> such as:
-<programlisting>    // { dg-do run "c++17" }</programlisting>
-    The <literal>dg-options</literal> directive must come first, so that
-    the <literal>-std</literal> flag has already been added to the options
-    before checking the <literal>c++17</literal> target.
+    N.B. when a <literal>dg-options</literal> directive is used, it must come
+    first so dejagnu will include those options when checking against any
+    effective targets in <literal>dg-do</literal> and
+    <literal>dg-require-effective-target</literal> directives.
+  </para>
+
+  <para>
+    Since GCC 14, tests which depend on a newer standard than the default
+    do not need to specify that standard in a <literal>dg-options</literal>
+    directive. The testsuite will detect when a test requires a newer standard
+    and will automatically add a suitable <option>-std</option> flag.
+  </para>
+
+  <para>
+    If a testcase requires the use of a strict language dialect, e.g.
+    <option>-std=c++11</option> rather than <option>-std=gnu++11</option>,
+    the following directive will cause that to be used when the testsuite
+    decides which <option>-std</option> options to use for the test:
+<programlisting>    // { dg-add-options strict_std }</programlisting>
   </para>
 
 <section xml:id="tests.dg.examples"><info><title>Examples of Test Directives</title></info>