diff options
Diffstat (limited to 'libstdc++-v3/docs/html/test.html')
| -rw-r--r-- | libstdc++-v3/docs/html/test.html | 722 |
1 files changed, 0 insertions, 722 deletions
diff --git a/libstdc++-v3/docs/html/test.html b/libstdc++-v3/docs/html/test.html deleted file mode 100644 index 8a8694c..0000000 --- a/libstdc++-v3/docs/html/test.html +++ /dev/null @@ -1,722 +0,0 @@ -<?xml version="1.0" encoding="ISO-8859-1"?> -<!DOCTYPE html - PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> - -<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> -<head> - <meta name="AUTHOR" content="bkoz@gcc.gnu.org (Benjamin Kosnik)" /> - <meta name="KEYWORDS" content="c++, libstdc++, test, regression, g++" /> - <meta name="DESCRIPTION" content="README for the GNU libstdc++ effort." /> - <meta name="GENERATOR" content="vi and eight fingers" /> - <title>libstdc++ Testing Instructions</title> -<link rel="StyleSheet" href="lib3styles.css" /> -</head> -<body> - -<h1 class="centered"><a name="top">Testing Details</a></h1> - -<p class="fineprint"><em> - The latest version of this document is always available at - <a href="http://gcc.gnu.org/onlinedocs/libstdc++/test.html"> - http://gcc.gnu.org/onlinedocs/libstdc++/test.html</a>. -</em></p> - -<p><em> - To the <a href="http://gcc.gnu.org/libstdc++/">libstdc++ homepage</a>. -</em></p> - -<!-- ####################################################### --> -<hr /> -<h2>Contents</h2> -<ul> - <li><a href="#org">Testsuite organization and naming conventions</a></li> - <li><a href="#util">Utilities: abicheck and libtestc++</a></li> - <li><a href="#new">How to write a new test case</a></li> - <li><a href="#check">Options for running the tests</a></li> - <li><a href="#debug">Running debug-mode tests</a></li> - <li><a href="#future">Future</a></li> - <li><a href="#internals">DejaGNU internals</a></li> -</ul> - -<hr /> - -<!-- ####################################################### --> - -<h2><a name="org">Testsuite organization and naming conventions</a></h2> - <p> - The directory <em>libsrcdir/testsuite</em> contains the - individual test cases organized in sub-directories corresponding - to chapters of the C++ standard (detailed below), the dejagnu - test harness support files, and sources to various testsuite - utilities that are packaged in a separate testing library. - </p> - - <p> All test cases for functionality required by the runtime - components of the C++ standard (ISO 14882) are files within the - following directories. - </p> - - <pre> -17_intro -18_support -19_diagnostics -20_util -21_strings -22_locale -23_containers -25_algorithms -26_numerics -27_io - </pre> - - <p> - In addition, the following directories include test files: - </p> - - <pre> -tr1 Tests for components as described by the Technical Report on Standard Library Extensions (TR1). -backward Tests for backwards compatibility and deprecated features. -demangle Tests for __cxa_demangle, the IA 64 C++ ABI demangler -ext Tests for extensions. -performance Tests for performance analysis, and performance regressions. -thread Tests for threads. - </pre> - - <p> - Some directories don't have test files, but instead contain - auxiliary information (<a href="#internals">more information</a>): - </p> - - <pre> -config Files for the dejagnu test harness. -lib Files for the dejagnu test harness. -libstdc++* Files for the dejagnu test harness. -data Sample text files for testing input and output. -util Files for libtestc++, utilities and testing routines. - </pre> - - <p> - Within a directory that includes test files, there may be - additional subdirectories, or files. Originally, test cases - were appended to one file that represented a particular section - of the chapter under test, and was named accordingly. For - instance, to test items related to <code> 21.3.6.1 - - basic_string::find [lib.string::find]</code> in the standard, - the following was used: - </p> - <pre> -21_strings/find.cc - </pre> - <p> - However, that practice soon became a liability as the test cases - became huge and unwieldy, and testing new or extended - functionality (like wide characters or named locales) became - frustrating, leading to aggressive pruning of test cases on some - platforms that covered up implementation errors. Now, the test - suite has a policy of one file, one test case, which solves the - above issues and gives finer grained results and more manageable - error debugging. As an example, the test case quoted above - becomes: - </p> - <pre> -21_strings/basic_string/find/char/1.cc -21_strings/basic_string/find/char/2.cc -21_strings/basic_string/find/char/3.cc -21_strings/basic_string/find/wchar_t/1.cc -21_strings/basic_string/find/wchar_t/2.cc -21_strings/basic_string/find/wchar_t/3.cc - </pre> - - <p> - All new tests should be written with the policy of one test - case, one file in mind. - </p> - - <p> - In addition, there are some special names and suffixes that are - used within the testsuite to designate particular kinds of - tests. - </p> - -<ul> -<li> - <em>_xin.cc</em> - <p> - This test case expects some kind of interactive input in order - to finish or pass. At the moment, the interactive tests are not - run by default. Instead, they are run by hand, like: - </p> - <pre> -g++ 27_io/objects/char/3_xin.cc -cat 27_io/objects/char/3_xin.in | a.out - </pre> -</li> -<li> - <em>.in</em> - <p> - This file contains the expected input for the corresponding <em> - _xin.cc</em> test case. - </p> -</li> -<li> - <em>_neg.cc</em> - <p> - This test case is expected to fail: it's a negative test. At the - moment, these are almost always compile time errors. - </p> -</li> -<li> - <em>char</em> - <p> - This can either be a directory name or part of a longer file - name, and indicates that this file, or the files within this - directory are testing the <code>char</code> instantiation of a - template. - </p> -</li> -<li> - <em>wchar_t</em> - <p> - This can either be a directory name or part of a longer file - name, and indicates that this file, or the files within this - directory are testing the <code>wchar_t</code> instantiation of - a template. Some hosts do not support <code>wchar_t</code> - functionality, so for these targets, all of these tests will not - be run. - </p> -</li> -<li> - <em>thread</em> - <p> - This can either be a directory name or part of a longer file - name, and indicates that this file, or the files within this - directory are testing situations where multiple threads are - being used. - </p> -</li> -<li> - <em>performance</em> - <p> - This can either be an enclosing directory name or part of a - specific file name. This indicates a test that is used to - analyze runtime performance, for performance regression testing, - or for other optimization related analysis. At the moment, these - test cases are not run by default. - </p> -</li> -</ul> - -<hr /> -<h2><a name="util">Utilities: abi_check and libtestc++</a></h2> - <p> - The testsuite directory also contains some files that implement - functionality that is intended to make writing test cases easier, - or to avoid duplication, or to provide error checking in a way that - is consistent across platforms and test harnesses. A stand-alone - executable, called <em>abi_check</em>, and a static library called - <em>libtestc++</em> are constructed. Both of these items are not - installed, and only used during testing. - </p> - - <p> - These files include the following functionality: - </p> - - <ul> - <li> - <em>testsuite_abi.h</em>, - <em>testsuite_abi.cc</em>, - <em>testsuite_abi_check.cc</em> - <p> - Creates the executable <em>abi_check</em>. - Used to check correctness of symbol versioning, visibility of - exported symbols, and compatibility on symbols in the shared - library, for hosts that support this feature. More information - can be found in the ABI documentation <a href="abi.html"> here</a> - </p> - </li> - <li> - <em>testsuite_allocator.h</em>, - <em>testsuite_allocator.cc</em> - <p> - Contains specialized allocators that keep track of construction - and destruction. Also, support for overriding global new and - delete operators, including verification that new and delete - are called during execution, and that allocation over max_size - fails. - </p> - </li> - <li> - <em>testsuite_character.h</em> - <p> - Contains <code>std::char_traits</code> and - <code>std::codecvt</code> specializations for a user-defined - POD. - </p> - </li> - <li> - <em>testsuite_hooks.h</em>, - <em>testsuite_hooks.cc</em> - <p> - A large number of utilities, including: - </p> - <ul> - <li>VERIFY</li> - <li>set_memory_limits</li> - <li>verify_demangle</li> - <li>run_tests_wrapped_locale</li> - <li>run_tests_wrapped_env</li> - <li>try_named_locale</li> - <li>try_mkfifo</li> - <li>func_callback</li> - <li>counter</li> - <li>copy_tracker</li> - <li>copy_constructor</li> - <li>assignment_operator</li> - <li>destructor</li> - <li>pod_char, pod_int and associated char_traits specializations</li> - </ul> - <p></p> - </li> - <li> - <em>testsuite_io.h</em> - <p> - Error, exception, and constraint checking for - <code>std::streambuf, std::basic_stringbuf, std::basic_filebuf</code>. - </p> - </li> - <li> - <em>testsuite_iterators.h</em> - <p> - Wrappers for various iterators. - </p> - </li> - <li> - <em>testsuite_performance.h</em> - <p> - A number of class abstractions for performance counters, and - reporting functions including: - </p> - <ul> - <li>time_counter</li> - <li>resource_counter</li> - <li>report_performance</li> - </ul> - <p></p> - </li> - </ul> - -<hr /> -<h2><a name="new">How to write a new test case</a></h2> - - <p> - The first step in making a new test case is to choose the correct - directory and file name, given the organization as previously - described. - </p> - - <p> - All files are copyright the FSF, and GPL'd: this is very - important. The first copyright year should correspond to the date - the file was checked in to SVN. - </p> - - <p> - As per the dejagnu instructions, always return 0 from main to - indicate success. - </p> - - <p> - A bunch of utility functions and classes have already been - abstracted out into the testsuite utility library, <code> - libtestc++</code>. To use this functionality, just include the - appropriate header file: the library or specific object files will - automatically be linked in as part of the testsuite run. - </p> - - <p> - For a test that needs to take advantage of the dejagnu test - harness, what follows below is a list of special keyword that - harness uses. Basically, a test case contains dg-keywords (see - dg.exp) indicating what to do and what kinds of behavior are to be - expected. New test cases should be written with the new style - DejaGnu framework in mind. - </p> - - <p> - To ease transition, here is the list of dg-keyword documentation - lifted from dg.exp. - </p> - -<pre> -# The currently supported options are: -# -# dg-prms-id N -# set prms_id to N -# -# dg-options "options ..." [{ target selector }] -# specify special options to pass to the tool (eg: compiler) -# -# dg-do do-what-keyword [{ target/xfail selector }] -# `do-what-keyword' is tool specific and is passed unchanged to -# ${tool}-dg-test. An example is gcc where `keyword' can be any of: -# preprocess|compile|assemble|link|run -# and will do one of: produce a .i, produce a .s, produce a .o, -# produce an a.out, or produce an a.out and run it (the default is -# compile). -# -# dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]] -# indicate an error message <regexp> is expected on this line -# (the test fails if it doesn't occur) -# Linenum=0 for general tool messages (eg: -V arg missing). -# "." means the current line. -# -# dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]] -# indicate a warning message <regexp> is expected on this line -# (the test fails if it doesn't occur) -# -# dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]] -# indicate a bogus error message <regexp> use to occur here -# (the test fails if it does occur) -# -# dg-build regexp comment [{ target/xfail selector }] -# indicate the build use to fail for some reason -# (errors covered here include bad assembler generated, tool crashes, -# and link failures) -# (the test fails if it does occur) -# -# dg-excess-errors comment [{ target/xfail selector }] -# indicate excess errors are expected (any line) -# (this should only be used sparingly and temporarily) -# -# dg-output regexp [{ target selector }] -# indicate the expected output of the program is <regexp> -# (there may be multiple occurrences of this, they are concatenated) -# -# dg-final { tcl code } -# add some tcl code to be run at the end -# (there may be multiple occurrences of this, they are concatenated) -# (unbalanced braces must be \-escaped) -# -# "{ target selector }" is a list of expressions that determine whether the -# test succeeds or fails for a particular target, or in some cases whether the -# option applies for a particular target. If the case of `dg-do' it specifies -# whether the test case is even attempted on the specified target. -# -# The target selector is always optional. The format is one of: -# -# { xfail *-*-* ... } - the test is expected to fail for the given targets -# { target *-*-* ... } - the option only applies to the given targets -# -# At least one target must be specified, use *-*-* for "all targets". -# At present it is not possible to specify both `xfail' and `target'. -# "native" may be used in place of "*-*-*". - -Example 1: Testing compilation only -// { dg-do compile } - -Example 2: Testing for expected warnings on line 36, which all targets fail -// { dg-warning "string literals" "" { xfail *-*-* } 36 - -Example 3: Testing for expected warnings on line 36 -// { dg-warning "string literals" "" { target *-*-* } 36 - -Example 4: Testing for compilation errors on line 41 -// { dg-do compile } -// { dg-error "no match for" "" { target *-*-* } 41 } - -Example 5: Testing with special command line settings, or without the -use of pre-compiled headers, in particular the stdc++.h.gch file. Any -options here will override the DEFAULT_CXXFLAGS and PCH_CXXFLAGS set -up in the normal.exp file. -// { dg-options "-O0" { target *-*-* } } -</pre> - - <p> - More examples can be found in the libstdc++-v3/testsuite/*/*.cc files. - </p> - -<hr /> -<h2><a name="check">Options for running the tests</a></h2> - - <p> There are several options for running tests, including testing - the regression tests, testing a subset of the regression tests, - testing the performance tests, testing just compilation, testing - installed tools, etc. In addition, there is a special rule for - checking the exported symbols of the shared library. - </p> - - <p>You can check the status of the build without installing it - using the dejagnu harness, much like the rest of the gcc tools.</p> - <pre> make check</pre> - <p>in the <em>libbuilddir</em> directory.</p> - <p>or</p> - <pre> make check-target-libstdc++-v3</pre> - <p>in the <em>gccbuilddir</em> directory.</p> - - <p> - These commands are functionally equivalent and will create a - 'testsuite' directory underneath <em>libbuilddir</em> containing - the results of the tests. Two results files will be generated: - <em> libstdc++.sum</em>, which is a PASS/FAIL summary for each - test, and <em>libstdc++.log</em> which is a log of the exact - command line passed to the compiler, the compiler output, and - the executable output (if any). - </p> - - -<p> -To debug the dejagnu test harness during runs, try invoking with a -specific argument to the variable RUNTESTFLAGS, as below. -</p> - -<pre> -make check-target-libstdc++-v3 RUNTESTFLAGS="-v" -</pre> -or -<pre> -make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v" -</pre> - -<p> -To run a subset of the library tests, you will need to generate the -<em>testsuite_files</em> file by running <tt>make testsuite_files</tt> -in the <em>libbuilddir/testsuite</em> directory, described below. -Edit the file to remove the tests you don't want and then run the -testsuite as normal. -</p> - - -<p> -There are two ways to run on a simulator: set up DEJAGNU to point to a -specially crafted site.exp, or pass down --target_board flags. -</p> -Example flags to pass down for various embedded builds are as follows: -<pre> ---target=powerpc-eabism (libgloss/sim) -make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim" - ---target=calmrisc32 (libgloss/sid) -make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid" - ---target=xscale-elf (newlib/sim) -make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim" -</pre> - -<p> Also, here is an example of how to run the libstdc++ testsuite for a -multilibed build directory with different ABI settings: -</p> -<pre> -make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"' -</pre> - -<p> -You can run the tests with a compiler and library that have already -been installed. Make sure that the compiler (e.g., <code>g++</code>) -is in your <code>PATH</code>. If you are using shared libraries, then -you must also ensure that the directory containing the shared version -of libstdc++ is in your <code>LD_LIBRARY_PATH</code>, or equivalent. -If your GCC source tree is at <code>/path/to/gcc</code>, then you can -run the tests as follows: -</p> -<pre> -runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite -</pre> -<p> -The testsuite will create a number of files in the directory in which you -run this command,. Some of those files might use the same name as -files created by other testsuites (like the ones for GCC and G++), so -you should not try to run all the testsuites in parallel from the same -directory. -</p> - - <p> In addition, there are some testing options that are mostly of - interest to library maintainers and system integrators. As such, - these tests may not work on all cpu and host combinations, and may need to - be executed in the <em>libbuilddir/testsuite</em> directory. These options - include, but are not necessarily limited to, the following: - </p> - - <pre> - make testsuite_files</pre> - <p> - Five files are generated that determine what test files - are run. These files are: - </p> - <ul> - <li> - <em>testsuite_files </em> - <p> This is a list of all the test cases that will be run. Each - test case is on a separate line, given with an absolute path - from the <em>libsrcdir/testsuite</em> directory. - </p> - </li> - - <li> - <em>testsuite_files_interactive </em> - <p> This is a list of all the interactive test cases, using the - same format as the file list above. These tests are not run by default. - </p> - </li> - - <li> - <em>testsuite_files_performance</em> - <p> This is a list of all the performance test cases, using the - same format as the file list above. These tests are not run by default. - </p> - </li> - - <li> - <em>testsuite_thread</em> - <p> This file indicates that the host system can run tests which - incolved multiple threads. - </p> - </li> - - <li> - <em>testsuite_wchar_t</em> - <p> This file indicates that the host system can run the wchar_t - tests, and corresponds to the macro definition <code> - _GLIBCXX_USE_WCHAR_T</code> in the file c++config.h. - </p> - </li> - </ul> - - <pre> - make check-abi</pre> - <p>The library ABI can be tested. This involves testing the shared - library against an ABI-defining previous version of symbol exports. </p> - - <pre> - make check-compile</pre> - <p>This rule compiles, but does not link or execute, the - <em>testsuite_files</em> test cases and displays the output on stdout.</p> - - <pre> - make check-performance</pre> - <p>This rule runs through the <em>testsuite_files_performance</em> - test cases and collects information for performance analysis and - can be used to spot performance regressions. Various timing - information is collected, as well as number of hard page faults, - and memory used. This is not run by default, and the implementation - is in flux. -</p> - - <p> - We are interested in any strange failures of the - testsuite; please see <a href="faq/index.html#2_4">FAQ 2.4</a> - for which files to examine. - </p> - -<hr/> -<h2><a name="debug">Running debug-mode tests</a></h2> -<p>To run the libstdc++ test suite under the <a - href="debug.html#safe">debug mode</a>, - edit <code>libstdc++-v3/scripts/testsuite_flags</code> to add the - compile-time flag <code>-D_GLIBCXX_DEBUG</code> to the result - printed by the <code>--build-cxx</code> option. Additionally, add - the <code>-D_GLIBCXX_DEBUG_PEDANTIC</code> flag to turn on pedantic - checking. The libstdc++ test suite should produce precisely the same - results under debug mode that it does under release mode: any - deviation indicates an error in either the library or the test - suite.</p> - -<hr /> -<h2><a name="future">Future</a></h2> - -<p> -Shared runs need to be implemented, for targets that support shared libraries. -</p> - -<p> -Diffing of expected output to standard streams needs to be finished off. -</p> - -<p> -The V3 testing framework supports, or will eventually support, -additional keywords for the purpose of easing the job of writing -test cases. All V3-keywords are of the form <code>@xxx@</code>. -Currently plans for supported keywords include: -</p> - -<dl> -<dt> <code> @require@ <files> </code> </dt> -<dd> - <p> - The existence of <files> is essential for the test to complete - successfully. For example, a test case foo.C using bar.baz as - input file could say - </p> - <pre> - // @require@ bar.baz</pre> - <p> - The special variable % stands for the rootname, e.g. the - file-name without its `.C' extension. Example of use (taken - verbatim from 27_io/filebuf.cc) - </p> - <pre> - // @require@ %-*.tst %-*.txt</pre> -</dd> -<dt> <code> @diff@ <first-list> <second-list> </code> </dt> -<dd> - <p> - After the test case compiles and ran successfully, diff - <first-list> against <second-list>, these lists should - have the same length. The test fails if diff returns non-zero a - pair of files. - </p> -</dd> -</dl> - -<hr /> -<h2><a name="internals">DejaGNU internals</a></h2> - -<p>This is information for those looking at making changes to the testsuite -structure, and/or needing to trace dejagnu's actions with --verbose. This -will not be useful to people who are "merely" adding new tests to the existing -structure. -</p> - -<p>The first key point when working with dejagnu is the idea of a "tool". -Files, directories, and functions are all implicitly used when they are -named after the tool in use. Here, the tool will always be "libstdc++". -</p> - -<p>The <code>lib</code> subdir contains support routines. The -<code>lib/libstdc++.exp</code> file ("support library") is loaded -automagically, and must explicitly load the others. For example, files can -be copied from the core compiler's support directory into <code>lib</code>. -</p> - -<p>Some routines in <code>lib/libstdc++.exp</code> are callbacks, some are -our own. Callbacks must be prefixed with the name of the tool. To easily -distinguish the others, by convention our own routines are named "v3-*". -</p> - -<p>The next key point when working with dejagnu is "test files". Any -directory whose name starts with the tool name will be searched for test files. -(We have only one.) In those directories, any <code>.exp</code> file is -considered a test file, and will be run in turn. Our main test file is called -<code>normal.exp</code>; it runs all the tests in testsuite_files using the -callbacks loaded from the support library. -</p> - -<p>The <code>config</code> directory is searched for any particular "target -board" information unique to this library. This is currently unused and sets -only default variables. -</p> - - -<!-- ####################################################### --> - -<hr /> -<p class="fineprint"><em> -See <a href="17_intro/license.html">license.html</a> for copying conditions. -Comments and suggestions are welcome, and may be sent to -<a href="mailto:libstdc++@gcc.gnu.org">the libstdc++ mailing list</a>. -</em></p> - - -</body> -</html> |