diff options
| author | Ben Elliston <bje@gnu.org> | 2004-02-03 13:37:17 +0000 |
|---|---|---|
| committer | Ben Elliston <bje@gnu.org> | 2004-02-03 13:37:17 +0000 |
| commit | d12323ba03dc5a0cb07d421de7233d4969415f5a (patch) | |
| tree | ddab7082bcd5e98ac4294a3c03b50c5c202aa7bd | |
| parent | 57ed4889db7f94b48f77a7ef42cf1bab51c0f1f1 (diff) | |
| download | dejagnu-d12323ba03dc5a0cb07d421de7233d4969415f5a.zip dejagnu-d12323ba03dc5a0cb07d421de7233d4969415f5a.tar.gz dejagnu-d12323ba03dc5a0cb07d421de7233d4969415f5a.tar.bz2 | |
* doc/overview.sgml (dejagnu-copyright): Update copyright year.
(dejagnu-code-copyright): Likewise.
(<bookinfo>): Include myself as an author.
(preface): Overhaul abstract.
(overview): Call this chapter "Introduction".
(install): Placeholder for chapter 2 on installing DejaGnu.
| -rw-r--r-- | ChangeLog | 13 | ||||
| -rw-r--r-- | doc/overview.sgml | 76 | ||||
| -rw-r--r-- | doc/user.sgml | 803 |
3 files changed, 479 insertions, 413 deletions
@@ -1,3 +1,16 @@ +2004-02-04 Ben Elliston <bje@wasabisystems.com> + + * doc/overview.sgml (dejagnu-copyright): Update copyright year. + (dejagnu-code-copyright): Likewise. + (<bookinfo>): Include myself as an author. + (preface): Overhaul abstract. + (overview): Call this chapter "Introduction". + (install): Placeholder for chapter 2 on installing DejaGnu. + +2004-02-03 Ben Elliston <bje@wasabisystems.com> + + * doc/user.sgml: Editorial changes throughout. + 2004-02-02 Ben Elliston <bje@wasabisystems.com> * doc/overview.sgml (Overview): Editorial changes. diff --git a/doc/overview.sgml b/doc/overview.sgml index b45f371..1ab59c6 100644 --- a/doc/overview.sgml +++ b/doc/overview.sgml @@ -8,15 +8,15 @@ <!ENTITY dj "DejaGnu"> <!ENTITY dejagnu-copyright " - <YEAR>2002</YEAR> + <YEAR>2004</YEAR> <HOLDER>Free Software Foundation, Inc.</HOLDER>"> <!ENTITY dejagnu-code-copyright " %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% This file documents the GNU Testing Framework ``DejaGnu'' -Copyright (C) 92 - 2001, 2002 Free Software -Foundation, Inc. +Copyright 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, + 2002, 2003, 2004 Free Software Foundation, Inc. This text may be freely distributed under the terms of the GNU General Public License. @@ -24,8 +24,8 @@ General Public License. "> <!ENTITY dejagnu-copyright " -Copyright 92 - 2001, 2002 Free Software -Foundation, Inc. +Copyright 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, + 2002, 2003, 2004 Free Software Foundation, Inc. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice @@ -54,7 +54,7 @@ into another language, under the above conditions for modified versions. <bookinfo> <title>&dj;</title> <subtitle>The GNU Testing Framework</subtitle> - <date>2002 Sept 04</date> + <date>3 February 2004</date> <edition> &version</edition> <releaseinfo> New release</releaseinfo> <authorgroup> @@ -72,6 +72,10 @@ into another language, under the above conditions for modified versions. </authorblurb> --> </author> + <author> + <firstname>Ben Elliston</firstname> + <affiliation><orgname>Free Software Foundation</orgname></affiliation> + </author> </authorgroup> <address> <email>rob@welcomehome.org</email> @@ -118,28 +122,21 @@ into another language, under the above conditions for modified versions. <preface id=preface> <title>Abstract</title> - <para>This document describes the functionality of &dj;, the - testing framework of the GNU project. &dj; is written in - <productname>Expect</productname>, which uses - <productname>Tcl</productname> as a command - language. <productname>Expect</productname> acts as a very - programmable shell. As with other Unix command shells, you can - run any program, but once the program is started, your test script - has programmable control over its input and output. This does not - just apply to the programs under test; - <productname>Expect</productname> can also run any auxiliary - program, such as <command>diff</command> or <command>sh</command>, - with full control over its input and output.</para> - - <para>&dj; itself is merely a framework for the creation of - testsuites. Testsuites are distributed with each - application.</para> - - </preface> - - <chapter id=overview xreflabel=Overview> - <title>Overview</title> - <sect1 id=whatis xreflabel="What is &dj; ?"> + <para>This document describes the functionality of DejaGnu, the + testing framework of the GNU Project. DejaGnu is a framework for + testing other programs and many widely used GNU packages such as + the GNU Compiler Collection (GCC) and the GNU Debugger (GDB) are + tested using DejaGnu. Its purpose is to provide a uniform testing + environment. Programs can be just as easily tested locally or on a + remote computer of a different type than the computer running + DejaGnu. Each program can have multiple testsuites, all supported + by a single test harness. &dj; is written in + <productname>Expect</productname> which in turn relies upon + <productname>Tcl</productname>.</para> </preface> + + <chapter id=overview xreflabel=Introduction> + <title>Introduction</title> + <sect1 id=whatis xreflabel="What is &dj;?"> <title>What is &dj; ?</title> <para><productname>&dj;</productname> is a framework for @@ -158,13 +155,12 @@ into another language, under the above conditions for modified versions. <ulink URL="http://expect.nist.gov">Expect home page</ulink>, respectively.</para> - <para>Julia Menapace first coined the term ``&dj;'' to describe - an earlier testing framework she had written at Cygnus Support - for <command>GDB</command>. When it was replaced it with the - Expect-based framework, it was like &dj; all over again. - More importantly, it was also named after my daughter, <ulink - URL="mailto:deja@welcomehome.org">Deja Snow Savoye</ulink> - (now 13 years old as of September 2003), who was a toddler + <para>Julia Menapace first coined the term ``&dj;'' to describe an + earlier testing framework she had written at Cygnus Support + for testing <command>GDB</command>. When it was replaced it + with the Expect-based framework, it was like &dj; all over + again. More importantly, it was also named after Deja Snow + Savoye (13 years old as of September 2003), who was a toddler during &dj;'s beginnings.</para> <para>There are several advantages to testing with &dj;:</para> @@ -207,7 +203,7 @@ into another language, under the above conditions for modified versions. <productname>Tcl</productname> script to run a testsuite that is not based on <productname>Expect</productname>. <productname>Expect</productname> - scriptsuse <emphasis>.exp</emphasis> as a filename extension as a + scripts use <emphasis>.exp</emphasis> as a filename extension as a convention.</para> </sect1> @@ -457,6 +453,14 @@ into another language, under the above conditions for modified versions. </chapter> +<chapter id=install> + <title>Installation</title> + <sect1> + <title>Foo</title> + <para>Foo.</para> + </sect1> +</chapter> + <!-- include the user manual --> &user; diff --git a/doc/user.sgml b/doc/user.sgml index 5f71f85..ffaa6a5 100644 --- a/doc/user.sgml +++ b/doc/user.sgml @@ -2,11 +2,19 @@ <title>Getting DejaGnu up and running</title> <para>This chapter was originally written by Niklaus Giger (ngiger@mus.ch) because he lost a week to figure out how DejaGnu works and how to write a first test.</para> -<para>Follow these instructions as closely a possible in order get a good insight into how DejaGnu works, else you might run into a lot of subtle problems. You have been warned.</para> -<para>It should be no big problems installing DejaGnu using your package manager or from the source code. Under a Debian/GNU/Linux systems just type (as root) <programlisting>apt-get dejagnu</programlisting>. These examples were run on a primary machine with a AMD K6 and a Mac Powerbook G3 serving as a remote target.</para> +<para>Follow these instructions as closely a possible in order get a +good insight into how DejaGnu works, otherwise you might run into a +lot of subtle problems.</para> -<para> The tests for Windows were run under Windows NT using the actual Cygwin version (1.3.x as of October 2001). It's target system was a PPC embedded system running vxWorks. -</para> +<para>It should be no problem to install DejaGnu using your package +manager or from the source code. Under a Debian/GNU/Linux systems just +type (as root) <programlisting>apt-get dejagnu</programlisting>. The +examples given in this chapter were run on an AMD K6 machine with a +AMD K6 and a Mac Powerbook G3 acting as a remote target.</para> + +<para>The tests for Windows were run under Windows NT using +Cygwin. Its target system was a PowerPC embedded system running +vxWorks.</para> <sect1> <title>Test your installation</title> @@ -18,7 +26,8 @@ dgt:~$ mkdir ~/dejagnu.test dgt:~$ cd ~/dejagnu.test </programlisting> -<para>Now you are ready to test DejaGnu's main program called runtest. The expecteted output is shown</para> +<para>Now you are ready to test DejaGnu's main program called +runtest. The expected output is shown below.</para> <example> <title>Runtest output in a empty directory @@ -36,7 +45,8 @@ Using /usr/share/dejagnu/config/unix.exp as generic interface file for target. ERROR: Couldn't find tool config file for unix. === Summary ===</programlisting> -<para>We will show you later how to get rid of all the WARNING- and ERROR-messages. The files testrun.sum and testrun.log have been created, which do not interest us at this point. Let's remove them.</para> +<para>Do not be concerned by the WARNING and ERROR messages at this +stage. The files testrun.sum and testrun.log will be created. They are of no interest at this point, so they can be removed:</para> <programlisting>:~/dejagnu.test$ rm testrun.sum testrun.log </programlisting> @@ -45,31 +55,46 @@ ERROR: Couldn't find tool config file for unix. <sect2> <title>Windows</title> -<para>On Windows systems DejaGnu is part of a port of a lot of Unix tools to the Windows OS, called Cygwin. Cygwin may be downloaded and installed from a mirror of http://www.cygwin.com/. All examples were also run on Windows NT. If nothing is said, you can assume that you should get the same output as on a Unix system.</para> +<para>On a Cygwin system, DejaGnu can be installed from the Cygwin +packages collection. Cygwin may be downloaded and installed from a +mirror of <ulink +URL="http://www.cygwin.com">http://www.cygwin.com</ulink>. Unless +mentioned explicitly, you can assume that the output is identical to +that of a Unix system.</para> -<para>You will need a telnet daemon if you want to use a Windows box as a remote target. There seems to be a freeware telnet daemon at http://www.fictional.net/.</para> +<para>You will need to install the telnet server from the Cygwin +inetutils package if you want to use a Cygwin system as a remote +target.</para> </sect2> <sect2> <title>Getting the source code for the calc example</title> -<para>If you are running a Debian distribution you can find the examples under /usr/share/doc/dejagnu/examples. -These examples seem to be missing in Red Hat's RPM. -In this case download the sources of DejaGnu and adjust the pathes to the DejaGnu examples accordingly.</para> +<para>If you are running a Debian distribution you can find the +examples under /usr/share/doc/dejagnu/examples. These examples seem +to be missing in Red Hat's RPM. In this case download the sources of +DejaGnu and adjust the pathes to the DejaGnu examples +accordingly.</para> </sect2> </sect1> <sect1> <title>Create a minimal project, e.g. calc</title> -<para>In this section you will to start a small project, -using the sample application calc, which is part of your DejaGnu distribution</para> +<para>In this section you will start a small project, using the sample +application calc, which is part of the DejaGnu distribution.</para> <sect2><title>A simple project without the GNU autotools</title> -<para>The runtest program can be run standalone. All the autoconf/automake support is just cause those programs are commonly used for other GNU applications. The key to running runtest standalone is having the local site.exp file setup correctly, which automake does.</para> +<para>The runtest program can be run standalone. All the +autoconf/automake support is because those programs are commonly used +for other GNU applications. The key to running runtest standalone is +having the local <filename>site.exp</filename> file setup correctly, +which automake does.</para> + +<para>The generated <filename>site.exp</filename> should look +like:</para> -<para>The generated site.exp should like like:</para> <programlisting> set tool calc set srcdir . @@ -79,47 +104,60 @@ set objdir /home/dgt/dejagnu.test <sect2> <title>Using autoconf/autoheader/automake</title> -<para>We have to prepare some input file in order to run autocon and automake. There is book “GNU autoconf, automake and libtool” by Garry V. Vaughan, et al. NewRider, ISBN 1-57870-190-2 which describes this process thoroughly.</para> +<para>We have to prepare some input file in order to run autocon and +automake. There is book “GNU autoconf, automake and +libtool” by Garry V. Vaughan, et al. NewRider, ISBN +1-57870-190-2 which describes this process thoroughly.</para> -<para>From the calc example distributed with the DejaGnu documentation you should copy the program file itself (calc.c) and some additional files, which you might examine a little bit close to derive their meanings.</para> +<para>From the calc example distributed with the DejaGnu documentation +you should copy the program file itself (calc.c) and some additional +files, which you might examine a little bit close to derive their +meanings.</para> <programlisting> dgt:~/dejagnu.test$ cp -r /usr/share/doc/dejagnu/examples/calc/\ {configure.in,Makefile.am,calc.c,testsuite} . </programlisting> -<para>In Makemake.am note the presence of the AUTOMAKE_OPTIONS = dejagnu. This option is needed.</para> +<para>In <filename>Makemake.am</filename>, note the presence of the +<programlisting>AUTOMAKE_OPTIONS = dejagnu</programlisting>. This +option is required.</para> -<para>Run aclocal to generate aclocal.m4, which is a collection of macros needed by configure.in</para> +<para>Run <command>aclocal</command> to generate +<filename>aclocal.m4</filename>, which is a collection of macros +needed by <productname>Autoconf</productname>.</para> <programlisting> dgt:~/dejagnu.test$ aclocal </programlisting> -<para>autoconf is another part of the auto-tools. -Run it to generate configure based on information contained in configure.in.</para> +<para>autoconf is another part of the auto-tools. Run it to generate +the <filename>configure</filename> script from +<filename>configure.in</filename>. <programlisting> dgt:~/dejagnu.test$ autoconf </programlisting> -<para>autoheader is another part of the auto-tools. -Run it to generate calc.h.in. </para> +<para>autoheader is another part of the auto-tools. Run it to +generate <filename>calc.h.in</filename>. </para> <programlisting> dgt:~/dejagnu.test$ autoheader </programlisting> -<para>The Makefile.am of this example was developed as port of the DejaGnu -distribution. -Adapt Makefile.am for this test. Replace the line +<para>The <filename>Makefile.am</filename> of this example was +developed as port of the DejaGnu distribution. Adapt +<filename>Makefile.am</filename> for this test. Replace the line “#noinst_PROGRAMS = calc” to “bin_PROGRAMS = calc”. Change the RUNTESTDEFAULTFLAGS from “$$srcdir/testsuite” to “./testsuite”.</para> -<para>Running automake at this point contains a series of warning in its output as shown in the following example:</para> +<para>Running <command>automake</command> at this point contains a +series of warning in its output as shown in the following +example:</para> <example> <title>Sample output of automake with missing files</title> @@ -140,12 +178,16 @@ Makefile.am:6: required directory ./doc does not exist </example> <para>Create a empty directory doc and empty files -INSTALL, NEWS, README, AUTHORS, ChangeLog and COPYING. -The default COPYING will point to the GNU Public License (GPL). -In a real project it would be time to add some meaningfull text in each file. -</para> +<filename>INSTALL</filename>, <filename>NEWS</filename>, +<filename>README</filename>, <filename>AUTHORS</filename>, +<filename>ChangeLog</filename> and <filename>COPYING</filename>. The +default <filename>COPYING</filename> will point to the GNU Public +License (GPL). In a real project it would be time to add some +meaningful text in each file.</para> + +<para>Adapt calc to your environment by running +<command>configure</command>.</para> -<para>Adapt calc to your environment by calling configure.</para> <example> <title>Sample output of configure </title> @@ -175,10 +217,6 @@ creating Makefile creating calc.h </programlisting> </example> -<para>If you are familiar with GNU software, -this output should not contain any surprise to you. -Any errors should be easy to fix for such a simple program.</para> - <para>Build the calc executable:</para> <example> @@ -225,9 +263,11 @@ calc: quit </programlisting> <para>Look at the intentional bug that 2 times 4 equals 12.</para> -<para>The tests run by DejaGnu need a file called site.exp, -which is automatically generated if we call “make site.exp”. -This was the purpose of the “AUTOMAKE_OPTIONS = dejagnu” in Makefile.am.</para> +<para>The tests run by DejaGnu need a file called +<filename>site.exp</filename>, which is automatically generated if we +run <command>make site.exp</command>. This was the purpose of the +<programlisting>AUTOMAKE_OPTIONS = dejagnu</programlisting> in +<filename>Makefile.am</filename>.</para> <example> <title>Sample output generating a site.exp</title> @@ -243,10 +283,13 @@ Making a new site.exp file... </sect1> <sect1> -<title>Our first automated tests</title> -<sect2><title>Running the test for the calc example</title> +<title>Running automated tests</title> +<sect2><title>Running the calc testsuite</title> -<para>Now we are ready to call the automated tests </para> +<para>This section describes how to run the DejaGnu testsuite for the +calc example program. Most packages provide a +<command>check</command> <filename>Makefile</filename> target for this +purpose.</para> <example> <title>Sample output of runtest in a configured directory</title> @@ -287,49 +330,60 @@ make[1]: Leaving directory `/home/Dgt/dejagnu.test' make: *** [check-am] Fehler </programlisting> </example> -<para>Did you see the line “FAIL:“? The test cases for calc catch the bug in the calc.c file. Fix the error in calc.c later as the following examples assume a unchanged calc.c.</para> - -<para>Examine the output files calc.sum and calc.log. -Try to understand the testcases written in ~/dejagnu.test/testsuite/calc.test/calc.exp. -To understand Expect you might take a look at the book "Exploring Expect", -which is an excellent resource for learning and using Expect. (Pub: O'Reilly, ISBN 1-56592-090-2) -The book contains hundreds of examples and also includes a tutorial on Tcl. -Exploring Expect is 602 pages long. </para> -</sect2> +<para>The “FAIL:“ line shows that test cases for +<productname>calc</productname> catch the bug in +<filename>calc.c</filename>.</para> +<para>Examine the output files <filename>calc.sum</filename> and +<filename>calc.log</filename>. Try to understand the tests in +<filename>example/calc/testsuite/calc.test/calc.exp</filename>. To +understand <productname>Expect</productname> you might take a look at +the book "Exploring Expect", which is an excellent resource +for learning and using Expect. (Pub: O'Reilly, ISBN 1-56592-090-2) The +book contains hundreds of examples and also includes a tutorial on +Tcl.</para> </sect2> -<sect2><title>The various config files or how to avoid warnings</title> +<sect2><title>The various configuration files (how to avoid warnings)</title> -<para>DejaGnu may be customized by each user. It first searches for a file called ~/.dejagnurc. Create the file ~/.dejagnurc and insert the following line:</para> +<para>DejaGnu may be customized by each user. It first searches for a +file called <filename>.dejagnurc</filename> in the user's home +directory. Create a <filename>.dejagnurc</filename> file and insert +the following line:</para> <programlisting> puts "I am ~/.dejagnurc" </programlisting> -<para>Rerun make check. Test whether the output contains "I am ~/.dejagnurc". -Create ~/my_dejagnu.exp and insert the following line:</para> +<para>Re-run <command>make check</command> and note that the test +output contains "I am ~/.dejagnurc". Now create +<filename>~/my_dejagnu.exp</filename> and insert the following +line into that file:</para> <programlisting> puts "I am ~/my_dejagnu.exp" </programlisting> -<para>In a Bash-Shell enter</para> +<para>In a Bourne shell, enter:</para> <programlisting> -dgt:~/dejagnu.test$ export DEJAGNU=~/my_dejagnu.exp +export DEJAGNU=~/my_dejagnu.exp </programlisting> -<para>Run “make check” again. The output should not contain -“WARNING: Couldn't find the global config file.”. -Create the sub-directory lib. Create the file “calc.exp” in it and insert the following line:</para> +<para>Run <command>make check</command> again. The output should not +contain a warning that reads “WARNING: Couldn't find the global +config file.”. Create a subdirectory called +<filename>lib</filename> and within that directory, create a file +called <filename>calc.exp</filename>. Insert the following line into +that file:</para> <programlisting> puts "I am lib/calc.exp" </programlisting> -<para>The last warning “WARNING: Couldn't find tool init file” -should not be part of the output of make check. -Create the directory ˜/boards. Create the file ˜/boards/standard.exp and insert the following line:</para> +<para>The last warning “WARNING: Couldn't find tool init +file” should now be excluded from the output of <command>make +check</command>. Create the directory ˜/boards. Create the file +˜/boards/standard.exp and insert the following line:</para> <programlisting> puts "I am boards/standard.exp" @@ -370,16 +424,14 @@ for target. </programlisting> </example> -<para>It is up to you to decide when and where to use any of the above -mentioned config files for customizing. -This chapters showed you where and in which order the different config -files are run.</para> </sect2> <sect2> <title>When trouble strikes</title> -<para>Calling runtest with the '-v'-flag shows you in even more details which files are searched in which order. Passing it several times gives more and more details. </para> +<para>Calling runtest with the '-v'-flag shows you in even more +details which files are searched in which order. Passing it several +times gives more and more detail.</para> <example> <title>Displaying details about runtest execution</title> <programlisting> @@ -711,34 +763,38 @@ powerpc-linux-gcc -g -O2 -o calc calc.o <chapter id=runningtests> <title>Running Tests</title> - <para>There are two ways to execute a testsuite. The most - common way is when there is existing support in the - <filename>Makefile</filename>. This support consists of a - <emphasis>check</emphasis> target. The other way is to execute the - <command>runtest</command> program directly. To run - <command>runtest</command> directcly from the command line requires - either all the correct options, or the <xref linkend=local> must be setup - correctly.</para> + <para>There are two ways to execute a testsuite. The most common + way is to rely upon support in a <filename>Makefile</filename> for + a <emphasis>check</emphasis> target. The other way is to execute + the <command>runtest</command> program directly. To run + <command>runtest</command> directcly from the command line + requires either all the correct options, or the <xref + linkend=local> must be setup correctly. Automake can generate a + <filename>Makefile</filename> that does all of the right things + when the user invokes <command>make check</command> and is the + preferred approach. Both ways of executing a testsuite will be + covered in more detail below.</para> <sect1 id=makecheck xreflabel="Make Check"> - <title>Make check</title> + <title>make check</title> - <para>To run tests from an existing collection, first use - <command>configure</command> as usual to set up the - build directory. Then try typing:</para> + <para>To run tests from an existing collection, use + <command>configure</command> to configure a build directory and + then type:</para> <screen> make check </screen> - <para>If the <emphasis>check</emphasis> target exists, it - usually saves you some trouble. For instance, it can set up any - auxiliary programs or other files needed by the tests. The most - common file the check builds is the - <emphasis>site.exp</emphasis>. The site.exp file contains - various variables that DejaGnu used to dertermine the - configuration of the program being tested. This is mostly for - supporting remote testing.</para> + <para>If the <filename>Makefile</filename> has a + <emphasis>check</emphasis> target, it saves some effort. For + instance, it can set up any auxiliary programs or other files + needed by the tests. The most common file the + <emphasis>check</emphasis> target creates is + <filename>site.exp</filename>. The <filename>site.exp</filename> + file contains various variables that DejaGnu uses to determine + the configuration of the program being tested. This is mostly + used to support remote testing.</para> <para>The <emphasis>check</emphasis> target is supported by GNU <productname>Automake</productname>. To have DejaGnu support added to your @@ -746,13 +802,12 @@ powerpc-linux-gcc -g -O2 -o calc calc.o dejagnu to the AUTOMAKE_OPTIONS variable in your <filename>Makefile.am</filename> file.</para> - <para>Once you have run <emphasis>make check</emphasis> to build + <para>Once you have run <command>make check</command> to build any auxiliary files, you can invoke the test driver - <command>runtest</command> directly to repeat the tests. - You will also have to execute <command>runtest</command> - directly for test collections with no - <emphasis>check</emphasis> target in the - <filename>Makefile</filename>.</para> + <command>runtest</command> directly to repeat the tests. You + will also have to execute <command>runtest</command> directly + for test collections with no <emphasis>check</emphasis> target + in the <filename>Makefile</filename>.</para> </sect1> @@ -761,9 +816,9 @@ powerpc-linux-gcc -g -O2 -o calc calc.o <para><command>runtest</command> is the executable test driver for DejaGnu. You can specify two kinds of things on the - <command>runtest</command> command line: command line options, - and Tcl variables for the test scripts. The options are listed - alphabetically below.</para> + <command>runtest</command> command line: options and Tcl + variable assignments for the test scripts. The options are + listed alphabetically below.</para> <para><command>runtest</command> returns an exit code of <emphasis>1</emphasis> if any test has an unexpected result; otherwise @@ -771,59 +826,59 @@ powerpc-linux-gcc -g -O2 -o calc calc.o as the exit code.</para> <sect2 id=outputs xreflabel="Output States"> - <title>Output States</title> + <title>Test Result States</title> <para><filename>runtest</filename> flags the outcome of each - test as one of these cases. <xref linkend=posix> for a - discussion of how POSIX specifies the meanings of these + test as one of the following cases. See <xref linkend=posix> + for a discussion of how POSIX specifies the meanings of these cases.</para> <variablelist> <varlistentry> <term>PASS</term> - <listitem><para>The most desirable outcome: the test succeeded, and - was expected to succeed.</para></listitem> + <listitem><para>The most desirable outcome: the test was + expected to succeed and did succeed.</para></listitem> </varlistentry> <varlistentry> <term>XPASS</term> - <listitem><para>A pleasant kind of failure: a test was expected to - fail, but succeeded. This may indicate progress; inspect the test - case to determine whether you should amend it to stop expecting - failure.</para></listitem> + <listitem><para>A pleasant kind of failure: a test was + expected to fail, but succeeded. This may indicate progress; + inspect the test case to determine whether you should amend + it to no longer expect failure.</para></listitem> </varlistentry> <varlistentry> <term>FAIL</term> - <listitem><para>A test failed, although it was expected to succeed. - This may indicate regress; inspect the test case and the failing - software to ocate the bug.</para></listitem> - </varlistentry> + <listitem><para>A test was expected to succeed, but failed. + This may indicate a regression; inspect the test case and + the failing software to locate the bug.</para></listitem> + </varlistentry> <varlistentry> <term>XFAIL</term> - <listitem><para>A test failed, but it was expected to fail. This - result indicates no change in a known bug. If a test fails because - the operating system where the test runs lacks some facility required - by the test, the outcome is <emphasis>UNSUPPORTED</emphasis> - instead.</para></listitem> - </varlistentry> + <listitem><para>A test that was expected to fail did fail. + This result indicates no change in a known bug. If a test + fails because the environment running the test lacks some + facility required by the test, the outcome is + <emphasis>UNSUPPORTED</emphasis> instead.</para></listitem> + </varlistentry> <varlistentry> <term>UNRESOLVED</term> - <listitem><para>Output from a test requires manual inspection; the - testsuite could not automatically determine the outcome. For - example, your tests can report this outcome is when a test does not - complete as expected.</para></listitem> + <listitem><para>Output from an unresolved test requires + manual inspection, as the testsuite could not automatically + determine the outcome. A test can report this outcome, for + instance, when a test is not completed as expected.</para></listitem> </varlistentry> <varlistentry> <term>UNTESTED</term> - <listitem><para>A test case is not yet complete, and in particular - cannot yet produce a <emphasis>PASS</emphasis> or - <emphasis>FAIL</emphasis>. You can also use this outcome in dummy - ``tests'' that note explicitly the absence of a real test case for a - particular property.</para></listitem> + <listitem><para>A test case is not yet complete, and in + particular cannot yet produce a <emphasis>PASS</emphasis> or + <emphasis>FAIL</emphasis>. You can also use this outcome for + placeholder tests that note explicitly the absence of a real + test case for a particular property.</para></listitem> </varlistentry> <varlistentry> @@ -836,7 +891,8 @@ powerpc-linux-gcc -g -O2 -o calc calc.o </varlistentry> </variablelist> - <para>runtest may also display the following messages:</para> + <para><command>runtest</command> may also display the following + messages:</para> <variablelist> <varlistentry> @@ -853,21 +909,20 @@ powerpc-linux-gcc -g -O2 -o calc calc.o <varlistentry> <term>WARNING</term> <listitem><para>Indicates a possible problem in running the - test. Usually warnings correspond to recoverable errors, or display - an important message about the following tests.</para></listitem> + test. Usually warnings correspond to recoverable errors or + display an important message.</para></listitem> </varlistentry> <varlistentry> <term>NOTE</term> - <listitem><para>An informational message about the test - case.</para></listitem> + <listitem><para>A message about the test case.</para></listitem> </varlistentry> </variablelist> </sect2> <sect2 id=invoking xreflabel="Invoking Runtest"> - <title>Invoking Runtest</title> + <title>Invoking runtest</title> <para>This is the full set of command line options that <filename>runtest</filename> recognizes. Arguments may be @@ -875,46 +930,50 @@ powerpc-linux-gcc -g -O2 -o calc calc.o <variablelist> <varlistentry> - <term><option>--all</option> (-a)</term> + <term><option>--all</option>, <option>-a</option></term> <listitem><para>Display all test output. By default, - <emphasis>runtest</emphasis> shows only the output of tests that - produce unexpected results; that is, tests with status - <emphasis>FAIL</emphasis> (unexpected failure), - <emphasis>XPASS</emphasis> (unexpected success), or - <emphasis>ERROR</emphasis> (a severe error in the test case - itself). Specify <emphasis>--all</emphasis> to see output for tests - with status <emphasis>PASS</emphasis> (success, as expected) - <emphasis>XFAIL</emphasis> (failure, as expected), or - <emphasis>WARNING</emphasis> (minor error in the test case - itself).</para></listitem> - </varlistentry> + <command>runtest</command> shows only the output of tests + that produce unexpected results. That is, tests with result + states of <emphasis>FAIL</emphasis>, + <emphasis>XPASS</emphasis>, or + <emphasis>ERROR</emphasis>. Specifying + <command>--all</command> will include output for tests with + result states <emphasis>PASS</emphasis>, + <emphasis>XFAIL</emphasis> and + <emphasis>WARNING</emphasis>.</para></listitem> + </varlistentry> <varlistentry> <term><option>--build [string]</option></term> - <listitem><para><emphasis>string</emphasis> is a full configuration - ``triple'' name as used by <command>configure</command>. This - is the type of machine DejaGnu and the tools to be tested are built - on. For a normal cross this is the same as the host, but for a - canadian cross, they are seperate.</para></listitem> + <listitem><para><emphasis>string</emphasis> is a system + triplet as used by <command>configure</command>. This is the + type of system the program to be tested is built on. For a + normal cross-compiler this is the same as the host triplet, + but for a Canadian cross-compiler, they are + distinct.</para></listitem> </varlistentry> <varlistentry> <term><option>--host [string]</option></term> - <listitem><para><symbol>string</symbol> is a full configuration - ``triple'' name as used by <emphasis>configure</emphasis>. Use this + + <listitem><para><emphasis>string</emphasis> is a system + triplet as used by <command>configure</command>. Use this option to override the default string recorded by your - configuration's choice of host. This choice does not change how - anything is actually configured unless --build is also specified; it - affects <emphasis>only</emphasis> DejaGnu procedures that compare the - host string with particular values. The procedures + configuration's choice of host. This choice does not change + how anything is actually configured unless + <option>--build</option> is also specified; it affects + <emphasis>only</emphasis> DejaGnu procedures that compare + the host string with particular values. The procedures <emphasis>ishost</emphasis>, <emphasis>istarget</emphasis>, - <emphasis>isnative</emphasis>, and <emphasis>setup</emphasis>xfail} - are affected by <emphasis>--host</emphasis>. In this usage, - <emphasis>host</emphasis> refers to the machine that the tests are to - be run on, which may not be the same as the - <emphasis>build</emphasis> machine. If <emphasis>--build</emphasis> - is also specified, then <emphasis>--host</emphasis> refers to the - machine that the tests wil, be run on, not the machine DejaGnu is run + <emphasis>isnative</emphasis>, and + <emphasis>setup</emphasis>xfail} are affected by + <emphasis>--host</emphasis>. In this usage, + <emphasis>host</emphasis> refers to the machine that the + tests are to be run on, which may not be the same as the + <emphasis>build</emphasis> machine. If + <emphasis>--build</emphasis> is also specified, then + <emphasis>--host</emphasis> refers to the machine that the + tests wil, be run on, not the machine DejaGnu is run on.</para></listitem> </varlistentry> @@ -925,152 +984,154 @@ powerpc-linux-gcc -g -O2 -o calc calc.o <varlistentry> <term><option>--target [string]</option></term> - <listitem><para>Use this option to override the default setting - (running native tests). <emphasis>string</emphasis> is a full - configuration ``triple'' name of the form - <emphasis>cpu-vendor-os</emphasis> as used by - <command>configure</command>. This option changes the - configuration <emphasis>runtest</emphasis> uses for the default tool - names, and other setup information.</para></listitem> + <listitem><para>Use this option to override the default + target setting. <emphasis>string</emphasis> is a system + triplet as used by <command>configure</command>. This option + changes the configuration <command>runtest</command> uses + for the default tool names, and other setup + information.</para></listitem> </varlistentry> <varlistentry> - <term><option>--debug</option> (-de)</term> - <listitem><para>Turns on the <emphasis>expect</emphasis> internal - debugging output. Debugging output is displayed as part of the - <emphasis>runtest</emphasis> output, and logged to a file called - <filename>dbg.log</filename>. The extra debugging output does - <emphasis>not</emphasis> appear on standard output, unless the - verbose level is greater than 2 (for instance, to see debug output - immediately, specify <emphasis>--debug</emphasis>-v -v}). The - debugging output shows all attempts at matching the test output of - the tool with the scripted patterns describing expected output. The - output generated with <emphasis>--strace</emphasis> also goes into - <filename>dbg.log</filename>.</para></listitem> + <term><option>--debug</option>, <option>-de</option></term> + <listitem><para>Enables internal + <productname>Expect</productname> debug output. Debug + output is displayed as part of the + <command>runtest</command> output and is additionally logged + to a file called <filename>dbg.log</filename>. The extra + debugging output does <emphasis>not</emphasis> appear on + standard output, unless the verbose level is greater than 2. + For instance, to see debug output immediately, specify + <option>--debug</option> <option>-v</option> + <option>-v</option>. The debugging output shows all + attempts at matching the output of the program under test + with the scripted patterns describing expected + output.</para></listitem> </varlistentry> <varlistentry> - <term><option>--help</option> (-he)</term> - <listitem><para>Prints out a short summary of the - <emphasis>runtest</emphasis> options, then exits (even if you also - specify other options).</para></listitem> + <term><option>--help</option>, <option>-he</option></term> + <listitem><para>Prints a summary of + <command>runtest</command> options and then exits. This + option overrides all other options in this + regard.</para></listitem> </varlistentry> <varlistentry> - <term><option>--ignore [name(s)] </option></term> + <term><option>--ignore [test(s)] </option></term> <listitem><para>The names of specific tests to ignore.</para></listitem> </varlistentry> <varlistentry> <term><option>--objdir [path]</option></term> - <listitem><para>Use <emphasis>path</emphasis> as the top directory - containing any auxiliary compiled test code. This defaults to - <filename>.</filename>. Use this option to locate pre-compiled test - code. You can normally prepare any auxiliary files needed with - <emphasis>make</emphasis>.</para></listitem> + <listitem><para>Use <emphasis>path</emphasis> as the top + directory containing any auxiliary pre-compiled test + code. This defaults to <filename>.</filename> and a + <filename>Makefile</filename> can be used to prepare any + auxiliary files that are needed.</para></listitem> </varlistentry> <varlistentry> <term><option>--outdir [path]</option></term> <listitem><para>Write output logs in directory - <filename>path</filename>. The default is <emphasis>.}, - the</emphasis> directory where you start - <emphasis>runtest</emphasis>. This option affects only the summary - and the detailed log files - <filename>tool.sum</filename> and - <filename>tool.log</filename>. The DejaGnu debug - log <filename>dbg.log</filename> always appears (when requested) in - the local directory.</para></listitem> + <filename>path</filename>. The default is + <filename>.</filename> and is the directory where you invoke + <command>runtest</command>. This option affects only the + summary (<filename>.sum</filename>) and the detailed log + (<filename>.log</filename>) files. The debug log + <filename>dbg.log</filename> is always written to the + current working directory.</para></listitem> </varlistentry> <varlistentry> <term><option>--reboot [name]</option></term> <listitem><para>Reboot the target board when - <emphasis>runtest</emphasis> initializes. Usually, when running tests - on a separate target board, it is safer to reboot the target to be - certain of its state. However, when developing test scripts, - rebooting takes a lot of time.</para></listitem> + <command>runtest</command> initializes. When running tests + on a separate target board, it is generally safer to reboot + the target to be certain of its state. However, when + developing test scripts, rebooting takes a lot of time and + can reduce the life of prototype boards.</para></listitem> </varlistentry> <varlistentry> <term><option>--srcdir [path]</option></term> - <listitem><para>Use <filename>path</filename> as the top directory - for test scripts to run. <emphasis>runtest</emphasis> looks in this - directory for any subdirectory whose name begins with the toolname - (specified with <emphasis>--tool</emphasis>). For instance, with - <emphasis>--tool</emphasis>gdb}, <emphasis>runtest</emphasis> uses - tests in subdirectories <filename>gdb.*</filename> (with the usual - shell-like filename expansion). If you do not use - <emphasis>--srcdir</emphasis>, <emphasis>runtest</emphasis> looks for - test directories under the current working + + <listitem><para>Use <emphasis>path</emphasis> as the top + directory for test scripts to + run. <command>runtest</command> looks in this directory for + any subdirectory whose name begins with the tool name + (specified with <emphasis>--tool</emphasis>). For instance, + with <option>--tool gdb</option>, <command>runtest</command> + searches subdirectories matching <filename>gdb.*</filename> + for test scripts. If you do not use + <option>--srcdir</option>, <command>runtest</command> looks + for test directories under the current working directory.</para></listitem> + </varlistentry> <varlistentry> <term><option>--strace [number]</option></term> <listitem><para>Turn on internal tracing for - <emphasis>expect</emphasis>, to n levels deep. By adjusting the - level, you can control the extent to which your output expands - multi-level Tcl statements. This allows you to ignore some levels of - <emphasis>case</emphasis> or <emphasis>if</emphasis> statements. - Each procedure call or control structure counts as one ``level''. The - output is recorded in the same file, <filename>dbg.log</filename>, - used for output from <emphasis>--debug</emphasis>.</para></listitem> + <productname>Expect</productname>, to <emphasis>n</emphasis> + levels deep. By adjusting the level, you can control the + extent to which your output expands multi-level Tcl + statements. This allows you to ignore some levels of + <emphasis>case</emphasis> or <emphasis>if</emphasis> + statements. Each procedure call or control structure counts + as one level. The output is recorded in the debug log file + <filename>dbg.log</filename>.</para></listitem> </varlistentry> <varlistentry> <term><option>--connect [program]</option></term> - <listitem><para>Connect to a target testing environment as specified - by <emphasis>type</emphasis>, if the target is not the computer - running <emphasis>runtest</emphasis>. For example, use - <emphasis>--connect</emphasis> to change the program used to connect - to a ``bare board'' boot monitor. The choices for - <emphasis>type</emphasis> in the DejaGnu 1.4 distribution are - <emphasis>rlogin</emphasis>, <emphasis>telnet</emphasis>, - <emphasis>rsh</emphasis>, <emphasis>tip</emphasis>, - <emphasis>kermit</emphasis>, and <emphasis>mondfe</emphasis>.</para> - - <para>The default for this option depends on the configuration most - convenient communication method available, but often other - alternatives work as well; you may find it useful to try alternative - connect methods if you suspect a communication problem with your - testing target.</para></listitem> + <listitem><para>Connect to a target system using by + <emphasis>program</emphasis>, if the target system is + distinct from the computer running + <command>runtest</command>. The possible values for + <emphasis>program</emphasis> in the DejaGnu 1.4.4 + distribution are <emphasis>rlogin</emphasis>, + <emphasis>telnet</emphasis>, <emphasis>rsh</emphasis>, + <emphasis>tip</emphasis>, <emphasis>kermit</emphasis> and + <emphasis>mondfe</emphasis>.</para> </varlistentry> <varlistentry> - <term><option>--baud [number]</option></term> - <listitem><para>Set the default baud rate to something other than - 9600. (Some serial interface programs, like <emphasis>tip</emphasis>, - use a separate initialization file instead of this - value.)</para></listitem> + <term><option>--baud [rate]</option></term> + <listitem><para>Set the baud rate to + <emphasis>rate</emphasis> bits per second. Some serial + interface programs, such as <command>tip</command>, use a + separate initialization file and will ignore this + option.</para></listitem> </varlistentry> <varlistentry> - <term><option>--target_board [name(s)]</option></term> + <term><option>--target_board [board(s)]</option></term> <listitem><para>The list of target boards to run tests on.</para></listitem> </varlistentry> <varlistentry id=tool-opt> - <term><option>--tool[name(s)]</option></term> - <listitem><para>Specifies which testsuite to run, and what - initialization module to use. <option>--tool</option> is used - <emphasis>only</emphasis> for these two purposes. It is - <emphasis>not</emphasis> used to name the executable program to - test. Executable tool names (and paths) are recorded in - <filename>site.exp</filename> and you can override them by specifying - Tcl variables on the command line.</para> - - <para>For example, including "<option>--tool</option> gcc" on the - <emphasis>runtest</emphasis> command line runs tests from all test - subdirectories whose names match <filename>gcc.*</filename>, and uses - one of the initialization modules named - <filename>config/*-gcc.exp</filename>. To specify the name of the - compiler (perhaps as an alternative path to what - <emphasis>runtest</emphasis> would use by default), use - <emphasis>GCC=binname</emphasis> on the <emphasis>runtest</emphasis> - command line.</para></listitem> + <term><option>--tool [tool(s)]</option></term> + + <listitem><para>Specifies which testsuite to run and what + initialization module to use. <option>--tool</option> is + used only for these two purposes. It is + <emphasis>not</emphasis> used to name the executable program + to test. Executable tool names and pathsare recorded in + <filename>site.exp</filename> and you can override them by + specifying Tcl variable values on the command line.</para> + + <para>For example, including <option>--tool gcc</option> on + the <command>runtest</command> command line will run tests + from all subdirectories whose names match + <filename>gcc.*</filename> and will use one of the + initialization modules named + <filename>config/*-gcc.exp</filename>. To specify the path + to the compiler, use <option>GCC=/path/to/gcc</option> on + the <command>runtest</command> command + line.</para></listitem> </varlistentry> <varlistentry> @@ -1086,69 +1147,69 @@ powerpc-linux-gcc -g -O2 -o calc calc.o </varlistentry> <varlistentry> - <term><option>--verbose</option> (-v)</term> - <listitem><para>Turns on more output. Repeating this option increases - the amount of output displayed. Level one (<emphasis>-v</emphasis>) - is simply test output. Level two (<emphasis>-v</emphasis>-v}) shows - messages on options, configuration, and process control. Verbose - messages appear in the detailed (<filename>*.log</filename>) log - file, but not in the summary (<filename>*.sum</filename>) log - file.</para></listitem> - </varlistentry> + <term><option>--verbose</option>, <option>-v</option></term> + <listitem><para>Raises the level of output from + <command>runtest</command>. Repeating this option increases + the amount of output displayed. Level one + (<emphasis>-v</emphasis>) is simply test output. Level two + (<emphasis>-v</emphasis>-v}) shows messages on options, + configuration, and process control. Verbose messages appear + in the detailed log file (<filename>*.log</filename>), but + not in the summary log file + (<filename>*.sum</filename>).</para></listitem> + </varlistentry> <varlistentry> - <term><option>--version</option> (-V)</term> - <listitem><para>Prints out the version numbers of DejaGnu, - <emphasis>expect</emphasis> and Tcl, and exits without running any - tests.</para></listitem> + <term><option>--version</option>, <option>-V</option></term> + <listitem><para>Prints the version numbers of &dj;, + <productname>Expect</productname> and + <productname>Tcl</productname> and then terminates without + running any tests.</para></listitem> </varlistentry> <varlistentry> <term><option>--D[0-1]</option></term> - <listitem><para>Start the internal Tcl debugger. The Tcl debugger - supports breakpoints, single stepping, and other common debugging - activities. See the document "Debugger for Tcl Applications" by Don - Libes. (Distributed in PostScript form with - <emphasis>expect</emphasis> as the file - <filename>expect/tcl-debug.ps.</filename>. If you specify - <emphasis>-D1</emphasis>, the <emphasis>expect</emphasis> shell stops - at a breakpoint as soon as DejaGnu invokes it. If you specify - <emphasis>-D0</emphasis>, DejaGnu starts as usual, but you can enter - the debugger by sending an interrupt (e.g. by typing - <keycombo><keycap>C</keycap><keycap>c</keycap></keycombo>). + <listitem><para>Start the internal Tcl debugger. The Tcl + debugger supports breakpoints, single stepping, and other + common debugging activities. If you specify + <option>-D1</option>, the <command>expect</command> shell + stops at a breakpoint as soon as DejaGnu invokes it. If you + specify <emphasis>-D0</emphasis>, DejaGnu starts as usual, + but you can enter the debugger by sending an interrupt with + <keycombo><keycap>C</keycap><keycap>c</keycap></keycombo>. </para></listitem> </varlistentry> <varlistentry> - <term><filename>testfile</filename>.exp[=arg(s)]</term> - <listitem><para>Specify the names of testsuites to run. By default, - <emphasis>runtest</emphasis> runs all tests for the tool, but you can - restrict it to particular testsuites by giving the names of the - <emphasis>.exp expect</emphasis> scripts that control - them. <emphasis>testsuite</emphasis>.exp may not include path - information; use plain filenames.</para></listitem> - </varlistentry> - - <varlistentry> - <term><filename>testfile</filename>.exp="testfile1 ..."</term> - <listitem><para>Specify a subset of tests in a suite to run. For - compiler or assembler tests, which often use a single - <emphasis>.exp</emphasis> script covering many different source - files, this option allows you to further restrict the tests by - listing particular source files to compile. Some tools even support - wildcards here. The wildcards supported depend upon the tool, but - typically they are <emphasis>?</emphasis>, <emphasis>*</emphasis>, - and <emphasis>[chars]</emphasis>.</para></listitem> + <term><filename>testfile.exp</filename>[=arg(s)]</term> + <listitem><para>Specify the names of testsuites to run. By + default, <command>runtest</command> runs all tests for the + tool, but you can restrict it to particular testsuites by + giving the names of the <filename>.exp</filename> + <productname>Expect</productname> scripts that control + them. <emphasis>testfile.exp</emphasis> may not include + directory names; use base filenames only.</para> + + <para>By listing filenames in <emphasis>arg(s)</emphasis>, + it is possible to specify a subset of tests in a suite to + run. For compiler or assembler tests, which often use a + single <productname>Expect</productname> script covering + many different input files, this option allows you to + further restrict the tests by listing particular input files + to test. Some tools additionally support wildcards. The + wildcards supported depend upon the tool, but typically they + are <emphasis>?</emphasis>, <emphasis>*</emphasis>, and + <emphasis>[chars]</emphasis>.</para></listitem> </varlistentry> <varlistentry> <term><symbol>tclvar</symbol>=value</term> - <listitem><para>You can define Tcl variables for use by your test - scripts in the same style used with <emphasis>make</emphasis> for - environment variables. For example, <emphasis>runtest - GDB=gdb.old</emphasis> defines a variable called - <command>GDB</command>; when your scripts refer to - <symbol>$GDB</symbol> in this run, they use the value + <listitem><para>You can define Tcl variables for use by your + test scripts in the same style used by + <command>make</command> for environment variables. For + example, <command>runtest GDB=gdb.old</command> defines a + Tcl variable called <symbol>GDB</symbol>. When test scripts + refer to <symbol>$GDB</symbol>, they will receive the value <emphasis>gdb.old</emphasis>.</para> <para>The default Tcl variables used for most tools are defined in @@ -1158,24 +1219,25 @@ powerpc-linux-gcc -g -O2 -o calc calc.o </variablelist> </sect2> - <sect2 id=common xreflabel="Common Operations"> + <sect2 id=common xreflabel="Common Options"> <title>Common Options</title> - <para>Typically, you don't need must to use any command-line options. - <option>--tool</option> used is only required when there are more than - one testsuite in the same directory. The default options are in the - local site.exp file, created by "make site.exp".</para> + <para>Typically, no command line options are required. The + <option>--tool</option> option is only required when there is + more than one testsuite in the same directory. The default + options are in the local site.exp file, created by + <command>make site.exp</command>.</para> <para>For example, if the directory <filename>gdb/testsuite</filename> contains a collection of DejaGnu tests for GDB, you can run them like this:</para> <screen> - eg$ cd gdb/testsuite - eg$ runtest --tool gdb + $ cd gdb/testsuite + $ runtest --tool gdb </screen> - <para>Test output follows, ending with:</para> + <para>Test output will follow, ending with:</para> <screen> === gdb Summary === @@ -1189,7 +1251,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o some other directory containing a collection of tests:</para> <screen> - eg$ runtest--srcdir /devo/gdb/testsuite + $ runtest --srcdir /devo/gdb/testsuite </screen> <para>By default, <command>runtest</command> prints only the @@ -1199,49 +1261,41 @@ powerpc-linux-gcc -g -O2 -o calc calc.o as expected), use the <emphasis>--all</emphasis> option. For more verbose output about processes being run, communication, and so on, use <emphasis>--verbose</emphasis>. To see even more output, use multiple - <emphasis>--verbose</emphasis> options. for a more detailed explanation - of each <command>runtest</command> option.</para> - - <para>Test output goes into two files in your current directory: - summary output in <filename>tool.sum</filename>, - and detailed output in <filename> - tool.log</filename>. (<emphasis>tool</emphasis> - refers to the collection of tests; for example, after a run with - <emphasis>--tool</emphasis> gdb, look for output files - <filename>gdb.sum</filename> and - <filename>gdb.log</filename>.)</para> - </sect2> - </sect1> + <emphasis>--verbose</emphasis> options.</para> - <sect1 id=outputfiles xreflabel="Output Files"> + <para>Test output goes into two files in your current + directory: summary output in <filename>tool.sum</filename>, + and detailed output in <filename> tool.log</filename>. Here, + <emphasis>tool</emphasis> refers to the collection of tests. + After a run with <emphasis>--tool</emphasis> gdb, the output + files will be named <filename>gdb.sum</filename> and + <filename>gdb.log</filename>.</para> </sect2> </sect1> - <title>The files DejaGnu produces.</title> + <sect1 id=outputfiles xreflabel="Output Files"> + <title>DejaGnu output files</title> - <para>DejaGnu always writes two kinds of output files: summary - logs and detailed logs. The contents of both of these are - determined by your tests.</para> + <para>When <command>runtest</command> is invoked, DejaGnu + generates two output files: a summary log and a detailed log. The + contents of these are determined by the test scripts.</para> - <para>For troubleshooting, a third kind of output file is useful: - use <option>--debug</option> to request an output file showing - details of what <productname>Expect</productname> is doing - internally.</para> + <para>For troubleshooting, a third kind of output file can be + requested with the <option>--debug</option> option. This file, + called <filename>dbg.log</filename> shows what + <productname>Expect</productname> is doing internally.</para> <sect2 id=sum xreflabel="Summary File"> <title>Summary File</title> - <para>DejaGnu always produces a summary output file - <filename>tool.sum</filename>. This summary shows the names of - all test files run; for each test file, one line of output from - each <command>pass</command> command (showing status - <emphasis>PASS</emphasis> or <emphasis>XPASS</emphasis>) or - <command>fail</command> command (status - <emphasis>FAIL</emphasis> or <emphasis>XFAIL</emphasis>); - trailing summary statistics that count passing and failing tests - (expected and unexpected); and the full pathname and version - number of the tool tested. (All possible outcomes, and all - errors, are always reflected in the summary output file, - regardless of whether or not you specify - <option>--all</option>.)</para> + <para>DejaGnu always produces a summary output file called + <filename>tool.sum</filename>, where <emphasis>tool</emphasis> + is the name of the tool under test. This summary shows the names + of all test scripts run and, for each test script, one line of + output for each test result, trailing summary statistics that + tally the number of passing and failing tests (both expected and + unexpected); and the full pathname and version number of the + tool tested. All possible test outcomes and errors are included + in the summary output file, regardless of whether or not you + specify the <option>--all</option> option.</para> <para>If any of your tests use the procedures <command>unresolved</command>, <command>unsupported</command>, @@ -1249,17 +1303,18 @@ powerpc-linux-gcc -g -O2 -o calc calc.o tabulates the corresponding outcomes.</para> <para>For example, after <command>runtest --tool - binutils</command>, look for a summary log in + binutils</command>, &dj; will produce a summary log file called <filename>binutils.sum</filename>. Normally, DejaGnu writes this file in your current working directory; use the <option>--outdir</option> option to select a different directory.</para> <example> - <title>Here is a short sample summary log</title> + <title>Sample summary log</title> <screen> - Test Run By rob on Mon May 25 21:40:57 PDT 1992 + Test Run By rob on Tue Feb 3 23:14:04 2004 + === gdb tests === Running ./gdb.t00/echo.exp ... PASS: Echo test @@ -1284,22 +1339,22 @@ powerpc-linux-gcc -g -O2 -o calc calc.o <sect2 id=log xreflabel="Log File"> <title>Log File</title> - <para>DejaGnu also saves a detailed log file + <para>DejaGnu also produces a detailed log file called <filename>tool.log</filename>, showing any output generated by tests as well as the summary output. For example, after - <command>runtest --tool binutils</command>, look for a detailed - log in <filename>binutils.log</filename>. Normally, DejaGnu - writes this file in your current working directory; use the + <command>runtest --tool binutils</command>, &dj; will produce a + detailed log file named + <filename>binutils.log</filename>. Normally, DejaGnu writes this + file in your current working directory; use the <option>--outdir</option> option to select a different directory.</para> <example> - <title>Here is a brief example showing a detailed log for - <productname>G++</productname> tests</title> + <title>Detailed log for <productname>G++</productname> tests</title> <screen> - Test Run By rob on Mon May 25 21:40:43 PDT 1992 + Test Run By rob on Tue Feb 3 23:16:23 2004 === g++ tests === @@ -1339,29 +1394,24 @@ powerpc-linux-gcc -g -O2 -o calc calc.o <sect2 id=debugfile xreflabel="Debug Log File"> <title>Debug Log File</title> - <para>With the <option>--debug</option> option, you can request - a log file showing the output from - <productname>Expect</productname> itself, running in debugging - mode. This file (<filename>dbg.log</filename>, in the directory - where you start <command>runtest</command>) shows each pattern - <productname>Expect</productname> considers in analyzing test - output.</para> + <para>The <option>--debug</option> option will generate a debug + log file showing the internal output from + <productname>Expect</productname> running in debugging + mode. This file, called <filename>dbg.log</filename>, is created + in the directory where <command>runtest</command>) is invoked + and shows each pattern <productname>Expect</productname> + considers in analyzing program output.</para> <para>This file reflects each <command>send</command> command, - showing the string sent as input to the tool under test; and + showing the string sent as input to the program under test; and each <productname>Expect</productname> command, showing each - pattern it compares with the tool output.</para> - - <example> - <title>The log messages begin with a message of the form</title> + pattern it compares with the program output. The log messages + begin with a message of the form:</para> <screen> - expect: does {<symbol>tool output</symbol>} (spawn_id <symbol>n</symbol>) match pattern {<emphasis>expected pattern</emphasis>}? - </screen> - </example> <para>For every unsuccessful match, <productname>Expect</productname> issues a @@ -1376,8 +1426,7 @@ powerpc-linux-gcc -g -O2 -o calc calc.o variables set to describe a successful match.</para> <example> - <title>Here is an excerpt from the debugging log for a - <productname>GDB</productname> test:</title> + <title>Debug log for a <productname>GDB</productname> test:</title> <screen> send: sent {break gdbme.c:34\n} to spawn id 6 |