aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorBen Elliston <bje@gnu.org>2004-02-02 09:50:18 +0000
committerBen Elliston <bje@gnu.org>2004-02-02 09:50:18 +0000
commit57ed4889db7f94b48f77a7ef42cf1bab51c0f1f1 (patch)
tree7855251e66ba9d72e477c069510fa4adbf23ed13
parentcee969ec3030db0a4cc6157499ada27bada70ac2 (diff)
downloaddejagnu-57ed4889db7f94b48f77a7ef42cf1bab51c0f1f1.zip
dejagnu-57ed4889db7f94b48f77a7ef42cf1bab51c0f1f1.tar.gz
dejagnu-57ed4889db7f94b48f77a7ef42cf1bab51c0f1f1.tar.bz2
* doc/overview.sgml (Overview): Editorial changes.
-rw-r--r--ChangeLog4
-rw-r--r--doc/overview.sgml284
2 files changed, 143 insertions, 145 deletions
diff --git a/ChangeLog b/ChangeLog
index 6aaeb2a..c5f7569 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,7 @@
+2004-02-02 Ben Elliston <bje@wasabisystems.com>
+
+ * doc/overview.sgml (Overview): Editorial changes.
+
2004-01-30 Ben Elliston <bje@wasabisystems.com>
Import orphaned patches from sources.redhat.com:
diff --git a/doc/overview.sgml b/doc/overview.sgml
index 928f4d0..b45f371 100644
--- a/doc/overview.sgml
+++ b/doc/overview.sgml
@@ -118,20 +118,20 @@ into another language, under the above conditions for modified versions.
<preface id=preface>
<title>Abstract</title>
- <para>This document describes the functionality of DejaGnu, the
- testing framework of the GNU project. DejaGnu is written in
+ <para>This document describes the functionality of &dj;, the
+ testing framework of the GNU project. &dj; is written in
<productname>Expect</productname>, which uses
<productname>Tcl</productname> as a command
language. <productname>Expect</productname> acts as a very
programmable shell. As with other Unix command shells, you can
run any program, but once the program is started, your test script
has programmable control over its input and output. This does not
- just apply to the programs under test; <command>expect</command>
- can also run any auxiliary program, such as
- <command>diff</command> or <command>sh</command>, with full
- control over its input and output.</para>
+ just apply to the programs under test;
+ <productname>Expect</productname> can also run any auxiliary
+ program, such as <command>diff</command> or <command>sh</command>,
+ with full control over its input and output.</para>
- <para>DejaGnu itself is merely a framework for the creation of
+ <para>&dj; itself is merely a framework for the creation of
testsuites. Testsuites are distributed with each
application.</para>
@@ -139,145 +139,138 @@ into another language, under the above conditions for modified versions.
<chapter id=overview xreflabel=Overview>
<title>Overview</title>
-
<sect1 id=whatis xreflabel="What is &dj; ?">
<title>What is &dj; ?</title>
- <para><productname>DejaGnu</productname> is a framework for
- testing other programs. Its purpose is to provide a single
- front end for all tests. Think of it as a custom library of
- Tcl procedures crafted to support writing a test harness. A
- <emphasis>Test Harness</emphasis> is the testing
- infrastructure that is created to support a specific program
- or tool. Each program can have multiple testsuites, all
- supported by a single test harness. DejaGnu is written in
- <productname>Expect</productname>, which in turn uses
- <productname>Tcl</productname> -- Tool command
- language. There is more information on Tcl at the <ulink
- URL="http://www.scriptics.com">Scriptics</ulink> web site and the
- Expect web site is at <ulink
- URL="http://expect.nist.gov">NIST</ulink>.</para>
+ <para><productname>&dj;</productname> is a framework for
+ testing other programs. Its purpose is to provide a uniform
+ testing environment. Think of it as a library of
+ <productname>Tcl</productname> procedures to support writing a
+ test harness. A <emphasis>test harness</emphasis> is the
+ testing infrastructure created to support a specific
+ program. Each program can have multiple testsuites, all
+ supported by a single test harness. &dj; is written in
+ <productname>Expect</productname>, which in turn relies upon
+ <productname>Tcl</productname>. More information on
+ <productname>Tcl</productname> and
+ <productname>Expect</productname> can be found at the <ulink
+ URL="http://www.tcltk.com">Tcl/Tk web site</ulink> and the
+ <ulink URL="http://expect.nist.gov">Expect home page</ulink>,
+ respectively.</para>
- <para>Julia Menapace first coined the term ``DejaGnu'' to describe
- an earlier testing framework at Cygnus Support she had written
- for <command>GDB</command>. When we replaced it with the
- Expect-based framework, it was like DejaGnu all over again.
+ <para>Julia Menapace first coined the term ``&dj;'' to describe
+ an earlier testing framework she had written at Cygnus Support
+ for <command>GDB</command>. When it was replaced it with the
+ Expect-based framework, it was like &dj; all over again.
More importantly, it was also named after my daughter, <ulink
URL="mailto:deja@welcomehome.org">Deja Snow Savoye</ulink>
(now 13 years old as of September 2003), who was a toddler
- during DejaGnu's beginnings.</para>
+ during &dj;'s beginnings.</para>
- <para>DejaGnu offers several advantages for testing:</para>
+ <para>There are several advantages to testing with &dj;:</para>
<itemizedlist mark="bullet" spacing="compact">
- <listitem><para>The flexibility and consistency of the DejaGnu
- framework make it easy to write tests for any program, with
- either batch oriented, or interactive programs.</para>
+ <listitem><para>The flexibility and consistency of the &dj;
+ framework make it easy to write tests for any program, including
+ batch-oriented and interactive programs.</para>
</listitem>
- <listitem><para>DejaGnu provides a layer of abstraction which
+ <listitem><para>&dj; provides a layer of abstraction which
allows you to write tests that are portable to any host or
target where a program must be tested. For instance, a test
- for <command>GDB</command> can run from any supported host
- system on any supported target system. DejaGnu runs tests on
- many single board computers, whose operating software ranges
- from a simple boot monitor to a real-time OS.</para>
+ for <productname>GDB</productname> can run from any supported
+ host system on any supported target system. &dj; runs tests
+ on many single board computers, whose operating software
+ ranges from a simple boot monitor to a real-time OS.</para>
</listitem>
<listitem><para>All tests have the same output format. This
makes it easy to integrate testing into other software
- development processes. DejaGnu's output is designed to be
- parsed by other filtering script and it is also human
- readable.</para>
+ development processes. &dj;'s output is human readable, but
+ can be easily parsed by other software.</para>
</listitem>
- <listitem><para>Using Tcl and Expect, it's easy to create wrappers
+ <listitem><para>Using <productname>Tcl</productname> and
+ <productname>Expect</productname>, it is easy to create wrappers
for existing testsuites. By incorporating existing tests under
- DejaGnu, it's easier to have a single set of report analyse
- programs..</para>
+ &dj;, it is easier to have a single set of reporting
+ programs.</para>
</listitem>
</itemizedlist>
<para>Running tests requires two things: the testing framework and
the testsuites themselves. Tests are usually written in
- <productname>Expect</productname> using Tcl, but you can also use
- a Tcl script to run a testsuite that is not based on
+ <productname>Expect</productname> using
+ <productname>Tcl</productname>, but you can also use a
+ <productname>Tcl</productname> script to run a testsuite that is
+ not based on
<productname>Expect</productname>. <productname>Expect</productname>
- script filenames conventionally use <emphasis>.exp</emphasis> as a
- suffix; for example, the main implementation of the DejaGnu test
- driver is in the file
- <productname>runtest.exp</productname>.)</para>
+ scriptsuse <emphasis>.exp</emphasis> as a filename extension as a
+ convention.</para>
</sect1>
-
<sect1 id=new xreflabel="Release Notes">
- <title>What's New In This Release</title>
-
- <para>This release has a number of substantial changes over version
- 1.3. The most visible change is that the version of Expect and Tcl
- included in the release are up-to-date with the current stable net
- releases. The biggest change is years of modifications to the
- target configuration system, used for cross testing. While this
- greatly improved cross testing, is has made that subsystem very
- complicated. The goal is to have this entirely rewritten using
- <productname>iTcl</productname> by the next release. Other changes
- are:</para>
+ <title>What's new in version &version;</title>
+
+ <para>Version &version; has a number of substantial changes over
+ version 1.3. The most visible change is that the version of
+ <productname>Expect</productname> and
+ <productname>Tcl</productname> included in the release are
+ up-to-date with the current stable net releases. The biggest
+ change is to the target configuration system. While this greatly
+ improved cross-testing, is has made that subsystem very
+ complicated. Other changes include:</para>
<itemizedlist>
- <listitem><para>More built-in support for building target binaries
- with the correct linker flags. Currently this only works with
- <productname>GCC</productname> as the cross compiler,
- preferably with a target supported by
- <xref linkend=libgloss>.</para></listitem>
-
- <listitem><para>Lots of little bug fixes from years of heavy
- use at Cygnus Solutions.</para></listitem>
-
- <listitem><para>DejaGnu now uses
- <productname>Automake</productname> for Makefile
- configuration.</para></listitem>
-
- <listitem><para>Updated documentation, now in SGML
- (using the <ulink
- URL="http://nis-www.lanl.gov/~rosalia/mydocs/docbook-intro.html">free
- GNU DocBook tools</ulink>) format.</para></listitem>
+ <listitem><para>More built-in support for building target
+ binaries with the correct linker flags. Currently this only
+ works with <productname>GCC</productname> as the
+ cross-compiler.</para></listitem>
+
+ <listitem><para>Lots of bug fixes from years of heavy use at
+ Cygnus Solutions.</para></listitem>
+
+ <listitem><para>The use of <productname>Autoconf</productname>
+ and <productname>Automake</productname> for
+ building.</para></listitem>
+
+ <listitem><para>Updated documentation in SGML format using the
+ <ulink
+ URL="http://nis-www.lanl.gov/~rosalia/mydocs/docbook-intro.html">GNU
+ DocBook tools</ulink>.</para></listitem>
- <listitem><para>Windows support. There is beta level support for
- Windows that is still a work in progress. This requires the
- <ulink URL="http://www.cygwin.com/">Cygwin</ulink> POSIX
- subsystem for Windows.</para></listitem>
+ <listitem><para>Beta level support for Windows, however this is
+ still work in progress. This requires the <ulink
+ URL="http://www.cygwin.com/">Cygwin</ulink> POSIX subsystem for
+ Windows.</para></listitem>
</itemizedlist>
<sect2 id=cygwin xreflabel="Windows Support">
<title>Windows Support</title>
- <para>To use DejaGnu on Windows, you need to first install the
+ <para>To use &dj; on Windows, you need to first install the
<ulink URL="http://www.cygwin.com/">Cygwin</ulink>
- release. This works as of the B20.1 release. Cygwin is a POSIX
- system for Windows. This covers both utility programs and a library
- that adds POSIX system calls to Windows. Among them is pseudo tty
- support for Windows that emulates the POSIX pty standard. The
- latest Cygwin is always available from <ulink
- URL="http://www.cygwin.com/">this location</ulink>. This
- works well enough to run <emphasis>"make check"</emphasis> of
- the GNU development tree on Windows after a native build. But the
- nature of ptys on Windows is still evolving. Your mileage may
- vary.</para>
-
+ system. Cygwin is a POSIX system for Windows. This covers both
+ utility programs and a library that adds POSIX system calls to
+ Windows. Among them is pseudo tty support for Windows that
+ emulates the POSIX pty standard. The latest Cygwin is always
+ available from <ulink URL="http://www.cygwin.com/">the Cygwin
+ home page</ulink>. This works well enough to run
+ <emphasis>"make check"</emphasis> on the GNU development tools
+ after building them on a native Cygwin system.</para>
</sect2>
-
</sect1>
<sect1 id=designgoals xreflabel="Design Goals">
<title>Design Goals</title>
- <para>DejaGnu grew out of the internal needs of Cygnus Solutions,
+ <para>&dj; grew out of the internal needs of Cygnus Solutions,
the company formerly known as Cygnus Support. Cygnus maintained
and enhanced a variety of free programs in many different
- environments and we needed a testing tool that:</para>
+ environments. A testing tool was needed that:</para>
<itemizedlist mark="bullet">
<listitem><para>was useful to developers while fixing
@@ -295,40 +288,42 @@ into another language, under the above conditions for modified versions.
</itemizedlist>
<para>Some of the requirements proved challenging. For example,
- interactive programs do not lend themselves very well to automated testing.
- But all the requirements are important: for instance, it is imperative to
- make sure that <command>GDB</command> works as well when cross-debugging
- as it does in a native configuration. </para>
+ interactive programs do not lend themselves very well to automated
+ testing, but this capability is important. For instance, it is
+ imperative to make sure that <productname>GDB</productname> works
+ as well when cross-debugging as it does in a native
+ configuration.</para>
<para>Probably the greatest challenge was testing in a
cross-development environment. Most cross-development
environments are customized by each developer. Even when buying
packaged boards from vendors there are many differences. The
communication interfaces vary from a serial line to Ethernet.
- DejaGnu was designed with a modular communication setup, so that
- each kind of communication can be added as required and supported
- thereafter. Once a communication procedure is coded, any test can
- use it. Currently DejaGnu can use <command>rsh</command>,
- <command>rlogin</command>, <command>telnet</command>,
- <command>tip</command>, <command>kermit</command> and
- <command>mondfe</command> for remote communications.</para>
+ &dj; was designed with a modular communication setup, so that
+ each mode of communication can be added as required and supported
+ thereafter. Once a communication procedure is implemented, any
+ testsuite can use it. Currently &dj; can use
+ <command>rsh</command>, <command>rlogin</command>,
+ <command>telnet</command>, <command>tip</command>,
+ <command>kermit</command> and <command>mondfe</command> for remote
+ communications.</para>
</sect1>
<sect1 id=posix xreflabel="A POSIX Conforming Test Framework">
<title>A POSIX conforming test framework</title>
- <para>DejaGnu conforms to the POSIX 1003.3 standard for test
- frameworks. Rob Savoye was a member of that committee.</para>
-
- <para>The POSIX standard 1003.3 defines what a testing framework needs to
- provide, in order to permit the creation of POSIX conformance test
- suites. This standard is primarily oriented to running POSIX conformance
- tests, but its requirements also support testing of features not related
- to POSIX conformance. POSIX 1003.3 does not specify a particular testing
- framework, but at this time there is only one other POSIX conforming test
- framework: TET. TET was created by Unisoft for a consortium comprised of
- X/Open, Unix International and the Open Software Foundation.</para>
+ <para>&dj; conforms to the POSIX 1003.3 standard for test
+ frameworks. The POSIX standard 1003.3 defines what a testing
+ framework needs to provide, in order to permit the creation of
+ POSIX conformance test suites. This standard is primarily
+ oriented to running POSIX conformance tests, but its
+ requirements also support testing of features not related to
+ POSIX conformance. POSIX 1003.3 does not specify a particular
+ testing framework, but at the time of writing there is only one
+ other POSIX conforming test framework: TET. TET was created by
+ Unisoft for a consortium comprised of X/Open, Unix International
+ and the Open Software Foundation.</para>
<para>The POSIX documentation refers to <firstterm>assertions</firstterm>.
An assertion is a description of behavior. For example, if a standard
@@ -343,7 +338,7 @@ into another language, under the above conditions for modified versions.
repeatedly reading the standard and experimenting. One of the main
things 1003.3 does specify is the set of allowed output messages and
their definitions. Four messages are supported for a required feature of
- POSIX conforming systems and a fifth for a conditional feature. DejaGnu
+ POSIX conforming systems and a fifth for a conditional feature. &dj;
supports the use of all five output messages. In this sense a testsuite
that uses exactly these messages can be considered POSIX conforming.
These definitions specify the output of a test
@@ -378,23 +373,26 @@ into another language, under the above conditions for modified versions.
</varlistentry>
<varlistentry>
- <term>UNRESOLVED</term>
- <listitem><para>A test produced indeterminate results. Usually, this
- means the test executed in an unexpected fashion; this outcome
- requires that a human being go over results, to determine if the test
- should have passed or failed. This message is also used for any test
- that requires human intervention because it is beyond the abilities
- of the testing framework. Any unresolved test should resolved to
- <emphasis>PASS</emphasis> or <emphasis>FAIL</emphasis> before a test
- run can be considered finished.</para>
-
- <para>Note that for POSIX, each assertion must produce a test result
- code. If the test isn't actually run, it must produce
- <emphasis>UNRESOLVED</emphasis> rather than just leaving that test
- out of the output. This means that you have to be careful when
- writing tests to not carelessly use Tcl commands like
- <emphasis>return</emphasis>---if you alter the flow of control of the
- Tcl code you must insure that every test still produces some result
+ <term>UNRESOLVED</term> <listitem><para>A test produced
+ indeterminate results. Usually, this means the test
+ executed in an unexpected fashion; this outcome requires a
+ human being to go over the results and determine if the test
+ should have passed or failed. This message is also used for
+ any test that requires human intervention because it is
+ beyond the abilities of the testing framework. Any
+ unresolved test should be resolved to
+ <emphasis>PASS</emphasis> or <emphasis>FAIL</emphasis>
+ before a test run can be considered finished.</para>
+
+ <para>Note that for POSIX compliance, each assertion must
+ produce a test result code. If the test isn't actually run,
+ it must produce <emphasis>UNRESOLVED</emphasis> rather than
+ just leaving that test out of the output. This means that
+ you have to be careful when writing tests to not carelessly
+ use <productname>Tcl</productname> commands like
+ <emphasis>return</emphasis>---if you alter the flow of
+ control of the <productname>Tcl</productname> code you must
+ insure that every test still produces some result
code.</para>
<para>Here are some of the ways a test may wind up
@@ -404,12 +402,12 @@ into another language, under the above conditions for modified versions.
</variablelist>
<itemizedlist mark=bullet>
- <listitem><para>A test's execution is
+ <listitem><para>The execution of a test is
interrupted.</para></listitem>
<listitem><para>A test does not produce a clear
result. This is usually because there was an
- <emphasis>ERROR</emphasis> from DejaGnu while processing
+ <emphasis>ERROR</emphasis> from &dj; while processing
the test, or because there were three or more
<emphasis>WARNING</emphasis> messages. Any
<emphasis>WARNING</emphasis> or <emphasis>ERROR</emphasis>
@@ -433,16 +431,12 @@ into another language, under the above conditions for modified versions.
</varlistentry>
</variablelist>
- <para>The only remaining output message left is intended to test
- features that are specified by the applicable POSIX standard as
- conditional:</para>
-
<variablelist>
<varlistentry>
<term>UNSUPPORTED</term>
<listitem><para>There is no support for the tested case. This may
mean that a conditional feature of an operating system, or of a
- compiler, is not implemented. DejaGnu also uses this message when
+ compiler, is not implemented. &dj; also uses this message when
a testing environment (often a ``bare board'' target) lacks basic
support for compiling or running the test case. For example, a
test for the system subroutine <emphasis>gethostname</emphasis>
@@ -451,9 +445,9 @@ into another language, under the above conditions for modified versions.
</varlistentry>
</variablelist>
- <para>DejaGnu uses the same output procedures to produce these messages
+ <para>&dj; uses the same output procedures to produce these messages
for all testsuites and these procedures are already known to conform
- to POSIX 1003.3. For a DejaGnu testsuite to conform to POSIX 1003.3,
+ to POSIX 1003.3. For a &dj; testsuite to conform to POSIX 1003.3,
you must avoid the <emphasis>setup</emphasis>xfail} procedure as
described in the <emphasis>PASS</emphasis> section above and you must
be careful to return <emphasis>UNRESOLVED</emphasis> where appropriate,