Add warnings about synchronization with the tested program

1 files changed, 80 insertions, 41 deletions

diff --git a/doc/dejagnu.texi b/doc/dejagnu.texi
index 8f23250..cd8a179 100644
--- a/doc/dejagnu.texi
+++ b/doc/dejagnu.texi
@@ -2175,53 +2175,91 @@ built up a family of Tcl procedures specialized for GDB testing.
 @cindex hints on writing a test case
 @cindex test cases, writing
 
-It is safest to write patterns that match all the output generated by
-the tested program; this is called closure.  If a pattern does not
-match the entire output, any output that remains will be examined by
-the next @code{expect} command.  In this situation, the precise
-boundary that determines which @code{expect} command sees what is very
-sensitive to timing between the Expect task and the task running the
-tested tool.  As a result, the test may sometimes appear to work, but
-is likely to have unpredictable results.  (This problem is
-particularly likely for interactive tools, but can also affect batch
-tools---especially for tests that take a long time to finish.) The
-best way to ensure closure is to use the @code{-re} option for the
-@code{expect} command to write the pattern as a full regular
-expressions; then you can match the end of output using a @emph{$}.
-It is also a good idea to write patterns that match all available
-output by using @emph{.*\} after the text of interest; this will also
-match any intervening blank lines.  Sometimes an alternative is to
-match end of line using @emph{\r} or @emph{\n}, but this is usually
-too dependent on terminal settings.
-
-Always escape punctuation, such as @emph{(} or @emph{"}, in your
-patterns; for example, write @emph{\(}.  If you forget to escape
-punctuation, you will usually see an error message like:
-
-@example
-extra characters after close-quote
-@end example
+To preserve basic sanity, no should test ever pass if there was any
+kind of problem in the test case.  To take an extreme case, tests that
+pass even when the tool will not spawn are misleading.  Ideally, a
+test in this sort of situation should not fail either.  Instead, print
+an error message by calling one of the DejaGnu procedures
+@code{perror} or @code{warning}.  Note that using @code{perror} will
+cause the next text result to be reported as @samp{UNRESOLVED}, so
+printing an error and allowing the test to fail is a good option.
 
 If you have trouble understanding why a pattern does not match the
 program output, try using the @code{--debug} option to @code{runtest},
 and examine the debug log carefully.
 
+If you use glob patterns, you will need to escape any @samp{*},
+@samp{?}, @samp{[}, @samp{]}, and @samp{\} characters that are meant
+to match literally.  If you use regular expressions, see the
+@cite{re_syntax(n)} manual page from Tcl for the syntax details, and
+be sure to understand what punctuation characters match literally and
+what characters have special meanings in regular expressions.
+
+Tcl has a few options for quoting; the most notable are @samp{@{@}}
+and @samp{""}.  These quotes behave differently: @samp{@{@}} must
+balance, while @samp{""} performs various interpolations.  In
+@samp{@{@}} quotes, unbalanced @samp{@{} or @samp{@}} characters must
+be escaped with @samp{\} and these escapes are @emph{not} removed;
+fortunately, backslash-escaped braces match literal braces in Tcl
+regular expressions.  In @samp{""} quotes, any embedded @samp{"}
+characters must be escaped, a literal @samp{$} begins a variable
+substitution, and unescaped @samp{[]} introduce a Tcl command
+substitution.
+
+@subsubheading Synchronization with the tested program
+
+A DejaGnu testsuite executes concurrently with the programs that it
+tests.  As a result, DejaGnu may see parts of the tested program's
+output while the tested program is still producing more output.
+Expect patterns must be written to handle the possibility that
+incomplete output from the tested program will be considered for
+matching.
+
+Expect reads the output from the tested program into an internal
+matching buffer and removes everything from the start of the buffer to
+the end of the match when a match is found.  Any given character can
+be matched at most once, or skipped if a match is found starting later
+in the buffer or the buffer reaches its capacity.  Anything left in
+the buffer after the end of the match remains in the buffer and is
+considered for the next @code{expect} command.  If @code{expect} is
+invoked and no patterns match, Expect waits for more text to arrive.
+New text is appended to the buffer as it is read.  If the buffer
+reaches its capacity, the entire contents of the buffer are discarded
+and Expect resumes reading.
+
+In Expect patterns, the regular expression anchors @samp{^} and
+@samp{$} match at the beginning and end of the @emph{buffer}, not at
+line boundaries.  The @samp{$} anchor must be used with care because
+it will match at the end of what Expect @emph{has} read, but the
+program may have produced more output that Expect @emph{has not yet}
+read.  Similarly, regular expressions ending with the @samp{*}
+quantifier can potentially match a prefix of the intended text, only
+for the rest to arrive shortly thereafter.
+
+Maintaining synchronization with the tested program is easier if the
+patterns match all of the output generated by the tested program; this
+is called closure.
+
+For interactive programs, a prompt is usually a good synchronization
+point, provided that the program's prompt can be uniquely recognized.
+Since the prompt is usually the last output until the program receives
+further input, the @samp{$} anchor can be useful here.
+
+If the output from the tested program is organized into lines,
+matching end-of-line using @samp{\n} is usually a good way to process
+one line at a time.  Note that terminal settings may result in the
+insertion of additional @samp{\r} characters, usually translating
+@samp{\n} to @samp{\r\n}.
+
 Be careful not to neglect output generated by setup rather than by the
-interesting parts of a test case.  For example, while testing GDB, I
-issue a send @emph{set height 0\n} command.  The purpose is simply to
-make sure GDB never calls a paging program.  The @emph{set height}
+interesting parts of a test case.  For example, while testing GDB, a
+@samp{set height 0\n} command is issued.  The purpose is simply to
+make sure GDB never calls a paging program.  The @samp{set height}
 command in GDB does not generate any output; but running any command
-makes GDB issue a new @emph{(gdb) } prompt.  If there were no
-@code{expect} command to match this prompt, the output @emph{(gdb) }
-begins the text seen by the next @code{expect} command---which might
-make that pattern fail to match.
-
-To preserve basic sanity, I also recommended that no test ever pass if
-there was any kind of problem in the test case.  To take an extreme
-case, tests that pass even when the tool will not spawn are
-misleading.  Ideally, a test in this sort of situation should not fail
-either.  Instead, print an error message by calling one of the DejaGnu
-procedures @code{error} or @code{warning}.
+makes GDB issue a new @samp{(gdb) } prompt.  If there were no
+@code{expect} command to match this prompt, the @samp{(gdb) } prompt
+will remain in the buffer and begin the text seen by the next
+@code{expect} command---which might make that pattern fail to match.
 
 @subsubheading Priority of Expect patterns
 
@@ -5968,4 +6006,5 @@ This makes @code{runtest} exit.  Abbreviation: @kbd{q}.
 @c  LocalWords:  subdirectory prepend prepended testsuite filename Expect's svn
 @c  LocalWords:  DejaGnu CVS RCS SCCS prepending subcommands Tcl Awk Readline
 @c  LocalWords:  POSIX KFAIL KPASS XFAIL XPASS hostname multitable gfortran api
-@c  LocalWords:  boolean subcommand testcase eval rustc executables
+@c  LocalWords:  boolean subcommand testcase eval rustc executables perror gdb
+@c  LocalWords:  interprocess lookahead