aboutsummaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2022-12-20Add support for errors and warnings in the unit test protocolJacob Bachmeyer1-0/+2
2022-11-28Add UNSUPPORTED test result to DejaGnu unit test protocolJacob Bachmeyer1-0/+1
2022-11-26Add explicit end-of-test marker to DejaGnu unit test protocolJacob Bachmeyer1-7/+6
2022-10-04Buffer test results in case the pruning callback aborts the testJacob Bachmeyer1-8/+23
2021-06-16Merge fixes from 1.6.3 release branchJacob Bachmeyer1-1/+1
Conflicts: ChangeLog configure configure.ac doc/version.texi runtest.exp
2021-05-20Fix spurious UNRESOLVED result when size(1) is not in PATHJacob Bachmeyer1-1/+1
2021-04-14Add initial infrastructure for DejaGnu "specs" stringsJacob Bachmeyer1-0/+294
2020-12-27Remove noise word "api" from "testsuite can call api" entrypointJacob Bachmeyer1-3/+3
2020-08-02Merge branch 'PR42399'Rob Savoye1-49/+39
Branch surived testing.
2020-07-22Revise host_execute to fix PR42399Jacob Bachmeyer1-42/+29
2020-07-20Add regression test for PR42399Jacob Bachmeyer1-7/+10
2020-07-13target: Wrap linker flags into `-largs'/`-margs' for AdaMaciej W. Rozycki1-14/+36
Unrecognized `gnatmake' switches are not implicitly passed on to the linker, so just pasting board `ldflags' and any other linker flags verbatim into `add_flags' to use for the invocation line of `gnatmake' will make them ignored at best. For example in a GCC test environment that has: set_board_info ldflags "-Wl,-dynamic-linker,.../sysroot/lib/ld-linux-riscv64-lp64d.so.1 -Wl,-rpath,.../sysroot/lib64/lp64d -Wl,-rpath,.../sysroot/usr/lib64/lp64d" so that sysroot paths are correctly embedded with the binaries linked for use with the dynamic loader and shared library dependencies, the setting will be ignored for the GNAT test suite making all the execution tests fail, e.g.: PASS: gnat.dg/abstract_with_anonymous_result.adb (test for excess errors) spawn qemu-riscv64 ./abstract_with_anonymous_result.exe /lib/ld-linux-riscv64-lp64d.so.1: No such file or directory FAIL: gnat.dg/abstract_with_anonymous_result.adb execution test For `gnatmake' to pass switches on to the linker the `-largs' switch has to be used, which affects all the switches that follow until a switch is seen that changes the selection, like `-margs', which resets to the initial state of the switch interpretation machine. Wrap linker flags into `-largs'/`-margs' for Ada then, carefully preserving the place these flags are placed at within `add_flags', as surely someone will have depended on that, correcting test failures like above: PASS: gnat.dg/abstract_with_anonymous_result.adb (test for excess errors) spawn qemu-riscv64 ./abstract_with_anonymous_result.exe PASS: gnat.dg/abstract_with_anonymous_result.adb execution test Pass multilib flags both to the compiler and to the linker as both build stages interpret them. Update the testsuite accordingly. * lib/target.exp (default_target_compile): Wrap linker flags into `-largs'/`-margs' for Ada. * testsuite/runtest.libs/target.test (compile_test): Update accordingly. Signed-off-by: Maciej W. Rozycki <macro@wdc.com>
2020-07-06Merge branch 'PR41918'Jacob Bachmeyer1-12/+48
Conflicts: ChangeLog
2020-07-06Merge branch 'timeout-fix-for-1.6.3'Jacob Bachmeyer1-2/+21
Conflicts: ChangeLog
2020-07-06Merge branch 'new-api-for-1.6.3'Jacob Bachmeyer1-1/+130
Conflicts: ChangeLog NEWS doc/dejagnu.texi
2020-07-06Merge branch 'gdb-upstream-for-1.6.3'Jacob Bachmeyer2-6/+138
Conflicts: ChangeLog
2020-06-29Add separator lines when repeating error messagesJacob Bachmeyer1-1/+3
2020-06-26Record Tcl errors and dump them again at the end of a runJacob Bachmeyer1-0/+9
2020-06-22remote: Fix a stuck remote call pipeline causing testing to hangMaciej W. Rozycki1-1/+17
Fix a stuck remote call pipeline comprised of multiple processes causing testing to hang and requiring a manual intervention to either terminate or proceed, like below (here with the GCC `c' testsuite invoked with `execute.exp=postmod-1.c' for 8 compilation and 8 execution tests on a remote QEMU target run in the system emulation mode): PASS: gcc.c-torture/execute/postmod-1.c -O0 (test for excess errors) Executing on remote-localhost: .../gcc/testsuite/gcc/postmod-1.exe (timeout = 15) spawn [open ...] WARNING: program timed out got a INT signal, interrupted by user === gcc Summary === # of expected passes 1 by not killing the pending force-kills in `close_wait_program' and also by setting the channel associated with the pipeline to the nonblocking mode when it is about to be closed afterwards. The situation here is as follows. A connection to the remote target board is requested by `rsh_exec' with input redirection requested from `/dev/null'. The request is handled by `local_exec' and the redirection causes a Tcl command pipeline channel to be opened. The list of PIDs of the processes comprising the pipeline is determined and then the channel is assigned an Expect spawn ID. The spawn ID is then waited for output produced by the remote target (here accessed with SSH) and, ultimately, completion marked by the end-of-file condition. As SSH gets stuck and does not complete the timeout eventually fires and a kill sequence is initiated, by calling `close_wait_program' with the list of PIDs previously obtained to kill given as one of the procedure's arguments. Seeing the list of PIDs rather than -1 `close_wait_program' issues SIGINT to all the requested processes right away and schedules a delayed sequence called "force-kills" to them, which sends SIGTERM and then, after a further delay, SIGKILL. Now `close_wait_program' calls `close' on the spawn ID associated with the pipeline, but this call doesn't affect the pipeline as its input has been redirected from `/dev/null'. As the next step `wait' is called on the same spawn ID and returns successfully right away with a result like {0 exp8 0 0} in `wres', where no PID is indicated, consistently with the null PID result of the original `spawn' command that assigned the spawn ID (`exp8' here) to the pipeline. The return from the `wait' command causes code to be executed for the pending force-kills to be killed. At this point the process situation is like below: PID TTY STAT TIME COMMAND 6908 pts/3 Sl 0:00 expect -- .../share/dejagnu/runtest.exp --tool gcc --target_board remote-localhost execute.exp=postmod-1.c 6976 pts/3 S 0:00 \_ ssh -p 2222 -l macro localhost sh -c '.../gcc/testsuite/gcc/postmod-1.exe ; echo XYZ${?}ZYX' 6977 pts/3 Z 0:00 \_ [cat] <defunct> 6991 pts/3 Z 0:00 \_ [sh] <defunct> so `cat' and `sh' have already terminated, the former presumably due to SIGINT sent previously and the latter having been the force-kills just killed, and only await being wait(2)ed for, however `ssh' is still live and in the interruptible sleep, presumably awaiting communication with the remote end. Since there is nothing else to do for `close_wait_program' it returns success to `local_exec', which then calls `close' on the pipeline to clean up after it. But that in turn causes wait(2) to be called on the individual PIDs comprising the pipeline and when the PID associated with `ssh' the call hangs indefinitely preventing the whole testsuite from proceeding. A similar situation triggers with GDB testing where a Tcl command pipeline channel is opened in `remote_spawn' instead, and then closed, after `close_wait_program' has been called, in `standard_close'. So the solution to the problem is twofold. First pending force-kills are not killed after `wait' if there are more than one PID in the list passed to `close_wait_program'. This follows the observation that if there was only one PID on the list, then the process must have been created directly by `spawn' rather than by assigning a spawn ID to a pipeline and the return from `wait' would mean the process associated with the PID must have already been cleaned up after, so it is only when there are more there is a possibility any may have been left behind live. Second if a pipeline has been used, then the channel associated with the pipeline is set to the nonblocking mode in case any of the processes that may have left live is stuck in the noninterruptible sleep (aka D) state. Such a process would necessarily ignore even SIGKILL so long as it remains in that state and would cause wait(2) called by `close' to hang possibly indefinitely, and we want the testsuite to proceed rather than hang even in bad circumstances. Finally it appears to be safe to leave pending force-kills to complete their job after `wait' has been called in `close_wait_program', because based on the observation made here the command does not actually call wait(2) if issued on a spawn ID associated with a pipeline created by `open' rather than a process created by `spawn'. Instead the PIDs from a pipeline are supposed to be cleaned up after by calling wait(2) from the `close' command call made on the pipeline channel. If on the other hand the channel is set to the nonblocking mode before `close', then even that command does not call wait(2) on the associated PIDs. Therefore the PIDs on the list passed are not subject to PID reuse and the force-kills won't accidentally kill an unrelated process, as a PID cannot be allocated by the kernel for a new process until any previous process's status has been consumed from its PID by wait(2). And then PIDs of any children that have actually terminated one way or another are wait(2)ed for by Tcl automatically in the event loop, so no mess is left behind. * lib/remote.exp (close_wait_program): Only kill the pending force-kills if the PID list has a single entry. (local_exec): Set the channel about to be closed to the nonblocking mode if we didn't see an EOF. (standard_close): Likewise, unconditionally. Signed-off-by: Maciej W. Rozycki <macro@wdc.com>
2020-06-22remote: Use `catch' in killing pending force-killsMaciej W. Rozycki1-1/+4
Address an execution race in `close_wait_program' and use `catch' in killing pending force-kills issued there in the recovery of a stuck test case, in case the force-kill sequence has completed before the command to kill the sequence had a chance to run, so that no error is thrown and a testsuite run does not get interrupted early like: PASS: gcc.c-torture/execute/postmod-1.c -O0 (test for excess errors) Executing on remote-localhost: .../gcc/testsuite/gcc/postmod-1.exe (timeout = 15) spawn [open ...] WARNING: program timed out ERROR: tcl error sourcing .../gcc/testsuite/gcc.c-torture/execute/execute.exp. ERROR: child process exited abnormally while executing "exec sh -c "exec > /dev/null 2>&1 && kill -9 $exec_pid"" (procedure "close_wait_program" line 57) invoked from within "close_wait_program $spawn_id $pid wres" (procedure "local_exec" line 104) [...] "uplevel #0 source .../gcc/testsuite/gcc.c-torture/execute/execute.exp" invoked from within "catch "uplevel #0 source $test_file_name"" testcase .../gcc/testsuite/gcc.c-torture/execute/execute.exp completed in 196 seconds === gcc Summary === # of expected passes 1 -- therefore not letting `execute.exp' continue (here with the GCC `c' testsuite invoked with `execute.exp=postmod-1.c' for 8 compilation and 8 execution tests). The completion of the force-kill sequence would have to happen in the window between the `wait' command has returned, which would at worst happen as a result of the final `kill -9' command in the sequence, and the `kill -9 $exec_pid' command issued here, and the `sleep 5' command issued at the end of the force-kill sequence makes the likelihood of such a scenario low, but this might still happen with a loaded host system and there is no drawback from using `catch' here, so let's do it. * lib/remote.exp (close_wait_program): Use `catch' in killing pending force-kills. Signed-off-by: Maciej W. Rozycki <macro@wdc.com>
2020-06-22Add "linker=" option to target_compile to support testingJacob Bachmeyer1-4/+8
2020-06-22Fix up upstreamed GDB testsuite patchesJacob Bachmeyer1-2/+11
2020-06-20Add Go support to default_target_compileTom Tromey2-0/+55
This adds Go support to default_target_compile. This comes from this gdb patch: commit a766d390bb857383a5f9ae80a102e1f8705f4c2e Author: Doug Evans <dje@google.com> Date: Wed Apr 25 14:07:23 2012 +0000 Initial pass at Go language support.
2020-06-20Add Rust support to default_target_compileTom Tromey2-1/+40
This adds support for the Rust language to default_target_compile. This comes from a gdb patch: commit 67218854b1987d89593ccaf5feaf5b29b1b976f2 Author: Tom Tromey <tom@tromey.com> Date: Tue Apr 26 19:38:43 2016 -0600 Update gdb test suite for Rust [...] 2016-05-17 Tom Tromey <tom@tromey.com> Manish Goregaokar <manishsmail@gmail.com>
2020-06-20Add early_flags to default_target_compileTom Tromey1-4/+29
This adds early_flags support to default_target_compile. This originated in this gdb patch: commit 6ebea266fd0a7a56c90db3ab6237ff9f6c919747 Author: Doug Evans <dje@google.com> Date: Fri Jul 24 15:24:37 2015 -0700 Workaround debian change to default value of --as-needed. gdb/testsuite/ChangeLog: * lib/future.exp (gdb_default_target_compile): New option "early_flags". * lib/gdb.exp (gdb_compile): Undo debian's change in default of --as-needed. This patch also pulls in the "linker_opts_order" code, though nothing uses it yet. A use will come in a subsequent patch.
2020-06-18Use consistent behavior for Tcl errors in test scriptsJacob Bachmeyer1-20/+26
2020-06-17Merge branch 'pr41914' into 'PR41824'Jacob Bachmeyer1-0/+1
2020-06-17Allow testing to continue after an undefined command is calledJacob Bachmeyer1-3/+21
2020-06-17Propagate return value of auto-loaded commandRob Savoye1-0/+3
2020-06-15Combine remote stderr into remote stdoutChristophe Lyon1-1/+1
* lib/ssh.exp (ssh_exec): Redirect stderr to stdout on the remote machine, to avoid race conditions.
2020-06-15Keep trailing newline in remote execution outputYvan Roux2-6/+0
* lib/rsh.exp (rsh_exec): Don't remove trailing newline. * lib/ssh.exp (rsh_exec): Likewise.
2020-06-06Add "testcase group" APIJacob Bachmeyer1-1/+109
2020-06-05Add "testsuite can call api" feature test APIJacob Bachmeyer1-0/+21
2020-05-25Use board_info correctly.Jacob Bachmeyer1-10/+10
2020-05-25Use host_info procedure to probe for a host configuration, instead of ↵Jacob Bachmeyer1-1/+1
checking a local empty target_info array due to lacking global target_info.
2020-05-25 Establish a default C compiler by evaluating [find_gcc] if no other ↵Jacob Bachmeyer1-1/+5
compiler is given.
2020-05-15Update Copyright datesRob Savoye18-18/+18
2019-01-10 * testsuite/runtest.libs/clone_output.test: Use new test harnessJacob Bachmeyer1-1/+3
for verifying output of clone_output. * testsuite/runtest.libs/default_procs.tcl (send_error): Roll into template using store_test_output. (send_log): Likewise. (send_user): Likewise. (clear_test_output): New proc. (store_test_output): New proc. * lib/framework.exp (clone_output): Actually send errors to the error stream; bug found while improving testsuite. Signed-off-by: Ben Elliston <bje@gnu.org>
2018-12-14 * lib/framework.exp (log_summary): Remove "testcnt" mechanism.Jacob Bachmeyer1-33/+0
* testsuite/lib/libsup.exp: Remove obsolete "testcnt" variable. Signed-off-by: Ben Elliston <bje@gnu.org>
2018-12-12 * config/gdb-comm.exp, config/gdb_stub.exp, config/vxworks.exp,Ben Elliston5-12/+12
lib/dg.exp, lib/ftp.exp, lib/kermit.exp, lib/rlogin.exp, lib/telnet.exp, runtest.exp, testsuite/lib/libsup.exp: Simplify some regular expressions in constant strings by placing them inside braces instead of quotes. This allows one level of backslash quoting to be removed.
2018-12-10 * lib/utils.exp (grep): Use a proper Tcl list for options.Ben Elliston1-5/+5
2018-12-10 * lib/utils.exp (getdirs): Use glob -nocomplain rather than globBen Elliston1-31/+22
and catching the "no files matched glob pattern" error message. Catching the error message was the wrong thing to do because the foreach loop then iterates over each word in the error message as if they were matches.
2018-12-10 * lib/debugger.exp (dumpvars): Remove unnecessary braces.Ben Elliston1-1/+1
2018-12-10 * runtest.exp (setup_target_hook): Use 'ne' and not '!='.Ben Elliston2-5/+5
(iterate_target_variants_two): Likewise. * lib/remote.exp (standard_download): Use 'eq' not '=='. (remote_upload): Likewise. * lib/framework.exp (open_logs): Likewise. (is_remote): Likewise.
2018-12-09Reindent lib/target.exp.Ben Elliston1-61/+61
2018-12-09 * baseboards/basic-sid.exp, baseboards/basic-sim.exp,Ben Elliston10-157/+157
baseboards/i386-sid.exp, baseboards/mt-sid.exp, baseboards/sh-sid.exp, config/adb.exp, config/gdb-comm.exp, config/gdb_stub.exp, config/sim.exp, config/unix.exp, config/vxworks.exp, lib/dejagnu.exp, lib/dg.exp, lib/framework.exp, lib/kermit.exp, lib/libgloss.exp, lib/remote.exp, lib/rsh.exp, lib/target.exp, lib/tip.exp, lib/utils.exp, testsuite/lib/libsup.exp, testsuite/runtest.all/options.exp,: Remove unnecessary braces around variable expansions.
2018-12-09Remove commented out lines in host_execute.Ben Elliston1-3/+0
2018-12-09 * lib/debugger.exp, lib/dg.exp, lib/framework.exp, lib/ftp.exp,Ben Elliston11-113/+113
lib/kermit.exp, lib/libgloss.exp, lib/remote.exp, lib/rlogin.exp, lib/target.exp, lib/tip.exp, lib/utils.exp, runtest.exp: Remove unnecessary quotes and braces around variable expansions.
2018-12-08 * lib/dejagnu.exp (host_execute): Remove unnecessary quotes aroundBen Elliston1-6/+6
variable expansions. * testsuite/runtest.all/clone_output.test: Likewise. * testsuite/runtest.all/target.test: Likewise. * testsuite/runtest.all/stats.exp: Likewise. * testsuite/runtest.all/remote.test: Likewise. * testsuite/runtest.all/config.test: Likewise. * testsuite/runtest.all/default_procs.tcl: Likewise. * testsuite/runtest.all/libs.exp: Likewise. * testsuite/runtest.all/options.exp: Likewise.
2018-12-08 * NEWS: Document 'testsuite' command.Jacob Bachmeyer1-0/+64
* doc/dejagnu.texi (testsuite procedure): Document multiplex entry point and "testsuite file" command. * lib/framework.exp (testsuite): New proc for multiplex commands. (testsuite_file): New proc implementing "testsuite file". * testsuite/runtest.all/testsuite_file.test: New file. * runtest.exp: Expect to find testsuite in ${srcdir}/testsuite, but also search $srcdir itself. (load_lib): Add explicit search for testsuite-local libraries. (load_tool_init): Use $testsuitedir in search. (load_config): Use $testsuitedir instead of $srcdir. (load_tool_target_config): Likewise. Add variable "testsuitedir" for testsuite root directory. Add internal global variables "testbuilddir" and "testdir" for use by "testsuite file". Ensure that $testsuitedir, $testbuilddir, and $objdir also avoid duplicated path delimiters. Add warning if no tests are found and fallback method of searching $srcdir is used. Signed-off-by: Ben Elliston <bje@gnu.org>