aboutsummaryrefslogtreecommitdiff
path: root/debug/testlib.py
AgeCommit message (Collapse)AuthorFilesLines
2024-06-27Merge pull request #561 from TommyMurphyTM1234/fix-list-testsAnatoly Parshintsev1-1/+11
Debug: suppress `PRNG seed ...` log messages when `gdbserver.py --list-tests <target>` used
2024-06-24Use Zvl/Zve to communicate VLEN/ELEN to target in debug tests (#567)Jerry Zhao1-2/+3
2024-06-06Fix pylint issues with previous commit changesTommy Murphy1-3/+2
2024-06-06Move PRNG seed generation/logging from gdbserver.py:main() into ↵Tommy Murphy1-0/+11
testlib.py:run_all_tests()
2024-05-14debug: workaround for sporadic failures of some tests due to unexpected data ↵Parshintsev Anatoly1-1/+6
present in pexpect match Problem was observed on UnavailableMultiTest - this test was sporadically failing. When the failure was observed the log of the failing test looked as follows: ``` File "/whatever/RISCVTests/debug/testlib.py", line 504, in <genexpr> if all(targets[hart.id]["State"] == "running" for hart in harts): ~~~~~~~~~~~~~~~~^^^^^^^^^ KeyError: 'State' ``` Adding this modification to testlib.py ``` --- a/debug/testlib.py +++ b/debug/testlib.py @@ -498,6 +498,10 @@ class Openocd: for line in lines[2:]: if line.strip(): data.append(dict(zip(headers, line.split()[1:]))) + str_data = str(data) + sys.stdout.flush() + sys.stdout.write(f"parsed targets:\n{result}\n===\n{str_data}\n---\n") + sys.stdout.flush() return data ``` Allowed me to root cause the issue. Namely we have the following situation: ``` parsed targets: Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f2dee64d1c0> Traceback (most recent call last): File "/usr/local/lib/python3.11/tempfile.py", line 450, in __del__ self.close() File "/usr/local/lib/python3.11/tempfile.py", line 446, in close unlink(self.name) FileNotFoundError: [Errno 2] No such file or directory: '/tmp/gdb@38873-8s6ud03x.log' ... TargetName Type Endian TapName State -- ------------------ ---------- ------ ------------------ ------------ 0 riscv.cpu0 riscv little riscv.cpu running 1* riscv.cpu1 riscv little riscv.cpu running === [{'Exception': '"/usr/local/lib/python3.11/tempfile.py",', 'ignored': 'line', 'in:': '450,', ... ``` The only reasonable (to me) explanation for the observed behavior is below. Here is how we connect to TCL-RPC server: ``` self.openocd_cli = pexpect.spawn(f"nc localhost {self.tclrpc_port}") tty.setraw(self.openocd_cli.child_fd) ``` Later we request target list by issuing "targets" command: ``` self.command("targets") ``` Internally, pexpect.spawn implemented as follows: - we fork the process - set up pty and then call execve - all these steps are written in python "Exception ignored" messages are result of exceptions thrown from finalizers of NamedTemporaryFile objects. When exception is thrown from the finalizer - python unconditionally prints a "warning" to stderr. It seems that these messages are polluting our output from "nc" since python GC can be invoked before the execve syscall. The workaround is just to make sure that execve was executed before we rely on the format of command output. To have such a guarantee we just issue a dummy "echo" command and check that we have a proper reply in the output stream. While this explanation looks convincing, the behavior above still looks strange, given that we have https://bugs.python.org/issue14548 which was resolved long ago. However, the proposed workaround fixes the issue.
2024-05-02debug: Fix loading of empty exclude lists with commentsMarek Vrbka1-1/+5
This patch fixes the case when we are using an empty exception list (for example just a YAML file with comments but without any test items to skip).
2024-02-01[debug tests] fix setting of remotetimeoutParshintsev Anatoly1-1/+0
fixes setting of `remotetimeout`. It was silently overwritten by default values from platform definition even if user specified one.
2024-02-01[debug tests] add option to log GDB remote serial protocolParshintsev Anatoly1-2/+21
introduce a new option to log communications over GDB remote serial protocol which is helpful for debugging some tests.
2024-02-01[debug tests] print selected seed for PRNGParshintsev Anatoly1-0/+3
Previously the seed was not printed and this created problems with reproduction of the issues. It's still not an ideal - meaning interactions between spike/gdb/openocd are inherently non-determistic (since time is involved), but at least we should get the same sources for the same seed now.
2023-11-09debug: use TCL-RPC to fetch results of OpenOCD commands instead of parsing ↵Parshintsev Anatoly1-29/+38
log file Quick and dirty fix for https://github.com/riscv-software-src/riscv-tests/issues/520
2023-10-11Merge pull request #503 from lz-bro/dis_timerTim Newsome1-0/+7
Disable timer interrupt to fix some bugs
2023-10-10Disable timer interrupt to fix some bugsliangzhen1-0/+7
Signed-off-by: liangzhen <zhen.liang@spacemit.com>
2023-10-03Merge pull request #508 from riscv-software-src/set_availableTim Newsome1-0/+24
debug: Add Openocd.set_available()
2023-09-29Merge pull request #509 from riscv-software-src/interlockTim Newsome1-0/+1
debug: Better interlock when interacting with gdb CLI.
2023-09-29debug: Add Openocd.set_available()Tim Newsome1-0/+24
This helper uses dmi_write commands to mark harts available/unavailable.
2023-09-29Merge pull request #507 from riscv-software-src/targetsTim Newsome1-1/+2
debug: Make Openocd.targets() tolerate blank lines.
2023-09-29Merge pull request #506 from riscv-software-src/interrupt_allTim Newsome1-3/+4
debug: Fix interrupt_all() to restore state.
2023-09-28debug: Better interlock when interacting with gdb CLI.Tim Newsome1-0/+1
Actually wait for the command to be echoed back. This means we won't be confused if there are extra newlines in gdb output.
2023-09-28debug: Make Openocd.targets() tolerate blank lines.Tim Newsome1-1/+2
2023-09-28debug: Fix interrupt_all() to restore state.Tim Newsome1-3/+4
2023-09-28debug: Add --hart command line option to gdbserver.pyTim Newsome1-3/+13
This lets you reproduce a test running on a specific hart.
2023-07-24debug: Tolerate more whitespace from OpenOCD CLITim Newsome1-1/+1
During the github workflow this character is \n, while on my computer it's ' '. I'm sure there's a good reason for that, but it doesn't seem worth figuring out what that reason is.
2023-07-17debug: CeaseRunTest -> UnavailableRunTestTim Newsome1-0/+5
Use new spike mechanism to test OpenOCD behavior when the current hart becomes unavailable while running. Create ThreadTerminated exception.
2023-07-17debug: CeaseMultiTest -> UnavailableMultiTestTim Newsome1-2/+17
Use the new spike mechanism to test OpenOCD behavior when a hart becomes unavailable while running. Create CommandException.
2023-07-17Interact with OpenOCD CLI over stdin/stdout.Tim Newsome1-39/+102
It's a bit messy to read the log file to get the output, but it seems to be flushed often so that this works. Also, added the `targets` method for retrieving the list of targets, and `wait_until_running` method to wait until all targets are in a running state.
2023-07-17Move "monitor targets" calls into a central place.Tim Newsome1-0/+1
2023-07-17Move `import random`Tim Newsome1-1/+2
Just so it's easier to quickly comment out code and hard-code the target to use without pylint complaining. This really should be a command line option.
2023-06-29Merge pull request #480 from riscv-software-src/pylintTim Newsome1-4/+7
Github workflow to run pylint against debug tests
2023-06-29Add --target-timeout to debug test script.Tim Newsome1-1/+7
I'm using this to greatly reduce the timeout when I'm reproducing a failure I know is going to time out.
2023-06-27Pylint fixes.Tim Newsome1-4/+7
2023-06-27Merge pull request #477 from MarekVCodasip/test-exclusionTim Newsome1-0/+32
Add a way to exclude tests by specifying an exclusion file
2023-06-27Add a way to exclude tests by specifying an exclusion fileMarek Vrbka1-0/+32
This patch adds a way to specify a yaml file which specifies either for each target individually or for all targets to exclude tests. Example file format is included in excluded.yaml.example.
2023-06-12Get gcc and gdb path from environment.Tim Newsome1-7/+10
If the environment variables aren't set, then use the same defaults as previously. My current set of tools use riscv64-elf-gcc and riscv64-elf-gdb, and this makes it trivial to use them.
2023-05-25debug: New pylint => new warnings => new cleanupsTim Newsome1-9/+13
- Replace general "Exception" with "GdbServerError" in gdbserver.py for when no samples are collected - Replace general "Exception" with "TargetsException" in targets.py for XLEN mismatch - Introduce "TestLibError" exception in testlib.py and replace general exceptions in various locations - Update pylint.rc to remove overgeneral-exceptions warning
2023-05-10New pylint, so make everything clean again.Tim Newsome1-8/+7
2023-02-21debug: fix pylint error W0621 redefined-outer-nameChao Du1-2/+2
2022-12-14debug: Add CeaseStepiTest.Tim Newsome1-3/+14
Test that we work correctly when the hart we're debugging ceases to respond during stepi. Add wait parameter to Gdb.stepi(), in case stepi isn't expected to complete. Parse "could not read registers" error from gdb
2022-12-14debug: Create CeaseMultiTest. (#436)Tim Newsome1-1/+9
Confirm basic debug still works when other harts have been parked using a `cease` instruction. Check that the unavailable harts are inaccessible from gdb. Add Gdb.expect() Parse "unknown thread" error from gdb.
2022-12-14debug: Remove unnecessary exit() functions. (#437)Tim Newsome1-0/+1
Also make the semi-hosting test program return 10. That's more fragile than returning 0, so makes for a better test.
2022-12-08Fix regression in VcsSim introduced by #334 (#440)Jerry Zhao1-0/+1
2022-12-01debug: Disassemble memory when a failure happens. (#432)Tim Newsome1-1/+1
This gives you less noise in the log, and more chance of figuring out what code was actually executed.
2022-12-01`flush regs` -> `maintenance flush register-cache` (#431)Tim Newsome1-1/+1
`flush regs` is being deprecated.
2022-12-01debug: Park unused harts with a cease instruction. (#434)Tim Newsome1-2/+8
`cease` is not a standard RISC-V extension, but is (was?) implemented in Rocket, and also exists in some SiFive cores. It's useful to test OpenOCD behavior when a hart becomes unavailable. See also https://github.com/chipsalliance/rocket-chip/issues/1868
2022-12-01Share exit() among more tests. (#433)Tim Newsome1-0/+8
2022-10-24Increase timeouts for multi-spike test. (#423)Tim Newsome1-1/+3
Between October 13 and October 19, something happened that makes the multi-spike tests 4 times slower. Rolling back spike, OpenOCD, or riscv-tests doesn't affect this. Presumably it's due to a kernel or python change in my Ubuntu system. I don't have time to look at this right now, so just increase the timeouts. :-( If I had to guess, there could be a bug in rbb_daisychain.py that wastes a lot of time.
2022-10-20Merge pull request #421 from riscv-software-src/pylintTim Newsome1-1/+2
Fix long line to make pylint happy.
2022-10-12Fix long line to make pylint happy.Tim Newsome1-1/+2
2022-10-12Get coverage of progbuf FPR accesses.Tim Newsome1-0/+5
Using the new spike support merged in https://github.com/riscv-software-src/riscv-isa-sim/pull/1109
2022-10-07debug: Add --debug_server arg to open gdb on OpenOCDTim Newsome1-2/+9
Not as useful as I'd like because we don't connect until after examine() has completed, and the test is likely to time out while debugging. But good to have, and maybe I'll expand on it one day.
2022-10-05Update testlib.py; remove ANSI escape sequencesYenHaoChen1-1/+2
The control sequences (^[[?2004h and ^[[?2004l) occur after the gdb.command, which results in Exception fault. This commit removes the control sequences and strips out the blank lines (^M).