aboutsummaryrefslogtreecommitdiff
path: root/tests/acceptance
AgeCommit message (Collapse)AuthorFilesLines
2021-03-12tests/acceptance: drop ARMBIAN_ARTIFACTS_CACHED condition for orangepi-pc, ↵Niek Linnenbank2-14/+0
cubieboard tests Previously the ARMBIAN_ARTIFACTS_CACHED pre-condition was added to allow running tests that have already existing armbian.com artifacts stored in the local avocado cache, but do not have working URLs to download a fresh copy. At this time of writing the URLs for artifacts on the armbian.com server are updated and working. Any future broken URLs will result in a skipped acceptance test, for example: (1/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi: CANCEL: Missing asset https://apt.armbian.com/pool/main/l/linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb (0.53 s) This commits removes the ARMBIAN_ARTIFACTS_CACHED pre-condition such that the acceptance tests for the orangepi-pc and cubieboard machines can run. Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-id: 20210310195820.21950-6-nieklinnenbank@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-12tests/acceptance: update sunxi kernel from armbian to 5.10.16Niek Linnenbank2-24/+24
The linux kernel 4.20.7 binary for sunxi has been removed from apt.armbian.com: $ ARMBIAN_ARTIFACTS_CACHED=yes AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run -t machine:orangepi-pc tests/acceptance/boot_linux_console.py Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi ... (1/6) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi: CANCEL: Missing asset https://apt.armbian.com/pool/main/l/linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb (0.55 s) This commit updates the sunxi kernel to 5.10.16 for the acceptance tests of the orangepi-pc and cubieboard machines. Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-id: 20210310195820.21950-5-nieklinnenbank@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-12tests/acceptance/boot_linux_console: change URL for ↵Niek Linnenbank1-1/+1
test_arm_orangepi_bionic_20_08 Update the download URL of the Armbian 20.08 Bionic image for test_arm_orangepi_bionic_20_08 of the orangepi-pc machine. The archive.armbian.com URL contains more images and should keep stable for a longer period of time than dl.armbian.com. Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-id: 20210310195820.21950-4-nieklinnenbank@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-12tests/acceptance/boot_linux_console: remove Armbian 19.11.3 bionic test for ↵Niek Linnenbank1-49/+23
orangepi-pc machine The image for Armbian 19.11.3 bionic has been removed from the armbian server. Without the image as input the test arm_orangepi_bionic_19_11 cannot run. This commit removes the test completely and merges the code of the generic function do_test_arm_orangepi_uboot_armbian back with the 20.08 test. Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-id: 20210310195820.21950-3-nieklinnenbank@gmail.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-10tests/avocado: add boot_xen testsAlex Bennée1-0/+118
These tests make sure we can boot the Xen hypervisor with a Dom0 kernel using the guest-loader. We currently have to use a kernel I built myself because there are issues using the Debian kernel images. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210303173642.3805-8-alex.bennee@linaro.org>
2021-02-24tests/acceptance: allow a "graceful" failing for virtio-gpu testAlex Bennée1-1/+6
This is a band-aid with a TODO for cases when QEMU doesn't start due to missing VirGL. Longer term we could do with some proper feature probing. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20210222101455.12640-4-alex.bennee@linaro.org>
2021-02-21tests/acceptance: Test PMON with Loongson-3A1000 CPUJiaxun Yang1-0/+39
Test booting of PMON bootloader on loongson3-virt platform. $ (venv) AVOCADO_ALLOW_UNTRUSTED_CODE=1 \ avocado --show=app,console \ run -t machine:loongson3-virt tests/acceptance Fetching asset from tests/acceptance/machine_mips_loongson3v.py:MipsLoongson3v.test_pmon_serial_console JOB ID : 8e202b3727847c9104d0d3d6546ed225d35f6706 JOB LOG : /home/flygoat/avocado/job-results/job-2021-01-12T10.02-8e202b3/job.log (1/1) tests/acceptance/machine_mips_loongson3v.py:MipsLoongson3v.test_pmon_serial_console: console: PMON2000 MIPS Initializing. Standby... console: 00000000Jump to 9fc console: Init Memory done. console: The uncache data is: console: 00000000: 5555555555555555 console: 00000008: aaaaaaaaaaaaaaaa console: 00000010: 3333333333333333 console: 00000018: cccccccccccccccc console: 00000020: 7777777777777777 console: 00000028: 8888888888888888 console: 00000030: 1111111111111111 console: 00000038: eeeeeeeeeeeeeeee console: The cached data is: console: 00000000: 5555555555555555 console: 00000008: aaaaaaaaaaaaaaaa console: 00000010: 3333333333333333 console: 00000018: cccccccccccccccc console: 00000020: 7777777777777777 console: 00000028: 8888888888888888 console: 00000030: 1111111111111111 console: 00000038: eeeeeeeeeeeeeeee console: Copy PMON to execute location... console: start = 0x8f900000 console: s0 = 0x30300000 console: _edata = 0x8f989010 console: _end = 0x8f98a028copy text section done. console: Copy PMON to execute location done. console: sp=8f8fc000 console: Uncompressing Bios............................................................................OK,Booting Bios console: FREQ console: DONE console: DEVI console: ENVI console: MAPV console: NVRAM@8f7ff898 console: STDV console: 80100000: memory between 8f7ff400-8f800000 is already been allocated,heap is already above this point console: SBDD console: P12PCIH console: PCIH console: PCID console: setting up 1 bus console: PCI bus 0 slot 1: probe...completed console: PCI bus 0 slot 1/0: vendor/product: 0x106b/0x003f (serialbus, USB, interface: 0x10, revision: 0x00) console: PCI bus 0 slot 1/0: reg 0x10 = 0xffffff00 console: PCI bus 0 slot 2: probe...completed console: PCI bus 0 slot 2/0: vendor/product: 0x1af4/0x1000 (network, ethernet, interface: 0x00, revision: 0x00) console: PCI bus 0 slot 2/0: reg 0x10 = 0xffffffe1 console: PCI bus 0 slot 2/0: reg 0x14 = 0xfffff000 console: PCI bus 0 slot 2/0: reg 0x20 = 0xffffc00c console: PCI bus 0 slot 2/0: reg 0x30 = 0xfffc0000 console: PCI bus 0 slot 3: probe...completed console: PCI bus 0 slot 4: probe...completed console: PCI bus 0 slot 5: probe...completed console: PCI bus 0 slot 6: probe...completed console: PCI bus 0 slot 7: probe...completed console: PCI bus 0 slot 8: probe...completed console: PCI bus 0 slot 9: probe...completed console: PCI bus 0 slot 10: probe...completed console: PCI bus 0 slot 11: probe...completed console: PCI bus 0 slot 12: probe...completed console: PCI bus 0 slot 13: probe...completed console: PCI bus 0 slot 14: probe...completed console: PCI bus 0 slot 15: probe...completed console: PCI bus 0 slot 16: probe...completed console: PCI bus 0 slot 17: probe...completed console: PCI bus 0 slot 18: probe...completed console: PCI bus 0 slot 19: probe...completed console: PCI bus 0 slot 20: probe...completed console: PCI bus 0 slot 21: probe...completed console: PCI bus 0 slot 22: probe...completed console: PCI bus 0 slot 23: probe...completed console: PCI bus 0 slot 24: probe...completed console: PCI bus 0 slot 25: probe...completed console: PCI bus 0 slot 26: probe...completed console: PCI bus 0 slot 27: probe...completed console: PCI bus 0 slot 28: probe...completed console: PCI bus 0 slot 29: probe...completed console: PCI bus 0 slot 30: probe...completed console: PCI bus 0 slot 31: probe...completed console: PCIS console: PCIR console: PCIW console: PCI bus 0 slot 2/0: mem @0x40000000, reg 0x30 262144 bytes console: PCI bus 0 slot 2/0: mem @0x40040000, reg 0x20 16384 bytes console: PCI bus 0 slot 2/0: mem @0x40044000, reg 0x14 4096 bytes console: PCI bus 0 slot 1/0: mem @0x40045000, reg 0x10 256 bytes console: PCI bus 0 slot 2/0: exp @0x40000000, 262144 bytes console: PCI bus 0 slot 2/0: i/o @0x00004000, reg 0x10 32 bytes console: NETI console: RTCL console: PCID console: VGAI console: memorysize=c000000,base=8f6ff508,sysMem=8f6ef500 console: in setup_int_vect!done!VESA console: vga bios init failed, rc=-1 console: in configure console: mainbus0 (root) console: localbus0 at mainbus0 console: loopdev0 at mainbus0pcibr0 at mainbus0 console: pci0 at pcibr0 bus 0 console: ohci0 at pci0 dev 1 function 0 vendor/product: 0x106b/0x003f (serialbus, USB, interface: 0x10, revision: 0x00)usb base addr : 0xc0045000, bus_base is : 0xc0000000 console: OHCI revision: 0x00000010 console: RH: a: 0x00000203 b: 0x00000000 console: early period(0x0) console: OHCI 8c01ec00 initialized ok console: New Device 0 console: usb_get_descriptor console: bLength = 12 console: bDescriptorType =1 console: bcdUSB = 110 console: bDeviceClass =9 console: bDeviceSubClass =0 console: bDeviceProtocol =0 console: bMaxPacketSize0 =8 console: set address 1 console: usb_get_descriptor console: bLength = 12 console: bDescriptorType =1 console: bcdUSB = 110 console: bDeviceClass =9 console: bDeviceSubClass =0 console: bDeviceProtocol =0 console: bMaxPacketSize0 =8 console: idVendor =0 console: idProduct =0 console: bcdDevice =0 console: iManufacturer=0 console: iProduct =1 console: iSerialNumber=0 console: bNumConfigurations=1 console: usb_get_descriptor console: usb_get_descriptor console: get_conf_no 0 Result 25, wLength 25 console: if 0, ep 0 console: bLength=9 console: bDescriptorType=2 console: wTotalLength=19 console: bNumInterfaces=1 console: bConfigurationValue=1 console: iConfiguration=0 console: bmAttributes=40 console: MaxPower=0 console: 09 04 00 00 01 09 00 00 00 07 05 81 03 02 00 ff console: ##EP epmaxpacketin[1] = 2 console: set configuration 1 console: new device strings: Mfr=0, Product=1, SerialNumber=0 console: USB device number 1 default language ID 0x409 console: Manufacturer console: Product OHCI Root Hub console: SerialNumber console: New Device 1 console: usb_get_descriptor console: bLength = 12 console: bDescriptorType =1 console: bcdUSB = 200 console: bDeviceClass =0 console: bDeviceSubClass =0 console: bDeviceProtocol =0 console: bMaxPacketSize0 =8 console: set address 2 console: usb_get_descriptor console: bLength = 12 console: bDescriptorType =1 console: bcdUSB = 200 console: bDeviceClass =0 console: bDeviceSubClass =0 console: bDeviceProtocol =0 console: bMaxPacketSize0 =8 console: idVendor =627 console: idProduct =1 console: bcdDevice =0 console: iManufacturer=1 console: iProduct =4 console: iSerialNumber=b console: bNumConfigurations=1 console: usb_get_descriptor console: usb_get_descriptor console: get_conf_no 0 Result 34, wLength 34 console: unknown Description Type : 21 console: 09 21 11 01 00 01 22 3F 00 console: if 0, ep 0 console: bLength=9 console: bDescriptorType=2 console: wTotalLength=22 console: bNumInterfaces=1 console: bConfigurationValue=1 console: iConfiguration=8 console: bmAttributes=a0 console: MaxPower=32 console: 09 04 00 00 01 03 01 01 00 09 21 11 01 00 01 22 3f 00 07 05 81 03 08 00 0a console: ##EP epmaxpacketin[1] = 8 console: set configuration 1 console: new device strings: Mfr=1, Product=4, SerialNumber=11 console: USB device number 2 default language ID 0x409 console: Manufacturer QEMU console: Product QEMU USB Keyboard console: SerialNumber 68284-0000:00:01.0-1 console: drive at ohci0 devnum 2, Product QEMU USB Keyboard console: not configured console: New Device 2 console: usb_get_descriptor console: bLength = 12 console: bDescriptorType =1 console: bcdUSB = 200 console: bDeviceClass =0 console: bDeviceSubClass =0 console: bDeviceProtocol =0 console: bMaxPacketSize0 =8 console: set address 3 console: usb_get_descriptor console: bLength = 12 console: bDescriptorType =1 console: bcdUSB = 200 console: bDeviceClass =0 console: bDeviceSubClass =0 console: bDeviceProtocol =0 console: bMaxPacketSize0 =8 console: idVendor =627 console: idProduct =1 console: bcdDevice =0 console: iManufacturer=1 console: iProduct =3 console: iSerialNumber=a console: bNumConfigurations=1 console: usb_get_descriptor console: usb_get_descriptor console: get_conf_no 0 Result 34, wLength 34 console: unknown Description Type : 21 console: 09 21 01 00 00 01 22 4A 00 console: if 0, ep 0 console: bLength=9 console: bDescriptorType=2 console: wTotalLength=22 console: bNumInterfaces=1 console: bConfigurationValue=1 console: iConfiguration=7 console: bmAttributes=a0 console: MaxPower=32 console: 09 04 00 00 01 03 00 00 00 09 21 01 00 00 01 22 4a 00 07 05 81 03 08 00 0a console: ##EP epmaxpacketin[1] = 8 console: set configuration 1 console: new device strings: Mfr=1, Product=3, SerialNumber=10 console: USB device number 3 default language ID 0x409 console: Manufacturer QEMU console: Product QEMU USB Tablet console: SerialNumber 28754-0000:00:01.0-2 console: drive at ohci0 devnum 3, Product QEMU USB Tablet console: not configured console: drive at ohci0 devnum 1, Product OHCI Root Hub console: not configured console: vendor/product: 0x1af4/0x1000 (network, ethernet, interface: 0x00, revision: 0x00) at pci0 dev 2 function 0 not configured console: out configure console: Press <Del> to set BIOS,waiting for 3 seconds here..... console: devconfig done. console: ifinit done. console: domaininit done. console: init_proc.... console: HSTI console: SYMI console: SBDE console: [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ console: [[ [[[[[[[[[ [[[[[ [[[[ [[[[[ [[[[[ [[[[[ [[[[[ [[[[ [[[[[ [[ console: [[ [[[[[[[[ [[[[ [[[ [[[[ [[[ [[[[ [[[[ [[[[ [[[ [[[[ [[[ [[[[ [[[ [[[[ [[ console: [[ [[[[[[[[ [[[[[[ [[[ [[[[[[ [[[ [ [[[ [[[ [[[[[[[[[[[[ [[[[[[[ [[[[[[ [[[ [ [[[ [[ console: [[ [[[[[[[[ [[[[[[ [[[ [[[[[[ [[[ [[ [[ [[[ [[[ [[[[[[[ [[[[ [[[[[[ [[[ [[ [[ [[ console: [[ [[[[[[[[ [[[[[[ [[[ [[[[[[ [[[ [[[ [ [[[ [[[[[ [[[[[[[[[[ [[[ [[[[[[ [[[ [[[ [ [[ console: [[ [[[[[[[[ [[[[ [[[ [[[[ [[[ [[[[ [[[ [[[[ [[[ [[[ [[[[ [[[[ [[[ [[[[ [[ console: [[ [[[[ [[[[[ [[[[ [[[[[ [[[[ [[[[[ [[[[[[ [[[[ [[[[[ [[ console: [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[2011 Loongson][[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ console: Configuration [Bonito,EL,NET,SCSI,IDE] console: Version: PMON2000 3.3 (Bonito) #0: Tue Dec 22 01:58:09 UTC 2020 commit b3ece66234adbf7d4e453f0ba4f326c099ac2a76 Author: Jiaxun Yang <jiaxun.yang@flygoat.com> Date: Tue Dec 22 09:51:10 2020 +0800 . console: Supported loaders [txt, srec, elf, bin] console: Supported filesystems [net, fat, fs, disk, iso9660, socket, tty, ram] console: This software may be redistributed under the BSD copyright. console: Copyright 2000-2002, Opsycon AB, Sweden. console: Copyright 2005, ICT CAS. console: CPU GODSON3 BogoMIPS: 1327 PASS (3.89 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 4.38 s Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20210112020708.62922-1-jiaxun.yang@flygoat.com> [PMD: Set timeout to 60sec, simply test for ''CPU GODSON3 BogoMIPS'] Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-02-18tests/acceptance: add a memory callback checkAlex Bennée1-0/+31
This test makes sure that the inline and callback based memory checks count the same number of accesses. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20210213130325.14781-24-alex.bennee@linaro.org>
2021-02-18tests/acceptance: add a new tests to detect counting errorsAlex Bennée1-0/+26
The insn plugin has a simple heuristic to detect if an instruction is detected running twice in a row. Check the plugin log after the run and pass accordingly. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20210213130325.14781-22-alex.bennee@linaro.org>
2021-02-18tests/acceptance: add a new set of tests to exercise pluginsAlex Bennée1-0/+91
This is just a simple test to count the instructions executed by a kernel. However a later test will detect a failure condition when icount is enabled. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20210213130325.14781-16-alex.bennee@linaro.org>
2021-02-15Acceptance Tests: set up existing ssh keys by defaultCleber Rosa1-0/+14
It's questionable whether it's necessary to create one brand new pair for each test. It's not questionable that it takes less time and resources to just use the keys available at "tests/keys" that exist for that exact reason. If a location for the public key is not given explicitly, the LinuxTest will now set up the existing pair of keys as the default. This removes the need for a lot of boilerplate code. To avoid the ssh client from erroring on permission issues, a directory with restrictive permissions is created for the private key. This should still be a lot cheaper than creating a new key. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210203172357.1422425-19-crosa@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> [marcandre: fix typos in commit message] Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15Acceptance Tests: fix population of public key in cloudinit imageCleber Rosa1-1/+3
Currently the path of the ssh public key is being set, but its content is obviously what's needed. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210203172357.1422425-18-crosa@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15Acceptance Tests: introduce method for requiring an acceleratorCleber Rosa3-29/+34
Some tests explicitly require a QEMU accelerator to be available. Given that this depends on some runtime aspects not known before the test is started, such as the currently set QEMU binary, it's left to be checked also at runtime. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210203172357.1422425-17-crosa@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15Acceptance Tests: introduce LinuxTest base classCleber Rosa3-93/+94
This is basically the infrastructure around "boot_linux.py" tests, but now made into a base class for general use. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210203172357.1422425-15-crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15tests/acceptance/virtio-gpu.py: preserve virtio-user-gpu logCleber Rosa1-2/+3
Preserve log at location already prepared for keeping the test's log files. While at it, log info about its location (in the main test log file), instead of printing it out. Reference: https://avocado-framework.readthedocs.io/en/85.0/api/test/avocado.html#avocado.Test.logdir Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> [philmd: use full sentence] Message-Id: <20210211220146.2525771-7-crosa@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15virtiofs_submounts.py test: Note on vmlinuz paramMax Reitz1-0/+12
From the cancel message, it is not entirely clear why this parameter is mandatory now, or that it will be optional in the future. Add such a more detailed explanation as a comment in the test source file. Suggested-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210212151649.252440-1-mreitz@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-08Merge remote-tracking branch ↵Peter Maydell8-105/+224
'remotes/philmd-gitlab/tags/integration-testing-20210208' into staging Integration testing patches Tests added: - Armbian 20.08 on Orange Pi PC (Philippe) - MPC8544ds machine (Thomas) - Virtex-ml507 ppc machine (Thomas) - Re-enable the microblaze test (Thomas) Various fixes and documentation improvements from Cleber. # gpg: Signature made Mon 08 Feb 2021 20:19:12 GMT # gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE # gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full] # Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE * remotes/philmd-gitlab/tags/integration-testing-20210208: Acceptance Tests: remove unnecessary tag from documentation example Acceptance tests: clarify ssh connection failure reason tests/acceptance/virtiofs_submounts: required space between IP and port tests/acceptance/virtiofs_submounts: standardize port as integer tests/acceptance/virtiofs_submounts: use a virtio-net device instead tests/acceptance/virtiofs_submounts: do not ask for ssh key password tests/acceptance/virtiofs_submounts: use workdir property tests/acceptance/boot_linux: rename misleading cloudinit method tests/acceptance/boot_linux: fix typo on cloudinit error message tests/acceptance: Re-enable the microblaze test tests/acceptance: Add a test for the virtex-ml507 ppc machine tests/acceptance: Test the mpc8544ds machine tests/acceptance: Move the pseries test to a separate file tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC tests/acceptance: Extract do_test_arm_orangepi_armbian_uboot() method tests/acceptance: Introduce tesseract_ocr() helper tests/acceptance: Extract tesseract_available() helper in new namespace Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-08Acceptance tests: clarify ssh connection failure reasonCleber Rosa2-2/+2
If the connection to the ssh server fails, it may indeed be a "sshd" issue, but it may also not be that. Let's state what we know: the establishment of the connection from the client side was not possible. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210203172357.1422425-13-crosa@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance/virtiofs_submounts: required space between IP and portCleber Rosa1-1/+1
AFAICT, there should not be a situation where IP and port do not have at least one whitespace character separating them. This may be true for other '\s*' patterns in the same regex too. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210203172357.1422425-10-crosa@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance/virtiofs_submounts: standardize port as integerCleber Rosa1-3/+4
Instead of having to cast it whenever it's going to be used, let's standardize it as an integer, which is the data type that will be used most often. Given that the regex will only match digits, it's safe that we'll end up getting a integer, but, it could as well be a zero. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Message-Id: <20210203172357.1422425-9-crosa@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance/virtiofs_submounts: use a virtio-net device insteadCleber Rosa1-1/+1
In a virtiofs based tests, it seems safe to assume that the guest will be capable of a virtio-net device. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20210203172357.1422425-7-crosa@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance/virtiofs_submounts: do not ask for ssh key passwordCleber Rosa1-1/+1
Tests are supposed to be non-interactive, and ssh-keygen is asking for a passphrase when creating a key. Let's set an empty passphrase to avoid the prompt. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20210203172357.1422425-6-crosa@redhat.com> [PMD: Reword description per Alex Bennée comment] Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance/virtiofs_submounts: use workdir propertyCleber Rosa1-4/+2
For Avocado Instrumented based tests, it's a better idea to just use the property. The environment variable is a fall back for tests not written using that Python API. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reference: https://avocado-framework.readthedocs.io/en/84.0/api/test/avocado.html#avocado.Test.workdir Message-Id: <20210203172357.1422425-5-crosa@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance/boot_linux: rename misleading cloudinit methodCleber Rosa1-6/+6
There's no downloading happening on that method, so let's call it "prepare" instead. While at it, and because of it, the current "prepare_boot" and "prepare_cloudinit" are also renamed. The reasoning here is that "prepare_" methods will just work on the images, while "set_up_" will make them effective to the VM that will be launched. Inspiration comes from the "virtiofs_submounts.py" tests, which this expects to converge more into. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Beraldo Leal <bleal@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20210203172357.1422425-3-crosa@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance/boot_linux: fix typo on cloudinit error messageCleber Rosa1-1/+1
Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Beraldo Leal <bleal@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210203172357.1422425-2-crosa@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance: Re-enable the microblaze testThomas Huth2-9/+35
The microblaze kernel sometimes gets stuck during boot (ca. 1 out of 200 times), so we disabled the corresponding acceptance tests some months ago. However, it's likely better to check that the kernel is still starting than to not testing it at all anymore. Move the test to a separate file, enable it again and check for an earlier console message that should always appear. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20210128152815.585478-1-thuth@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance: Add a test for the virtex-ml507 ppc machineThomas Huth1-0/+18
The "And a hippo new year" image from the QEMU advent calendar 2020 can be used to test the virtex-ml507 ppc machine. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Acked-by: David Gibson <david@gibson.dropbear.id.au> Message-Id: <20210112164045.98565-4-thuth@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance: Test the mpc8544ds machineThomas Huth1-0/+17
We can use the "Stupid creek" image to test the mpc8544ds ppc machine. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Acked-by: David Gibson <david@gibson.dropbear.id.au> Message-Id: <20210112164045.98565-3-thuth@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance: Move the pseries test to a separate fileThomas Huth2-19/+34
Let's gather the POWER-related tests in a separate file. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: David Gibson <david@gibson.dropbear.id.au> Message-Id: <20210112164045.98565-2-thuth@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PCPhilippe Mathieu-Daudé1-0/+25
Test U-Boot and Linux on the recent Armbian release 20.08. Suggested-by: Cleber Rosa <crosa@redhat.com> Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com> Tested-by: Bin Meng <bin.meng@windriver.com> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20201023131808.3198005-5-f4bug@amsat.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance: Extract do_test_arm_orangepi_armbian_uboot() methodPhilippe Mathieu-Daudé1-21/+24
As we want to reuse the same U-Boot test for multiple Armbian releases, extract the common part as do_test_arm_orangepi_armbian_uboot(). Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Bin Meng <bin.meng@windriver.com> Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com> Message-Id: <20201023131808.3198005-4-f4bug@amsat.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance: Introduce tesseract_ocr() helperPhilippe Mathieu-Daudé2-16/+23
We are going to reuse the tesseract OCR code. Create a new tesseract_ocr() helper and use it. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20201021105035.2477784-5-f4bug@amsat.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance: Extract tesseract_available() helper in new namespacePhilippe Mathieu-Daudé2-22/+31
We are going to reuse tesseract_available(). Extract it to a new 'tesseract_utils' namespace. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20201021105035.2477784-4-f4bug@amsat.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08tests/acceptance: Increase the timeout in the replay testsThomas Huth1-1/+1
Our gitlab-CI just showed a failed test_ppc_mac99 since it was apparently killed some few seconds before the test finished. Allow it some more time to complete. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Acked-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20210127065222.48650-1-thuth@redhat.com>
2021-02-04tests: add some virtio-gpu & vhost-user-gpu acceptance testMarc-André Lureau1-0/+161
This will check virtio/vhost-user-vga & virgl are correctly initialized by the Linux kernel on an egl-headless display. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20210204105232.834642-21-marcandre.lureau@redhat.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2021-01-23acceptance: switch to QMP change-vnc-password commandPaolo Bonzini1-12/+6
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-11tests/acceptance: Fix race conditions in s390x tests & skip fedora on gitlab-CIThomas Huth1-2/+12
There was a race condition in the first test where there was already the "crw" output in the dmesg, but the "0.0.4711" entry has not been created in the /sys fs yet. Fix it by waiting until it is there. The second test has even more problems on gitlab-CI. Even after adding some more synchronization points (that wait for some messages in the "dmesg" output to make sure that the modules got loaded correctly), there are still occasionally some hangs in this test when it is running in the gitlab-CI. So far I was unable to reproduce these hangs locally on my computer, so this issue might take a while to debug. Thus disable the 2nd test in the gitlab-CI until the problems are better understood and fixed. Signed-off-by: Thomas Huth <thuth@redhat.com> Tested-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210108185645.86351-1-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-06tests/acceptance: bound the size of readline in s390_ccw_virtioAlex Bennée1-1/+1
The read binary data as text via a PPM export of the frame buffer seems a bit sketchy and it did blow up in the real world when the assertion failed: https://gitlab.com/qemu-project/qemu/-/jobs/943183183 However short of cleaning up the test to be more binary focused at least limit the attempt to dump the whole file as hexified zeros in the logs. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Acked-by: Halil Pasic <pasic@linux.ibm.com> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20210105124405.15424-1-alex.bennee@linaro.org>
2021-01-04tests/acceptance: Test boot_linux_console for fuloong2eJiaxun Yang1-0/+21
The kernel comes from debian archive so it's trusted. Invoking the test can be done as follows: $ avocado --show=app,console run -t machine:fuloong2e tests/acceptance/ (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_mips64el_fuloong2e: console: [ 0.000000] Initializing cgroup subsys cpuset console: [ 0.000000] Initializing cgroup subsys cpu console: [ 0.000000] Initializing cgroup subsys cpuacct console: [ 0.000000] Linux version 3.16.0-6-loongson-2e (debian-kernel@lists.debian.org) (gcc version 4.8.4 (Debian 4.8.4-1) ) #1 Debian 3.16.56-1+deb8u1 (2018-05-08) console: [ 0.000000] memsize=256, highmemsize=0 console: [ 0.000000] CpuClock = 533080000 console: [ 0.000000] bootconsole [early0] enabled console: [ 0.000000] CPU0 revision is: 00006302 (ICT Loongson-2) console: [ 0.000000] FPU revision is: 00000501 console: [ 0.000000] Checking for the multiply/shift bug... no. console: [ 0.000000] Checking for the daddiu bug... no. console: [ 0.000000] Determined physical RAM map: console: [ 0.000000] memory: 0000000010000000 @ 0000000000000000 (usable) console: [ 0.000000] memory: 0000000004000000 @ 0000000010000000 (reserved) console: [ 0.000000] memory: 0000000003ffffff @ 000000001c000001 (reserved) console: [ 0.000000] Initrd not found or empty - disabling initrd console: [ 0.000000] Zone ranges: console: [ 0.000000] DMA [mem 0x00000000-0x00ffffff] console: [ 0.000000] Normal [mem 0x01000000-0x0fffffff] console: [ 0.000000] Movable zone start for each node console: [ 0.000000] Early memory node ranges console: [ 0.000000] node 0: [mem 0x00000000-0x0fffffff] console: [ 0.000000] Reserving 0MB of memory at 0MB for crashkernel console: [ 0.000000] Primary instruction cache 64kB, VIPT, direct mapped, linesize 32 bytes. console: [ 0.000000] Primary data cache 64kB, 4-way, VIPT, no aliases, linesize 32 bytes console: [ 0.000000] Unified secondary cache 512kB 4-way, linesize 32 bytes. console: [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 16327 console: [ 0.000000] Kernel command line: printk.time=0 console=ttyS0 PASS (2.27 s) Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20201224031750.52146-9-jiaxun.yang@flygoat.com> [PMD: Added command line example] Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2020-12-22tests/acceptance: Add a test with the Fedora 31 kernel and initrdThomas Huth1-0/+110
This initrd contains a virtio-net and a virtio-gpu kernel module, so we can check that we can set a MAC address for the network device and whether we can hot-plug and -unplug a virtio-crypto device. But the most interesting part is maybe that we can also successfully write some stuff into the emulated framebuffer of the virtio-gpu device and make sure that we can read back that data from a screenshot. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20201221143423.23607-1-thuth@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Tested-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2020-12-21tests/acceptance: Test the virtio-balloon device on s390xThomas Huth1-1/+11
Inflate the balloon and check whether the size of the memory changes. Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Tested-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20201215183623.110128-4-thuth@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2020-12-21tests/acceptance: Test virtio-rng on s390 via /dev/hwrngThomas Huth1-2/+15
/dev/hwrng is only functional if virtio-rng is working right, so let's add a sanity check for this device node. Reviewed-by: Willian Rampazzo <willianr@redhat.com> Tested-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20201215183623.110128-3-thuth@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2020-12-21tests/acceptance: Extract the code to clear dmesg and wait for CRW reportsThomas Huth1-13/+17
We will use this in more spots soon, so it's easier to put this into a separate function. Reviewed-by: Willian Rampazzo <willianr@redhat.com> Tested-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20201215183623.110128-2-thuth@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2020-12-21tests/acceptance: test hot(un)plug of ccw devicesCornelia Huck1-0/+24
Hotplug a virtio-net-ccw device, and then hotunplug it again. Signed-off-by: Cornelia Huck <cohuck@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Tested-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20201208122843.147186-1-cohuck@redhat.com>
2020-12-11tests/acceptance: test s390x zpci fid propagationCornelia Huck1-3/+9
Verify that a fid specified on the command line shows up correctly as the function_id in the guest. Signed-off-by: Cornelia Huck <cohuck@redhat.com> [re-formatted overlong lines] Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20201130180216.15366-4-cohuck@redhat.com>
2020-12-11tests/acceptance: verify s390x device detectionCornelia Huck1-0/+11
The kernel/initrd combination does not provide the virtio-net driver; therefore, simply check whether the presented device type is indeed virtio-net for the two virtio-net-{ccw,pci} devices. Signed-off-by: Cornelia Huck <cohuck@redhat.com> [re-formatted overlong lines] Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20201130180216.15366-3-cohuck@redhat.com>
2020-12-11tests/acceptance: test virtio-ccw revision handlingCornelia Huck1-2/+18
The max_revision prop of virtio-ccw devices can be used to force an older revision for compatibility handling. The easiest way to check this is to force a device to revision 0, which turns off virtio-1. Signed-off-by: Cornelia Huck <cohuck@redhat.com> [re-formatted overlong lines] Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20201130180216.15366-2-cohuck@redhat.com>
2020-12-11tests/acceptance: add a test for devices on s390xCornelia Huck1-0/+68
This adds a very basic test for checking that we present devices in a way that Linux can consume: boot with both virtio-net-ccw and virtio-net-pci attached and then verify that Linux is able to see and detect these devices. Signed-off-by: Cornelia Huck <cohuck@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20201126130158.1471985-1-cohuck@redhat.com>
2020-11-23tests/avocado: clean-up socket directory after runAlex Bennée1-2/+3
Previously we were leaving temporary directories behind. While the QEMUMachine does make efforts to clean up after itself the directory belongs to the calling function. We use TemporaryDirectory to wrap this although we explicitly clear the reference in tearDown() as it doesn't get cleaned up otherwise. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20201117173635.29101-4-alex.bennee@linaro.org>
2020-11-23tests: add prefixes to the bare mkdtemp callsAlex Bennée1-1/+2
The first step to debug a thing is to know what created the thing in the first place. Add some prefixes so random tmpdir's have something grep in the code. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20201117173635.29101-3-alex.bennee@linaro.org>
generated by cgit v1.1 at 2024-11-03 04:18:37 +0000