aboutsummaryrefslogtreecommitdiff
path: root/tests/qemu-iotests
AgeCommit message (Collapse)AuthorFilesLines
2019-04-02tests/qemu-iotests/235: Allow fallback to tcgThomas Huth1-1/+1
iotest 235 currently only works with KVM - this is bad for systems where it is not available, e.g. CI pipelines. The test also works when using "tcg" as accelerator, so we can simply add that to the list of accelerators, too. Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-02block: test block-stream with a base node that is used by block-commitAlberto Garcia2-2/+19
The base node of a block-stream operation indicates the first image from the backing chain starting from which no data is copied to the top node. The block-stream job allows others to use that base image, so a second block-stream job could be writing to it at the same time. An important restriction is that the base image must not disappear while the stream job is ongoing. stream_start() freezes the backing chain from top to base with that purpose but it does it too late in the code so there is a race condition there. This bug was fixed in the previous commit, and this patch contains an iotest for this scenario. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-02tests/qemu-iotests: Remove redundant COPYING fileThomas Huth1-339/+0
The file tests/qemu-iotests/COPYING is the same text as in the COPYING file in the main directory. So as far as I can see, we don't need the duplicate here. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-02iotests: Fix test 200 on s390x without virtio-pciThomas Huth1-3/+11
virtio-pci is optional on s390x, e.g. in downstream RHEL builds, it is disabled. On s390x, virtio-ccw should be used instead. Other tests like 051 or 240 already use virtio-scsi-ccw instead of virtio-scsi-pci on s390x, so let's do the same here and always use virtio-scsi-ccw on s390x. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-01nbd/server: Advertise actual minimum block sizeEric Blake3-7/+9
Both NBD_CMD_BLOCK_STATUS and structured NBD_CMD_READ will split their reply according to bdrv_block_status() boundaries. If the block device has a request_alignment smaller than 512, but we advertise a block alignment of 512 to the client, then this can result in the server reply violating client expectations by reporting a smaller region of the export than what the client is permitted to address (although this is less of an issue for qemu 4.0 clients, given recent client patches to overlook our non-compliance at EOF). Since it's always better to be strict in what we send, it is worth advertising the actual minimum block limit rather than blindly rounding it up to 512. Note that this patch is not foolproof - it is still possible to provoke non-compliant server behavior using: $ qemu-nbd --image-opts driver=blkdebug,align=512,image.driver=file,image.filename=/path/to/non-aligned-file That is arguably a bug in the blkdebug driver (it should never pass back block status smaller than its alignment, even if it has to make multiple bdrv_get_status calls and determine the least-common-denominator status among the group to return). It may also be possible to observe issues with a backing layer with smaller alignment than the active layer, although so far I have been unable to write a reliable iotest for that scenario (but again, an issue like that could be argued to be a bug in the block layer, or something where we need a flag to bdrv_block_status() to state whether the result must be aligned to the current layer's limits or can be subdivided for accuracy when chasing backing files). Anyways, as blkdebug is not normally used, and as this patch makes our server more interoperable with qemu 3.1 clients, it is worth applying now, even while we still work on a larger patch series for the 4.1 timeframe to have byte-accurate file lengths. Note that the iotests output changes - for 223 and 233, we can see the server's better granularity advertisement; and for 241, the three test cases have the following effects: - natural alignment: the server's smaller alignment is now advertised, and the hole reported at EOF is now the right result; we've gotten rid of the server's non-compliance - forced server alignment: the server still advertises 512 bytes, but still sends a mid-sector hole. This is still a server compliance bug, which needs to be fixed in the block layer in a later patch; output does not change because the client is already being tolerant of the non-compliance - forced client alignment: the server's smaller alignment means that the client now sees the server's status change mid-sector without any protocol violations, but the fact that the map shows an unaligned mid-sector hole is evidence of the block layer problems with aligned block status, to be fixed in a later patch Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190329042750.14704-7-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> [eblake: rebase to enhanced iotest 241 coverage]
2019-03-30nbd/client: Report offsets in bdrv_block_statusEric Blake3-14/+14
It is desirable for 'qemu-img map' to have the same output for a file whether it is served over file or nbd protocols. However, ever since we implemented block status for NBD (2.12), the NBD protocol forgot to inform the block layer that as the final layer in the chain, the offset is valid; without an offset, the human-readable form of qemu-img map gives up with the unhelpful: $ nbdkit -U - data data="1" size=512 --run 'qemu-img map $nbd' Offset Length Mapped to File qemu-img: File contains external, encrypted or compressed clusters. The --output=json form always works, because it is reporting the lower-level bdrv_block_status results directly rather than trying to filter out sparse ranges for human consumption - but now it also shows the offset member. With this patch, the human output changes to: Offset Length Mapped to File 0 0x200 0 nbd+unix://?socket=/tmp/nbdkitOxeoLa/socket This change is observable to several iotests. Fixes: 78a33ab5 Reported-by: Richard W.M. Jones <rjones@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190329042750.14704-4-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-03-30iotests: Add 241 to test NBD on unaligned imagesEric Blake3-0/+127
Add a test for the NBD client workaround in the previous patch. It's not really feasible for an iotest to assume a specific tracing engine, so we can't really probe trace_nbd_parse_blockstatus_compliance to see if the server was fixed vs. whether the client just worked around the server (other than by rearranging order between code patches and this test). But having a successful exchange sure beats the previous state of an error message. Since format probing can change alignment, we can use that as an easy way to test several configurations. Not tested yet, but worth adding to this test in future patches: an NBD server that can advertise a non-sector-aligned size (such as nbdkit) causes qemu as the NBD client to misbehave when it rounds the size up and accesses beyond the advertised size. Qemu as NBD server never advertises a non-sector-aligned size (since bdrv_getlength() currently rounds up to sector boundaries); until qemu can act as such a server, testing that flaw will have to rely on external binaries. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190329042750.14704-2-eblake@redhat.com> Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> [eblake: add forced-512 alignment, and nbdkit reproducer comment]
2019-03-26iotests: add 248: test resume mirror after auto pause on ENOSPCVladimir Sementsov-Ogievskiy3-0/+80
Test that mirror job actually resume on resume command after being automatically paused on ENOSPC error. It's a follow-up test for 8d9648cbf3e "blockjob: fix user pause in block_job_error_action" Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Tested-by: John Snow <jsnow@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-19qemu-iotests: Treat custom TEST_DIR in 051Lukáš Doktor1-1/+1
When custom TEST_DIR is specified the output includes it without leading '/': $ TEST_DIR=/var/tmp ./check -file -qcow2 051 .... -drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": SNAPSHOT_PATH}} (qcow2, read-only) +drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/vl.ziHfeP"}} (qcow2, read-only) Let's remove it from the sed regexp. Signed-off-by: Lukáš Doktor <ldoktor@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-19qemu-iotests: Fix 232 for non-qcow2Kevin Wolf5-50/+102
232 is marked as generic, but commit 12efe428c9e added code that assumes qcow2. What the new test really needs is backing files and support for updating the backing file link (.bdrv_change_backing_file). Split the non-generic code into a new test case 247 and make it work with qed, too. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-19iotests: 153: Wait for an answer to QMP commandsSergio Lopez2-6/+12
There are various actions in this test that must be executed sequentially, as the result of it depends on the state triggered by the previous one. If the last argument of _send_qemu_cmd() is an empty string, it just sends the QMP commands without waiting for an answer. While unlikely, it may happen that the next action in the test gets invoked before QEMU processes the QMP request. This issue seems to be easier to reproduce on servers with limited resources or highly loaded. With this change, we wait for an answer on all _send_qemu_cmd() calls. Signed-off-by: Sergio Lopez <slp@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-13Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into ↵Peter Maydell6-2/+553
staging Pull request # gpg: Signature made Tue 12 Mar 2019 20:23:08 GMT # gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E # gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full] # Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB # Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E * remotes/jnsnow/tags/bitmaps-pull-request: (22 commits) tests/qemu-iotests: add bitmap resize test 246 block/qcow2-bitmap: Allow resizes with persistent bitmaps block/qcow2-bitmap: Don't check size for IN_USE bitmap docs/interop/qcow2: Improve bitmap flag in_use specification bitmaps: Fix typo in function name block/dirty-bitmaps: implement inconsistent bit block/dirty-bitmaps: disallow busy bitmaps as merge source block/dirty-bitmaps: prohibit removing readonly bitmaps block/dirty-bitmaps: prohibit readonly bitmaps for backups block/dirty-bitmaps: add block_dirty_bitmap_check function block/dirty-bitmap: add inconsistent status block/dirty-bitmaps: add inconsistent bit iotests: add busy/recording bit test to 124 blockdev: remove unused paio parameter documentation block/dirty-bitmaps: move comment block block/dirty-bitmaps: unify qmp_locked and user_locked calls block/dirty-bitmap: explicitly lock bitmaps with successors nbd: change error checking order for bitmaps block/dirty-bitmap: change semantics of enabled predicate block/dirty-bitmap: remove set/reset assertions against enabled bit ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org> # Conflicts: # tests/qemu-iotests/group
2019-03-12qemu-iotests: Test the x-blockdev-reopen QMP commandAlberto Garcia3-0/+997
This patch adds several tests for the x-blockdev-reopen QMP command. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12file-posix: Make auto-read-only dynamicKevin Wolf1-6/+6
Until now, with auto-read-only=on we tried to open the file read-write first and if that failed, read-only was tried. This is actually not good enough for libvirt, which gives QEMU SELinux permissions for read-write only as soon as it actually intends to write to the image. So we need to be able to switch between read-only and read-write at runtime. This patch makes auto-read-only dynamic, i.e. the file is opened read-only as long as no user of the node has requested write permissions, but it is automatically reopened read-write as soon as the first writer is attached. Conversely, if the last writer goes away, the file is reopened read-only again. bs->read_only is no longer set for auto-read-only=on files even if the file descriptor is opened read-only because it will be transparently upgraded as soon as a writer is attached. This changes the output of qemu-iotests 232. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12file-posix: Fix bdrv_open_flags() for snapshot=onKevin Wolf3-0/+25
Using a different read-only setting for bs->open_flags than for the flags to the driver's open function is just inconsistent and a bad idea. After this patch, the temporary snapshot keeps being opened read-only if read-only=on,snapshot=on is passed. If we wanted to change this behaviour to make only the orginal image file read-only, but the temporary overlay read-write (as the comment in the removed code suggests), that change would have to be made in bdrv_temp_snapshot_options() (where the comment suggests otherwise). Addressing this inconsistency before introducing dynamic auto-read-only is important because otherwise we would immediately try to reopen the temporary overlay even though the file is already unlinked. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12qemu-iotests: commit to backing file with auto-read-onlyKevin Wolf2-0/+51
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-12tests/qemu-iotests: add bitmap resize test 246John Snow3-0/+410
Test that we can actually resize qcow2 images with persistent bitmaps correctly. Throw some other goofy stuff at the test while we're at it, like adding bitmaps of different granularities and at different times. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Tested-by: Eric Blake <eblake@redhat.com> Message-id: 20190311185147.52309-5-vsementsov@virtuozzo.com [vsmentsov: drop \n from the end of test output, test output changed a bit: some bitmaps goes in other order int the output] Signed-off-by: John Snow <jsnow@redhat.com>
2019-03-12iotests: add busy/recording bit test to 124John Snow2-2/+115
This adds a simple test that ensures the busy bit works for push backups, as well as doubling as bonus test for incremental backups that get interrupted by EIO errors. Recording bit tests are already handled sufficiently by 236. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Message-id: 20190223000614.13894-11-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2019-03-12block/dirty-bitmap: add recording and busy propertiesJohn Snow1-0/+28
The current API allows us to report a single status, which we've defined as: Frozen: has a successor, treated as qmp_locked, may or may not be enabled. Locked: no successor, qmp_locked. may or may not be enabled. Disabled: Not frozen or locked, disabled. Active: Not frozen, locked, or disabled. The problem is that both "Frozen" and "Locked" mean nearly the same thing, and that both of them do not intuit whether they are recording guest writes or not. This patch deprecates that status field and introduces two orthogonal properties instead to replace it. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20190223000614.13894-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2019-03-09Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-03-08' into ↵Peter Maydell4-3/+42
staging nbd patches for 2019-03-08 - support TLS client authorization in NBD servers - iotest 223 race fix # gpg: Signature made Fri 08 Mar 2019 17:37:59 GMT # gpg: using RSA key A7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full] # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full] # gpg: aka "[jpeg image of size 6874]" [full] # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-nbd-2019-03-08: iotests: Wait for qemu to end in 223 nbd: fix outdated qapi docs syntax for tls-creds nbd: allow authorization with nbd-server-start QMP command qemu-nbd: add support for authorization of TLS clients Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-08qemu-iotests: amend with external data fileKevin Wolf2-1/+133
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08qemu-iotests: General tests for qcow2 with external data fileKevin Wolf3-0/+326
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08qemu-iotests: Preallocation with external data fileKevin Wolf2-0/+54
Test that preallocating metadata results in a somewhat larger qcow2 file, but preallocating data only affects the disk usage of the data file and the qcow2 file stays small. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08qcow2: Implement data-file-raw create optionKevin Wolf1-0/+27
Provide an option to force QEMU to always keep the external data file consistent as a standalone read-only raw image. At the moment, this means making sure that write_zeroes requests are forwarded to the data file instead of just updating the metadata, and checking that no backing file is used. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08qcow2: Store data file name in the imageKevin Wolf1-0/+27
Rather than requiring that the external data file node is passed explicitly when creating the qcow2 node, store the filename in the designated header extension during .bdrv_create and read it from there as a default during .bdrv_open. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset()Kevin Wolf1-1/+1
qcow2_alloc_compressed_cluster_offset() used to return the cluster offset for success and 0 for error. This doesn't only conflict with 0 as a valid host offset, but also loses the error code. Similar to the change made to qcow2_alloc_cluster_offset() for uncompressed clusters in commit 148da7ea9d6, make the function return 0/-errno and return the allocated cluster offset in a by-reference parameter. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08qcow2: Basic definitions for external data filesKevin Wolf3-13/+13
This adds basic constants, struct fields and helper function for external data file support to the implementation. QCOW2_INCOMPAT_MASK and QCOW2_AUTOCLEAR_MASK are not updated yet so that opening images with an external data file still fails (we don't handle them correctly yet). Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08qemu-iotests: Test qcow2 preallocation modesKevin Wolf3-0/+90
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-08qemu-iotests: Ensure GNU sed is usedPhilippe Mathieu-Daudé2-18/+31
Various sed regexp from common.filter use sed GNU extensions. Instead of spending time to write these regex to be POSIX compliant, verify the GNU sed is available and use it. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08qemu-iotests: Improve portability by searching bash in the $PATHPhilippe Mathieu-Daudé185-185/+185
Bash is not always installed as /bin/bash. In particular on OpenBSD, the package installs it in /usr/local/bin. Use the 'env' shebang to search bash in the $PATH. Patch created mechanically by running: $ git grep -lE '#! ?/bin/bash' -- tests/qemu-iotests \ | while read f; do \ sed -i 's|^#!.\?/bin/bash$|#!/usr/bin/env bash|' $f; \ done Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08iotests: check whitelisted formatsAndrey Shinkevich1-0/+3
Some test cases require specific formats. The method decorator skip_if_unsupported() checks if requested formats are whitelisted. The test #139 was selected for a sample output, after running $ ./check -qcow2 131-140 137 3s ... 138 0s ... 139 2s ... [case not run] testBlkDebug (__main__.TestBlockdevDel): formats ['blkdebug'] are not whitelisted [case not run] testBlkVerify (__main__.TestBlockdevDel): formats ['blkverify'] are not whitelisted [case not run] testQuorum (__main__.TestBlockdevDel): formats ['quorum'] are not whitelisted 140 0s ... Not run: 131 135 136 Some cases not run in: 139 Passed all 7 tests Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08iotests: ask QEMU for supported formatsAndrey Shinkevich2-1/+55
Supported formats listed by 'qemu' may differ from those listed by 'qemu-img' due to whitelists. Some test cases require specific formats that may be used with qemu. They can be inquired directly by running 'qemu -drive format=help'. The response takes whitelists into account. The method supported_formats() serves for that. The method decorator skip_if_unsupported() checks if all requested formats are whitelisted. If not, the test case will be skipped. That has been implemented in the 'check' file in the way similar to the 'test notrun' mechanism. Suggested-by: Roman Kagan <rkagan@virtuozzo.com> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08iotests: open notrun files in text modeAndrey Shinkevich1-1/+1
Replace the binary mode with the default text one when *.notrun files are opened for skipped tests. That change is made for the compatibility with Python 3 which returns error otherwise. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08iotests: use iotests.VM in 238Stefan Hajnoczi1-6/+1
Test 238 does not require the kvm accelerator. Using the qtest accelerator allows the test to run in both non-kvm and non-tcg environments. iotests.VM implicitly uses the qtest accelerator and is really the class that this test should be using. Switch to that instead of qemu.QEMUMachine. Suggested-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-07Merge remote-tracking branch 'remotes/cleber/tags/python-next-pull-request' ↵Peter Maydell3-4/+4
into staging Python queue, 2019-02-22 Python: * introduce "python" directory with module namespace * log QEMU launch command line on qemu.QEMUMachine Acceptance Tests: * initrd 4GiB+ test * migration test * multi vm support in test class * bump Avocado version and drop ":avocado: enable" # gpg: Signature made Fri 22 Feb 2019 19:37:07 GMT # gpg: using RSA key 657E8D33A5F209F3 # gpg: Good signature from "Cleber Rosa <crosa@redhat.com>" [marginal] # gpg: WARNING: This key is not certified with sufficiently trusted signatures! # gpg: It is not certain that the signature belongs to the owner. # Primary key fingerprint: 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 * remotes/cleber/tags/python-next-pull-request: Acceptance tests: expect boot to extract 2GiB+ initrd with linux-v4.16 Acceptance tests: use linux-3.6 and set vm memory to 4GiB tests.acceptance: adds simple migration test tests.acceptance: adds multi vm capability for acceptance tests scripts/qemu.py: log QEMU launch command line Introduce a Python module structure Acceptance tests: drop usage of ":avocado: enable" Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-06iotests: Wait for qemu to end in 223Eric Blake2-0/+2
When iotest 223 was first written, it didn't matter if we waited for the qemu process to clean up. But with the introduction of a later qemu-nbd process trying to reuse the same file, there is a race where even though the asynchronous qemu process has responded to "quit", it has not yet had time to unlock the file and exit, resulting in: -[{ "start": 0, "length": 65536, "depth": 0, "zero": false, "data": false}, -{ "start": 65536, "length": 2031616, "depth": 0, "zero": false, "data": true}, -{ "start": 2097152, "length": 2097152, "depth": 0, "zero": false, "data": false}] +qemu-nbd: Failed to blk_new_open 'tests/qemu-iotests/scratch/t.qcow2': Failed to get shared "write" lock +Is another process using the image [tests/qemu-iotests/scratch/t.qcow2]? +qemu-img: Could not open 'driver=nbd,server.type=unix,server.path=tests/qemu-iotests/scratch/qemu-nbd.sock,x-dirty-bitmap=qemu:dirty-bitmap:b': Failed to connect socket tests/qemu-iotests/scratch/qemu-nbd.sock: Connection refused +./common.nbd: line 33: kill: (11122) - No such process Fixes: ddd09448 Reported-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190305182908.13557-1-eblake@redhat.com> Tested-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2019-03-06qemu-nbd: add support for authorization of TLS clientsDaniel P. Berrange2-3/+40
Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-28Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-02-25-v2' ↵Peter Maydell3-19/+40
into staging nbd patches for 2019-02-25 - iotest failure fixes for tests related to NBD # gpg: Signature made Tue 26 Feb 2019 16:46:15 GMT # gpg: using RSA key A7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full] # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full] # gpg: aka "[jpeg image of size 6874]" [full] # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-nbd-2019-02-25-v2: iotests: avoid broken pipe with certtool iotests: ensure we print nbd server log on error iotests: handle TypeError for Python 3 in test 242 Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-02-26iotests: avoid broken pipe with certtoolDaniel P. Berrangé1-16/+32
When we run "certtool 2>&1 | head -1" the latter command is likely to complete and exit before certtool has written everything it wants to stderr. In at least the RHEL-7 gnutls 3.3.29 this causes certtool to quit with broken pipe before it has finished writing the desired output file to disk. This causes non-deterministic failures of the iotest 233 because the certs are sometimes zero length files. If certtool fails the "head -1" means we also lose any useful error message it would have printed. Thus this patch gets rid of the pipe and post-processes the output in a more flexible & reliable manner. Reported-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20190220145819.30969-3-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-26iotests: ensure we print nbd server log on errorDaniel P. Berrangé1-0/+3
If we abort the iotest early the server.log file might contain useful information for diagnosing the problem. Ensure its contents are displayed in this case. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20190220145819.30969-2-berrange@redhat.com> [eblake: fix shell quoting] Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-26iotests: handle TypeError for Python 3 in test 242Andrey Shinkevich1-3/+5
The data type for bytes in Python 3 differs from the one in Python 2. The type cast that is compatible with both versions was applied. Signed-off-by: Nir Soffer <nsoffer@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reported-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <1551197495-24425-1-git-send-email-andrey.shinkevich@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2019-02-25iotests: Skip 211 on insufficient memoryMax Reitz2-2/+7
VDI keeps the whole bitmap in memory, and the maximum size (which is tested here) is 2 GB. This may not be available on all machines, and it rarely is available when running a 32 bit build. Fix this by making VM.run_job() return the error string if an error occurred, and checking whether that contains "Could not allocate bmap" in 211. If so, the test is skipped. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190218180646.30282-1-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25iotests: add LUKS payload overhead to 178 qemu-img measure testStefan Hajnoczi2-0/+32
The previous patch includes the LUKS payload overhead into the qemu-img measure calculation for qcow2. Update qemu-iotests 178 to exercise this new code path. Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20190218104525.23674-3-stefanha@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25iotests.py: s/_/-/g on keys in qmp_log()Max Reitz8-138/+140
This follows what qmp() does, so the output will correspond to the actual QMP command. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190210145736.1486-11-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25iotests: Let 045 be run concurrentlyMax Reitz1-1/+1
Adding a telnet monitor for no real purpose on a fixed port is not so great. Just use a null monitor instead. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20190210145736.1486-10-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25iotests: Filter SSH pathsMax Reitz1-0/+1
8908b253c4ad5f8874c8d13abec169c696a5cd32 has implemented filtering of remote paths for NFS, but forgot SSH. This patch takes care of that. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20190210145736.1486-9-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25iotests.py: Filter filename in any string valueMax Reitz1-1/+1
filter_qmp_testfiles() currently filters the filename only for specific keys. However, there are more keys that take filenames (such as block-commit's @top and @base, or ssh's @path), and it does not make sense to list them all here. "$TEST_DIR/$PID-" should have enough entropy not to appear anywhere randomly. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20190210145736.1486-8-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25iotests.py: Add is_str()Max Reitz1-0/+6
On Python 2.x, strings are not always unicode strings. This function checks whether a given value is a plain string, or a unicode string (if there is a difference). Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20190210145736.1486-7-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25iotests: Fix 207 to use QMP filters for qmp_logMax Reitz2-5/+9
Fixes: 08fcd6111e1949f456e1b232ebeeb0cc17019a92 Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20190210145736.1486-6-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25iotests: Fix 232 for LUKSMax Reitz1-0/+4
With IMGOPTSSYNTAX, $TEST_IMG is useless for this test (it only tests the file-posix protocol driver). Therefore, if $TEST_IMG_FILE is set, use that instead. Because this test requires the file protocol, $TEST_IMG_FILE will always be set if $IMGOPTSSYNTAX is true. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20190210145736.1486-5-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>