aboutsummaryrefslogtreecommitdiff
path: root/tests/qemu-iotests
AgeCommit message (Collapse)AuthorFilesLines
2020-09-30meson: clean up build_by_defaultPaolo Bonzini1-2/+1
Build all executables by default except for the known-broken ones. This also allows running qemu-iotests without manually building socket_scm_helper. Reported-by: Max Reitz <mreitz@redhat.com> Tested-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-09-29tests/acpi: drop unnecessary filesMichael S. Tsirkin1-0/+0
The last pull added a ton of useless files by mistake. Drop them all. Fixes: 0ed93f4c05896 ("update golden master DSDT binary table blobs for q35") Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-09-29tests/acpi: update golden master DSDT binary table blobs for q35Ani Sinha1-0/+0
In the previously applied commit ("piix4: don't reserve hw resources when hotplug is off globally"), we make changes to the ACPI DSDT tables such that some ACPI code are not generated when bsel is absent. Since as of this point in time, in q35 machines, we do not use bsel for pci buses, we need to update the DSDT table blobs. This patch updates the DSDT golden master tables for q35 machines. At the same time, we clear bios-tables-test-allowed-diff.h for future changes which update tables. Following is a typical diff between the q35 acpi DSDT table blobs: @@ -1,30 +1,30 @@ /* * Intel ACPI Component Architecture * AML/ASL+ Disassembler version 20180105 (64-bit version) * Copyright (c) 2000 - 2018 Intel Corporation * * Disassembling to symbolic ASL+ operators * - * Disassembly of tests/data/acpi/q35/DSDT, Tue Sep 15 18:52:47 2020 + * Disassembly of /tmp/aml-3O0DR0, Tue Sep 15 18:52:47 2020 * * Original Table Header: * Signature "DSDT" - * Length 0x00001DFE (7678) + * Length 0x00001DF6 (7670) * Revision 0x01 **** 32-bit table (V1), no 64-bit math support - * Checksum 0xAC + * Checksum 0x17 * OEM ID "BOCHS " * OEM Table ID "BXPCDSDT" * OEM Revision 0x00000001 (1) * Compiler ID "BXPC" * Compiler Version 0x00000001 (1) */ DefinitionBlock ("", "DSDT", 1, "BOCHS ", "BXPCDSDT", 0x00000001) { Scope (\) { OperationRegion (DBG, SystemIO, 0x0402, One) Field (DBG, ByteAcc, NoLock, Preserve) { DBGB, 8 } @@ -3113,24 +3113,20 @@ Name (_ADR, 0x00010000) // _ADR: Address Method (_S1D, 0, NotSerialized) // _S1D: S1 Device State { Return (Zero) } Method (_S2D, 0, NotSerialized) // _S2D: S2 Device State { Return (Zero) } Method (_S3D, 0, NotSerialized) // _S3D: S3 Device State { Return (Zero) } } - - Method (PCNT, 0, NotSerialized) - { - } } } } Signed-off-by: Ani Sinha <ani@anisinha.ca> Acked-by: Igor Mammedov <imammedo@redhat.com> Message-Id: <20200918084111.15339-12-ani@anisinha.ca> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-09-21qemu-img: Support bitmap --merge into backing imageEric Blake2-0/+68
If you have the chain 'base.qcow2 <- top.qcow2' and want to merge a bitmap from top into base, qemu-img was failing with: qemu-img: Could not open 'top.qcow2': Could not open backing file: Failed to get shared "write" lock Is another process using the image [base.qcow2]? The easiest fix is to not open the entire backing chain of either image (source or destination); after all, the point of 'qemu-img bitmap' is solely to manipulate bitmaps directly within a single qcow2 image, and this is made more precise if we don't pay attention to other images in the chain that may happen to have a bitmap by the same name. However, note that on a case-by-case analysis, there _are_ times where we treat it as a feature that we can access a bitmap from a backing layer in association with an overlay BDS. A demonstration of this is using NBD to expose both an overlay BDS (for constant contents) and a bitmap (for learning which blocks are interesting) during an incremental backup: Base <- Active <- Temporary \--block job ->/ where Temporary is being fed by a backup 'sync=none' job. When exposing Temporary over NBD, referring to a bitmap that lives only in Active is less effort than having to copy a bitmap into Temporary [1]. So the testsuite additions in this patch check both where bitmaps get allocated (the qemu-img info output), and that qemu-nbd is indeed able to access a bitmap inherited from the backing chain since it is a different use case than 'qemu-img bitmap'. [1] Full disclosure: prior to the recent commit 374eedd1c4 and friends, we were NOT able to see bitmaps through filters, which meant that we actually did not have nice clean semantics for uniformly being able to pick up bitmaps from anywhere in the backing chain (seen as a change in behavior between qemu 4.1 and 4.2 at commit 00e30f05de, when block-copy swapped from a one-off to a filter). Which means libvirt was already coded to copy bitmaps around for the sake of older qemu, even though modern qemu no longer needs it. Oh well. Fixes: http://bugzilla.redhat.com/1877209 Reported-by: Eyal Shenitzky <eshenitz@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200914191009.644842-1-eblake@redhat.com> [eblake: more commit message tweaks, per Max Reitz review] Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-09-16iotests: Drop readlink -fMax Reitz1-1/+1
On macOS, (out of the box) readlink does not have -f. We do not really need readlink here, though, it was just a replacement for realpath (which is not available on our BSD test systems), which we needed to make the $(dirname) into an absolute path. Instead of using either, just use "cd; pwd" like is done for $source_iotests. Fixes: b1cbc33a3971b6bb005d5ac3569feae35a71de0f ("iotests: Allow running from different directory") Suggested-by: Peter Maydell <peter.maydell@linaro.org> Reported-by: Claudio Fontana <cfontana@suse.de> Reported-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200914145606.94620-1-mreitz@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-15qcow2: Make preallocate_co() resize the image to the correct sizeAlberto Garcia2-20/+52
This function preallocates metadata structures and then extends the image to its new size, but that new size calculation is wrong because it doesn't take into account that the host_offset variable is always cluster-aligned. This problem can be reproduced with preallocation=metadata when the original size is not cluster-aligned but the new size is. In this case the final image size will be shorter than expected. qemu-img create -f qcow2 img.qcow2 31k qemu-img resize --preallocation=metadata img.qcow2 128k Signed-off-by: Alberto Garcia <berto@igalia.com> Message-Id: <adeb8b059917b141d5f5b3bd2a016262d3052c79.1599833007.git.berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> [mreitz: Mark compat=0.10 unsupported for iotest 125] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-15iotests: Skip test_stream_parallel in test 030 when doing "make check"Thomas Huth1-0/+2
The test_stream_parallel test still occasionally fails in the CI. Thus let's disable it during "make check" for now so that it does not cause trouble during merge tests. We can enable it again once the problem has been resolved. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200907113824.134788-1-thuth@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-15qcow2: Fix removal of list members from BDRVQcow2State.cluster_allocsAlberto Garcia3-0/+91
When a write request needs to allocate new clusters (or change the L2 bitmap of existing ones) a QCowL2Meta structure is created so the L2 metadata can be later updated and any copy-on-write can be performed if necessary. A write request can span a region consisting of an arbitrary combination of previously unallocated and allocated clusters, and if the unallocated ones can be put contiguous to the existing ones then QEMU will do so in order to minimize the number of write operations. In practice this means that a write request has not just one but a number of QCowL2Meta structures. All of them are added to the cluster_allocs list that is stored in BDRVQcow2State and is used to detect overlapping requests. After the write request finishes all its associated QCowL2Meta are removed from that list. calculate_l2_meta() takes care of creating and putting those structures in the list, and qcow2_handle_l2meta() takes care of removing them. The problem is that the error path in handle_alloc() also tries to remove an item in that list, a remnant from the time when this was handled there (that code would not even be correct anymore because it only removes one struct and not all the ones from the same write request). This can trigger a double removal of the same item from the list, causing a crash. This is not easy to reproduce in practice because it requires that do_alloc_cluster_offset() fails after a successful previous allocation during the same write request, but it can be reproduced with the included test case. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-Id: <3440a1c4d53c4fe48312b478c96accb338cbef7c.1599150873.git.berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-15qemu-img: avoid unaligned read requests during convertPeter Lieven1-2/+5
in case of large continous areas that share the same allocation status it happens that the value of s->sector_next_status is unaligned to the cluster size or even request alignment of the source. Avoid this by stripping down the s->sector_next_status position to cluster boundaries. Signed-off-by: Peter Lieven <pl@kamp.de> Message-Id: <20200901125129.6398-1-pl@kamp.de> [mreitz: Disable vhdx for 251] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-15qemu-iotests: Simplify FilePath __init__Nir Soffer1-3/+2
Use list comprehension instead of append loop. Signed-off-by: Nir Soffer <nsoffer@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200828232152.205833-6-nsoffer@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-15qemu-iotests: Merge FilePaths and FilePathNir Soffer5-18/+15
FilePath creates now one temporary file: with FilePath("a") as a: Or more: with FilePath("a", "b", "c") as (a, b, c): This is also the behavior of the file_path() helper, used by some of the tests. Now we have only 2 helpers for creating temporary files instead of 3. Signed-off-by: Nir Soffer <nsoffer@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200828232152.205833-5-nsoffer@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-15qemu-iotests: Support varargs syntax in FilePathsNir Soffer3-12/+10
Accept variable number of names instead of a sequence: with FilePaths("a", "b", "c") as (a, b, c): The disadvantage is that base_dir must be used as kwarg: with FilePaths("a", "b", base_dir=soc_dir) as (sock1, sock2): But this is more clear and calling optional argument as positional arguments is bad idea anyway. Signed-off-by: Nir Soffer <nsoffer@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200828232152.205833-4-nsoffer@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-15qemu-iotests: Fix FilePaths docstringNir Soffer1-6/+13
When this class was extracted from FilePath, the docstring was not updated for generating multiple files, and the example usage was referencing unrelated file. While fixing the docstring, add example for creating sockets, which should use iotests.sock_dir instead of the default base_dir. Fixes: de263986b5dc Signed-off-by: Nir Soffer <nsoffer@redhat.com> Message-Id: <20200828232152.205833-3-nsoffer@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-15qemu-iotests: Fix FilePaths cleanupNir Soffer1-4/+4
If os.remove() fails to remove one of the paths, for example if the file was removed by the test, the cleanup loop would exit silently, without removing the rest of the files. Fixes: de263986b5dc Signed-off-by: Nir Soffer <nsoffer@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200828232152.205833-2-nsoffer@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-11Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into stagingPeter Maydell20-24/+539
Block layer patches: - qemu-img create: Fail gracefully when backing file is an empty string - Fixes related to filter block nodes ("Deal with filters" series) - block/nvme: Various cleanups required to use multiple queues - block/nvme: Use NvmeBar structure from "block/nvme.h" - file-win32: Fix "locking" option - iotests: Allow running from different directory # gpg: Signature made Thu 10 Sep 2020 10:11:19 BST # gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6 # gpg: issuer "kwolf@redhat.com" # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kevin/tags/for-upstream: (65 commits) block/qcow2-cluster: Add missing "fallthrough" annotation block/nvme: Pair doorbell registers block/nvme: Use generic NvmeBar structure block/nvme: Group controller registers in NVMeRegs structure file-win32: Fix "locking" option iotests: Allow running from different directory iotests: Test committing to overridden backing iotests: Add test for commit in sub directory iotests: Add filter mirror test cases iotests: Add filter commit test cases iotests: Let complete_and_wait() work with commit iotests: Test that qcow2's data-file is flushed block: Leave BDS.backing_{file,format} constant block: Inline bdrv_co_block_status_from_*() blockdev: Fix active commit choice block: Drop backing_bs() qemu-img: Use child access functions nbd: Use CAF when looking for dirty bitmap commit: Deal with filters backup: Deal with filters ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-09-10iotests: Allow running from different directoryKevin Wolf1-1/+1
It is convenient to be able to edit the tests and run them without changing the current working directory back and forth. Instead of assuming that $PWD is the qemu-iotests build directory, derive the build directory from the executed script. This allows 'check' to find the required files even when called from another directory. The scratch directory will still be in the current working directory. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200902110326.257115-1-kwolf@redhat.com> Reviewed-by: Claudio Fontana <cfontana@suse.de> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-09-08iotests: Test committing to overridden backingMax Reitz2-2/+63
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-08iotests: Add test for commit in sub directoryMax Reitz2-0/+54
Add a test for committing an overlay in a sub directory to one of the images in its backing chain, using both relative and absolute filenames. Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-08iotests: Add filter mirror test casesMax Reitz2-3/+147
This patch adds some test cases how mirroring relates to filters. One of them tests what happens when you mirror off a filtered COW node, two others use the mirror filter node as basically our only example of an implicitly created filter node so far (besides the commit filter). Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-08iotests: Add filter commit test casesMax Reitz2-2/+179
This patch adds some tests on how commit copes with filter nodes. Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-08qemu-iotests: move check-block back to MakefilesPaolo Bonzini1-4/+0
check-block has its own test harness, unlike every other test. If we capture its output, as is in general nicer to do without V=1, there will be no sign of progress. So for lack of a better option just move the invocation of the test back to Makefile rules. As a side effect, this will also fix "make check" in --disable-tools builds, as they were trying to run qemu-iotests without having made qemu-img before. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-09-07iotests: Let complete_and_wait() work with commitMax Reitz1-3/+7
complete_and_wait() and wait_ready() currently only work for mirror jobs. Let them work for active commit jobs, too. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07iotests: Test that qcow2's data-file is flushedMax Reitz2-0/+56
Flushing a qcow2 node must lead to the data-file node being flushed as well. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07block: Leave BDS.backing_{file,format} constantMax Reitz4-9/+11
Parts of the block layer treat BDS.backing_file as if it were whatever the image header says (i.e., if it is a relative path, it is relative to the overlay), other parts treat it like a cache for bs->backing->bs->filename (relative paths are relative to the CWD). Considering bs->backing->bs->filename exists, let us make it mean the former. Among other things, this now allows the user to specify a base when using qemu-img to commit an image file in a directory that is not the CWD (assuming, everything uses relative filenames). Before this patch: $ ./qemu-img create -f qcow2 foo/bot.qcow2 1M $ ./qemu-img create -f qcow2 -b bot.qcow2 foo/mid.qcow2 $ ./qemu-img create -f qcow2 -b mid.qcow2 foo/top.qcow2 $ ./qemu-img commit -b mid.qcow2 foo/top.qcow2 qemu-img: Did not find 'mid.qcow2' in the backing chain of 'foo/top.qcow2' $ ./qemu-img commit -b foo/mid.qcow2 foo/top.qcow2 qemu-img: Did not find 'foo/mid.qcow2' in the backing chain of 'foo/top.qcow2' $ ./qemu-img commit -b $PWD/foo/mid.qcow2 foo/top.qcow2 qemu-img: Did not find '[...]/foo/mid.qcow2' in the backing chain of 'foo/top.qcow2' After this patch: $ ./qemu-img commit -b mid.qcow2 foo/top.qcow2 Image committed. $ ./qemu-img commit -b foo/mid.qcow2 foo/top.qcow2 qemu-img: Did not find 'foo/mid.qcow2' in the backing chain of 'foo/top.qcow2' $ ./qemu-img commit -b $PWD/foo/mid.qcow2 foo/top.qcow2 Image committed. With this change, bdrv_find_backing_image() must look at whether the user has overridden a BDS's backing file. If so, it can no longer use bs->backing_file, but must instead compare the given filename against the backing node's filename directly. Note that this changes the QAPI output for a node's backing_file. We had very inconsistent output there (sometimes what the image header said, sometimes the actual filename of the backing image). This inconsistent output was effectively useless, so we have to decide one way or the other. Considering that bs->backing_file usually at runtime contained the path to the image relative to qemu's CWD (or absolute), this patch changes QAPI's backing_file to always report the bs->backing->bs->filename from now on. If you want to receive the image header information, you have to refer to full-backing-filename. This necessitates a change to iotest 228. The interesting information it really wanted is the image header, and it can get that now, but it has to use full-backing-filename instead of backing_file. Because of this patch's changes to bs->backing_file's behavior, we also need some reference output changes. Along with the changes to bs->backing_file, stop updating BDS.backing_format in bdrv_backing_attach() as well. This way, ImageInfo's backing-filename and backing-filename-format fields will represent what the image header says and nothing else. iotest 245 changes in behavior: With the backing node no longer overriding the parent node's backing_file string, you can now omit the @backing option when reopening a node with neither a default nor a current backing file even if it used to have a backing node at some point. 273 also changes: The base image is opened without a format layer, so ImageInfo.backing-filename-format used to report "file" for the base image's overlay after blockdev-snapshot. However, the image header never says "file" anywhere, so it now reports $IMGFMT. Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07qemu-img: Use child access functionsMax Reitz1-0/+1
This changes iotest 204's output, because blkdebug on top of a COW node used to make qemu-img map disregard the rest of the backing chain (the backing chain was broken by the filter). With this patch, the allocation in the base image is reported correctly. Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07block: Use child access functions for QAPI queriesMax Reitz1-1/+7
query-block, query-named-block-nodes, and query-blockstats now return any filtered child under "backing", not just bs->backing or COW children. This is so that filters do not interrupt the reported backing chain. This changes the output for iotest 184, as the throttled node now appears as a backing child. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07block/null: Implement bdrv_get_allocated_file_sizeMax Reitz3-5/+6
It is trivial, so we might as well do it. Remove _filter_actual_image_size from iotest 184, so we get to see the result in its reference output. Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07block: Raise an error when backing file parameter is an empty stringConnor Kuehl2-0/+9
Providing an empty string for the backing file parameter like so: qemu-img create -f qcow2 -b '' /tmp/foo allows the flow of control to reach and subsequently fail an assert statement because passing an empty string to bdrv_get_full_backing_filename_from_filename() simply results in NULL being returned without an error being raised. To fix this, let's check for an empty string when getting the value from the opts list. Reported-by: Attila Fazekas <afazekas@redhat.com> Fixes: https://bugzilla.redhat.com/1809553 Signed-off-by: Connor Kuehl <ckuehl@redhat.com> Message-Id: <20200813134722.802180-1-ckuehl@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-09-02iotests/259: Fix reference outputMax Reitz1-1/+1
The error message has changed recently, breaking the test. Fix it. Fixes: a2b333c01880f56056d50c238834d62e32001e54 ("block: nbd: Fix convert qcow2 compressed to nbd") Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200811080830.289136-1-mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-09-02iotests/059: Fix reference outputMax Reitz1-1/+1
As of the patch to flush qemu-img's "Formatting" message before the error message, 059 has been broken for vmdk. Fix it. Fixes: 4e2f4418784da09cb106264340241856cd2846df ("qemu-img: Flush stdout before before potential stderr messages") Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200811084150.326377-1-mreitz@redhat.com> Reviewed-by: Eric blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-28migration: improve error reporting of block driver state nameDaniel P. Berrangé1-2/+2
With blockdev, a BlockDriverState may not have a device name, so using a node name is required as an alternative. Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200827111606.1408275-2-berrange@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-08-26iotests: Add tests for qcow2 images with extended L2 entriesAlberto Garcia3-0/+1628
Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <e6dd0429cafe84ca603179c298a8703bddca2904.1594396418.git.berto@igalia.com> [mreitz: Use env in shebang line] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-08-25qcow2: Assert that expand_zero_clusters_in_l1() does not support subclustersAlberto Garcia2-0/+11
This function is only used by qcow2_expand_zero_clusters() to downgrade a qcow2 image to a previous version. This would require transforming all extended L2 entries into normal L2 entries but this is not a simple task and there are no plans to implement this at the moment. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <15e65112b4144381b4d8c0bdf8fb76b0d813e3d1.1594396418.git.berto@igalia.com> [mreitz: Fixed comment style] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-08-25qcow2: Allow preallocation and backing files if extended_l2 is setAlberto Garcia1-1/+1
Traditional qcow2 images don't allow preallocation if a backing file is set. This is because once a cluster is allocated there is no way to tell that its data should be read from the backing file. Extended L2 entries have individual allocation bits for each subcluster, and therefore it is perfectly possible to have an allocated cluster with all its subclusters unallocated. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <6d5b0f38e7dc5f2f31d8cab1cb92044e9909aece.1594396418.git.berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-08-25qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bitAlberto Garcia21-133/+183
Now that the implementation of subclusters is complete we can finally add the necessary options to create and read images with this feature, which we call "extended L2 entries". Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <6476caaa73216bd05b7bb2d504a20415e1665176.1594396418.git.berto@igalia.com> [mreitz: %s/5\.1/5.2/; fixed 302's and 303's reference output] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-08-25qcow2: Add subcluster support to check_refcounts_l2()Alberto Garcia1-1/+1
The offset field of an uncompressed cluster's L2 entry must be aligned to the cluster size, otherwise it is invalid. If the cluster has no data then it means that the offset points to a preallocation, so we can clear the offset field without affecting the guest-visible data. This is what 'qemu-img check' does when run in repair mode. On traditional qcow2 images this can only happen when QCOW_OFLAG_ZERO is set, and repairing such entries turns the clusters from ZERO_ALLOC into ZERO_PLAIN. Extended L2 entries have no ZERO_ALLOC clusters and no QCOW_OFLAG_ZERO but the idea is the same: if none of the subclusters are allocated then we can clear the offset field and leave the bitmap untouched. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <9f4ed1d0a34b0a545b032c31ecd8c14734065342.1594396418.git.berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-08-21iotests: Test node/bitmap aliases during migrationMax Reitz3-0/+599
Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200820150725.68687-4-mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> [eblake: fold in python cleanups recommended by Vladimir] Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21iotests.py: Let wait_migration() return on failureMax Reitz1-6/+12
Let wait_migration() return on failure (with the return value indicating whether the migration was completed or has failed), so we can use it for migrations that are expected to fail, too. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20200820150725.68687-3-mreitz@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21iotests: dump QCOW2 header in JSON in #303Andrey Shinkevich2-0/+79
Extend the test case #303 by dumping QCOW2 image metadata in JSON format. Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-12-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21qcow2_format.py: support dumping metadata in JSON formatAndrey Shinkevich1-0/+17
Implementation of dumping QCOW2 image metadata. The sample output: { "Header_extensions": [ { "name": "Feature table", "magic": 1745090647, "length": 192, "data_str": "<binary>" }, { "name": "Bitmaps", "magic": 595929205, "length": 24, "data": { "nb_bitmaps": 2, "reserved32": 0, "bitmap_directory_size": 64, "bitmap_directory_offset": 1048576, "bitmap_directory": [ { "name": "bitmap-1", "bitmap_table_offset": 589824, "bitmap_table_size": 1, "flags": 2, "type": 1, "granularity_bits": 15, "name_size": 8, "extra_data_size": 0, "bitmap_table": [ { "type": "serialized", "offset": 655360 }, ... Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-11-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21qcow2_format.py: collect fields to dump in JSON formatAndrey Shinkevich1-0/+36
As __dict__ is being extended with class members we do not want to print, add the to_json() method to classes that returns a json-dumpable object with desired fields and their values. Extend it in subclass when necessary to print the final dictionary in the JSON output which follows. Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Message-Id: <1596742557-320265-10-git-send-email-andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21qcow2.py: Introduce '-j' key to dump in JSON formatAndrey Shinkevich2-6/+16
Add the command key to the qcow2.py arguments list to dump QCOW2 metadata in JSON format. Here is the suggested way to do that. The implementation of the dump in JSON format is in the patch that follows. Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-9-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21qcow2_format.py: Dump bitmap table serialized entriesAndrey Shinkevich2-0/+54
Add bitmap table information to the QCOW2 metadata dump. Bitmap name bitmap-1 ... Bitmap table type size offset 0 serialized 65536 10092544 1 all-zeroes 0 0 2 all-zeroes 0 0 Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-8-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21qcow2_format.py: pass cluster size to substructuresAndrey Shinkevich1-6/+11
The cluster size of an image is the QcowHeader class member and may be obtained by dependent extension structures such as Qcow2BitmapExt for further bitmap table details print. Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-7-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21qcow2_format.py: Dump bitmap directory informationAndrey Shinkevich2-0/+65
Read and dump entries from the bitmap directory of QCOW2 image. Header extension: magic 0x23852875 (Bitmaps) ... Bitmap name bitmap-1 bitmap_table_offset 0xf0000 bitmap_table_size 1 flags 0x2 (['auto']) type 1 granularity_bits 16 name_size 8 extra_data_size 0 Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-6-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21qcow2_format.py: dump bitmap flags in human readable way.Andrey Shinkevich1-0/+16
Introduce the class BitmapFlags that parses a bitmap flags mask. Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-5-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21qcow2_format.py: change Qcow2BitmapExt initialization methodAndrey Shinkevich1-14/+22
There are two ways to initialize a class derived from Qcow2Struct: 1. Pass a block of binary data to the constructor. 2. Pass the file descriptor to allow reading the file from constructor. Let's change the Qcow2BitmapExt initialization method from 1 to 2 to support a scattered reading in the initialization chain. The implementation comes with the patch that follows. Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-4-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21qcow2_format.py: make printable data an extension class memberAndrey Shinkevich1-6/+8
Let us differ binary data type from string one for the extension data variable and keep the string as the QcowHeaderExtension class member. Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-3-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21iotests: add test for QCOW2 header dumpAndrey Shinkevich3-0/+121
The simple script creates a QCOW2 image and fills it with some data. Two bitmaps are created as well. Then the script reads the image header with extensions from the disk by running the script qcow2.py and dumps the information to the output. Other entities, such as snapshots, may be added to the test later. Suggested-by: Eric Blake <eblake@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-2-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21meson: convert check-blockPaolo Bonzini1-0/+10
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
generated by cgit v1.1 at 2024-11-03 05:10:59 +0000