aboutsummaryrefslogtreecommitdiff
path: root/block
AgeCommit message (Collapse)AuthorFilesLines
2018-05-15block: Support BDRV_REQ_WRITE_UNCHANGED in filtersMax Reitz7-14/+28
Update the rest of the filter drivers to support BDRV_REQ_WRITE_UNCHANGED. They already forward write request flags to their children, so we just have to announce support for it. This patch does not cover the replication driver because that currently does not support flags at all, and because it just grabs the WRITE permission for its children when it can, so we should be fine just submitting the incoming WRITE_UNCHANGED requests as normal writes. It also does not cover format drivers for similar reasons. They all use bdrv_format_default_perms() as their .bdrv_child_perm() implementation so they just always grab the WRITE permission for their file children whenever possible. In addition, it often would be difficult to ascertain whether incoming unchanging writes end up as unchanging writes in their files. So we just leave them as normal potentially changing writes. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Message-id: 20180421132929.21610-7-mreitz@redhat.com Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-05-15block/quorum: Support BDRV_REQ_WRITE_UNCHANGEDMax Reitz1-6/+13
We just need to forward it to quorum's children (except in case of a rewrite because of corruption), but for that we first have to support flags in child requests at all. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Message-id: 20180421132929.21610-6-mreitz@redhat.com Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-05-15block: Set BDRV_REQ_WRITE_UNCHANGED for COR writesMax Reitz1-2/+4
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Message-id: 20180421132929.21610-5-mreitz@redhat.com Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-05-15block: Add BDRV_REQ_WRITE_UNCHANGED flagMax Reitz1-1/+5
This flag signifies that a write request will not change the visible disk content. With this flag set, it is sufficient to have the BLK_PERM_WRITE_UNCHANGED permission instead of BLK_PERM_WRITE. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Message-id: 20180421132929.21610-4-mreitz@redhat.com Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-05-15block: Add COR filter driverMax Reitz2-1/+172
This adds a simple copy-on-read filter driver. It relies on the already existing COR functionality in the central block layer code, which may be moved here once we no longer need it there. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20180421132929.21610-2-mreitz@redhat.com Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-05-15qcow2: Give the refcount cache the minimum possible size by defaultAlberto Garcia2-16/+19
The L2 and refcount caches have default sizes that can be overridden using the l2-cache-size and refcount-cache-size (an additional parameter named cache-size sets the combined size of both caches). Unless forced by one of the aforementioned parameters, QEMU will set the unspecified sizes so that the L2 cache is 4 times larger than the refcount cache. This is based on the premise that the refcount metadata needs to be only a fourth of the L2 metadata to cover the same amount of disk space. This is incorrect for two reasons: a) The amount of disk covered by an L2 table depends solely on the cluster size, but in the case of a refcount block it depends on the cluster size *and* the width of each refcount entry. The 4/1 ratio is only valid with 16-bit entries (the default). b) When we talk about disk space and L2 tables we are talking about guest space (L2 tables map guest clusters to host clusters), whereas refcount blocks are used for host clusters (including L1/L2 tables and the refcount blocks themselves). On a fully populated (and uncompressed) qcow2 file, image size > virtual size so there are more refcount entries than L2 entries. Problem (a) could be fixed by adjusting the algorithm to take into account the refcount entry width. Problem (b) could be fixed by increasing a bit the refcount cache size to account for the clusters used for qcow2 metadata. However this patch takes a completely different approach and instead of keeping a ratio between both cache sizes it assigns as much as possible to the L2 cache and the remainder to the refcount cache. The reason is that L2 tables are used for every single I/O request from the guest and the effect of increasing the cache is significant and clearly measurable. Refcount blocks are however only used for cluster allocation and internal snapshots and in practice are accessed sequentially in most cases, so the effect of increasing the cache is negligible (even when doing random writes from the guest). So, make the refcount cache as small as possible unless the user explicitly asks for a larger one. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 9695182c2eb11b77cb319689a1ebaa4e7c9d6591.1523968389.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-05-15Fix error message about compressed clusters with OFLAG_COPIEDAlberto Garcia1-2/+2
Compressed clusters are not supposed to have the COPIED bit set. "qemu-img check" detects that and prints an error message reporting the number of the affected host cluster. This doesn't make much sense because compressed clusters are not aligned to host clusters, so it would be better to report the offset instead. Plus, the calculation is wrong and it uses the raw L2 entry as if it was simply an offset. This patch fixes the error message and reports the offset of the compressed cluster. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: 0f687957feb72e80c740403191a47e607c2463fe.1523376013.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-05-15blockjob: Add block_job_driver()Kevin Wolf1-3/+5
The backup block job directly accesses the driver field in BlockJob. Add a wrapper for getting it. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
2018-05-15blockjob: Introduce block_job_ratelimit_get_delay()Kevin Wolf4-19/+12
This gets us rid of more direct accesses to BlockJob fields from the job drivers. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
2018-05-15blockjob: Implement block_job_set_speed() centrallyKevin Wolf4-61/+6
All block job drivers support .set_speed and all of them duplicate the same code to implement it. Move that code to blockjob.c and remove the now useless callback. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
2018-05-15blockjob: Move RateLimit to BlockJobKevin Wolf4-12/+9
Every block job has a RateLimit, and they all do the exact same thing with it, so it should be common infrastructure. Move the struct field for a start. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
2018-05-15blockjob: Wrappers for progress counter accessKevin Wolf4-29/+34
Block job drivers are not expected to mess with the internals of the BlockJob object, so provide wrapper functions for one of the cases where they still do it: Updating the progress counter. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
2018-05-15block: Merge .bdrv_co_writev{,_flags} in driversEric Blake10-22/+32
We have too many driver callback interfaces; simplify the mess somewhat by merging the flags parameter of .bdrv_co_writev_flags() into .bdrv_co_writev(). Note that as long as a driver doesn't set .supported_write_flags, the flags argument will be 0 and behavior is identical. Also note that the public function bdrv_co_writev() still lacks a flags argument; so the driver signature is thus intentionally slightly different. But that's not the end of the world, nor the first time that the driver interface differs slightly from the public interface. Ideally, we should be rewriting all of these drivers to use modern byte-based interfaces. But that's a more invasive patch to write and audit, compared to the simplification done here. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-05-15block: Drop last of the sector-based aio callbacksEric Blake1-48/+36
We are gradually moving away from sector-based interfaces, towards byte-based. Now that all drivers with aio callbacks are using the byte-based interfaces, we can remove the sector-based versions. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-05-15vxhs: Switch to byte-based callbacksEric Blake1-21/+22
We are gradually moving away from sector-based interfaces, towards byte-based. Make the change for the last few sector-based callbacks in the vxhs driver. Note that the driver was already using byte-based calls for performing actual I/O, so this just gets rid of a round trip of scaling; however, as I don't know if VxHS is tolerant of non-sector AIO operations, I went with the conservative approach of adding .bdrv_refresh_limits to override the block layer defaults back to the pre-patch value of 512. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-05-15rbd: Switch to byte-based callbacksEric Blake1-18/+22
We are gradually moving away from sector-based interfaces, towards byte-based. Make the change for the last few sector-based callbacks in the rbd driver. Note that the driver was already using byte-based calls for performing actual I/O, so this just gets rid of a round trip of scaling; however, as I don't know if RBD is tolerant of non-sector AIO operations, I went with the conservate approach of adding .bdrv_refresh_limits to override the block layer defaults back to the pre-patch value of 512. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-05-15null: Switch to byte-based read/writeEric Blake1-22/+23
We are gradually moving away from sector-based interfaces, towards byte-based. Make the change for the last few sector-based callbacks in the null-co and null-aio drivers. Note that since the null driver does nothing on writes, it trivially supports the BDRV_REQ_FUA flag (all writes have already landed to the same bit-bucket without needing an extra flush call). Also, since the null driver does just as well with byte-based requests, we can now avoid cycles wasted on read-modify-write by taking advantage of the block layer now defaulting the alignment to 1 instead of 512. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-05-15file-win32: Switch to byte-based callbacksEric Blake2-21/+31
We are gradually moving away from sector-based interfaces, towards byte-based. Make the change for the last few sector-based callbacks in the file-win32 driver. Note that the driver was already using byte-based calls for performing actual I/O, so this just gets rid of a round trip of scaling; however, as I don't know if Windows is tolerant of non-sector AIO operations, I went with the conservative approach of modifying .bdrv_refresh_limits to override the block layer defaults back to the pre-patch value of 512. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-05-15block: Support byte-based aio callbacksEric Blake1-9/+29
We are gradually moving away from sector-based interfaces, towards byte-based. Add new sector-based aio callbacks for read and write, to match the fact that bdrv_aio_pdiscard is already byte-based. Ideally, drivers should be converted to use coroutine callbacks rather than aio; but that is not quite as trivial (and if we were to do that conversion, the null-aio driver would disappear), so for the short term, converting the signature but keeping things with aio is easier. However, we CAN declare that a driver that uses the byte-based aio interfaces now defaults to byte-based operations, and must explicitly provide a refresh_limits override to stick with larger alignments (making the alignment issues more obvious directly in the drivers touched in the next few patches). Once all drivers are converted, the sector-based aio callbacks will be removed; in the meantime, a FIXME comment is added due to a slight inefficiency that will be touched up as part of that later cleanup. Simplify some instances of 'bs->drv' into 'drv' while touching this, since the local variable already exists to reduce typing. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-05-15block-backend: simplify blk_get_aio_contextDaniel Henrique Barboza1-7/+1
blk_get_aio_context verifies if BlockDriverState bs is not NULL, return bdrv_get_aio_context(bs) if true or qemu_get_aio_context() otherwise. However, bdrv_get_aio_context from block.c already does this verification itself, also returning qemu_get_aio_context() if bs is NULL: AioContext *bdrv_get_aio_context(BlockDriverState *bs) { return bs ? bs->aio_context : qemu_get_aio_context(); } This patch simplifies blk_get_aio_context to simply call bdrv_get_aio_context instead of replicating the same logic. Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-05-11block/file-posix: add x-check-page-cache=on|off optionStefan Hajnoczi1-2/+98
mincore(2) checks whether pages are resident. Use it to verify that page cache has been dropped. You can trigger a verification failure by mmapping the image file from another process that loads a byte from a page, forcing it to become resident. bdrv_co_invalidate_cache() will fail while that process is alive. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Message-id: 20180427162312.18583-3-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-05-11block/file-posix: implement bdrv_co_invalidate_cache() on LinuxStefan Hajnoczi1-0/+46
On Linux posix_fadvise(POSIX_FADV_DONTNEED) invalidates pages*. Use this to drop page cache on the destination host during shared storage migration. This way the destination host will read the latest copy of the data and will not use stale data from the page cache. The flow is as follows: 1. Source host writes out all dirty pages and inactivates drives. 2. QEMU_VM_EOF is sent on migration stream. 3. Destination host invalidates caches before accessing drives. This patch enables live migration even with -drive cache.direct=off. * Terms and conditions may apply, please see patch for details. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Message-id: 20180427162312.18583-2-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-05-08sheepdog: Fix sd_co_create_opts() memory leaksKevin Wolf1-1/+3
Both the option string for the 'redundancy' option and the SheepdogRedundancy object that is created accordingly could be leaked in error paths. This fixes the memory leaks. Reported by Coverity (CID 1390614 and 1390641). Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20180503153509.22223-1-kwolf@redhat.com Reviewed-by: Jeff Cody <jcody@redhat.com> Signed-off-by: Jeff Cody <jcody@redhat.com>
2018-05-08block/mirror: Make cancel always cancel pre-READYMax Reitz1-1/+3
Commit b76e4458b1eb3c32e9824fe6aa51f67d2b251748 made the mirror block job respect block-job-cancel's @force flag: With that flag set, it would now always really cancel, even post-READY. Unfortunately, it had a side effect: Without that flag set, it would now never cancel, not even before READY. Considering that is an incompatible change and not noted anywhere in the commit or the description of block-job-cancel's @force parameter, this seems unintentional and we should revert to the previous behavior, which is to immediately cancel the job when block-job-cancel is called before source and target are in sync (i.e. before the READY event). Cc: qemu-stable@nongnu.org Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1572856 Reported-by: Yanan Fu <yfu@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-id: 20180501220509.14152-2-mreitz@redhat.com Reviewed-by: Jeff Cody <jcody@redhat.com> Signed-off-by: Jeff Cody <jcody@redhat.com>
2018-05-08block/mirror: honor ratelimit againStefan Hajnoczi1-3/+5
Commit b76e4458b1eb3c32e9824fe6aa51f67d2b251748 ("block/mirror: change the semantic of 'force' of block-job-cancel") accidentally removed the ratelimit in the mirror job. Reintroduce the ratelimit but keep the block-job-cancel force=true behavior that was added in commit b76e4458b1eb3c32e9824fe6aa51f67d2b251748. Note that block_job_sleep_ns() returns immediately when the job is cancelled. Therefore it's safe to unconditionally call block_job_sleep_ns() - a cancelled job does not sleep. This commit fixes the non-deterministic qemu-iotests 185 output. The test relies on the ratelimit to make the job sleep until the 'quit' command is processed. Previously the job could complete before the 'quit' command was received since there was no ratelimit. Cc: Liang Li <liliang.opensource@gmail.com> Cc: Jeff Cody <jcody@redhat.com> Cc: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20180424123527.19168-1-stefanha@redhat.com Signed-off-by: Jeff Cody <jcody@redhat.com>
2018-05-04Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2018-05-04' into ↵Peter Maydell1-3/+7
staging nbd patches for 2018-05-04 - Vladimir Sementsov-Ogievskiy: 0/2 fix coverity bugs - Eric Blake: nbd/client: Fix error messages during NBD_INFO_BLOCK_SIZE - Eric Blake: nbd/client: Relax handling of large NBD_CMD_BLOCK_STATUS reply # gpg: Signature made Fri 04 May 2018 14:25:55 BST # gpg: using RSA key A7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" # gpg: aka "[jpeg image of size 6874]" # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-nbd-2018-05-04: nbd/client: Relax handling of large NBD_CMD_BLOCK_STATUS reply nbd/client: Fix error messages during NBD_INFO_BLOCK_SIZE migration/block-dirty-bitmap: fix memory leak in dirty_bitmap_load_bits nbd/client: fix nbd_negotiate_simple_meta_context Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-05-04nbd/client: Relax handling of large NBD_CMD_BLOCK_STATUS replyEric Blake1-3/+7
The NBD spec is proposing a relaxation of NBD_CMD_BLOCK_STATUS where a server may have the final extent per context give a length beyond the original request, if it can easily prove that subsequent bytes have the same status, on the grounds that a client can take advantage of this information for fewer block status requests. Since qemu 2.12 as a client always sends NBD_CMD_FLAG_REQ_ONE, and rejects a server that sends extra length, the upstream NBD spec will probably limit this behavior to clients that don't request REQ_ONE semantics; but it doesn't hurt to relax qemu to always be permissive of this server behavior, even if it continues to use REQ_ONE. CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20180503222626.1303410-1-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2018-05-04qobject: Modify qobject_ref() to return objMarc-André Lureau5-14/+11
For convenience and clarity, make it possible to call qobject_ref() at the time when the reference is associated with a variable, or argument, by making qobject_ref() return the same pointer as given. Use that to simplify the callers. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180419150145.24795-5-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Useless change to qobject_ref_impl() dropped, commit message improved slightly] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-05-04qobject: Replace qobject_incref/QINCREF qobject_decref/QDECREFMarc-André Lureau24-53/+53
Now that we can safely call QOBJECT() on QObject * as well as its subtypes, we can have macros qobject_ref() / qobject_unref() that work everywhere instead of having to use QINCREF() / QDECREF() for QObject and qobject_incref() / qobject_decref() for its subtypes. The replacement is mechanical, except I broke a long line, and added a cast in monitor_qmp_cleanup_req_queue_locked(). Unlike qobject_decref(), qobject_unref() doesn't accept void *. Note that the new macros evaluate their argument exactly once, thus no need to shout them. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180419150145.24795-4-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Rebased, semantic conflict resolved, commit message improved] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-04-16qcow2: try load bitmaps only onceVladimir Sementsov-Ogievskiy2-8/+9
Checking reopen by existence of some bitmaps is wrong, as it may be some other bitmaps, or on the other hand, user may remove bitmaps. This criteria is bad. To simplify things and make behavior more predictable let's just add a flag to remember, that we've already tried to load bitmaps on open and do not want do it again. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20180411122606.367301-2-vsementsov@virtuozzo.com [mreitz: Changed comment wording according to Eric Blake's suggestion] Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-04-10commit/stream: Reset delay_nsKevin Wolf2-0/+4
Streaming and the commit block job only want to apply throttling when they actually copied data instead of skipping it, so they made the calculation of delay_ns conditional. However, delay_ns isn't reset when skipping some sectors, so instead of not waiting, the old delay is applied again. Properly reset delay_ns where needed. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2018-04-04block/rbd: remove processed options from qdictJeff Cody1-0/+7
Commit 4bfb274 added some QAPIfication of option parsing in qemu_rbd_open(). We need to remove all the options we processed, otherwise in bdrv_open_inherit() we will think the remaining options are invalid. (This needs to go in 2.12 to avoid a regression that prevents rbd from being opened.) Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2018-04-04Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into stagingPeter Maydell1-4/+15
Block layer patches # gpg: Signature made Tue 03 Apr 2018 16:48:53 BST # gpg: using RSA key 7F09B272C88F2FD6 # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kevin/tags/for-upstream: iotests: Test abnormally large size in compressed cluster descriptor qemu-iotests: Use ppc64 qemu_arch on ppc64le host iotests: Test preallocated truncate of 2G image block/file-posix: Fix fully preallocated truncate iotests: fix 208 for luks format iotests: Update 186 after commit ac64273c66ab136c44043259162 iotests: Update 051 and 186 after commit 1454509726719e0933c block: handle invalid lseek returns gracefully gluster: Fix blockdev-add with server.N.type=unix Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-04-04Merge remote-tracking branch 'remotes/cody/tags/block-pull-request' into stagingPeter Maydell1-4/+17
# gpg: Signature made Tue 03 Apr 2018 17:10:22 BST # gpg: using RSA key BDBE7B27C0DE3057 # gpg: Good signature from "Jeffrey Cody <jcody@redhat.com>" # gpg: aka "Jeffrey Cody <jeff@codyprime.org>" # gpg: aka "Jeffrey Cody <codyprime@gmail.com>" # Primary key fingerprint: 9957 4B4D 3474 90E7 9D98 D624 BDBE 7B27 C0DE 3057 * remotes/cody/tags/block-pull-request: gluster: Fix blockdev-add with server.N.type=unix blockjob: use qapi enum helpers blockjob: leak fix, remove from txn when failing early Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-04-03block/file-posix: Fix fully preallocated truncateMax Reitz1-2/+3
Storing the lseek() result in an int results in it overflowing when the file is at least 2 GB big. Then, we have a 50 % chance of the result being "negative" and thus thinking an error occurred when actually everything went just fine. So we should use the correct type for storing the result: off_t. Reported-by: Daniel P. Berrange <berrange@redhat.com> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1549231 Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20180228131315.30194-2-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-04-03gluster: Fix blockdev-add with server.N.type=unixKevin Wolf1-4/+17
The legacy command line interface gets the socket path from an option called 'socket'. QAPI in contract uses SocketAddress, where the corresponding option is called 'path'. Fix the gluster block driver to accept both 'socket' and 'path', with 'path' being the preferred syntax. https://bugzilla.redhat.com/show_bug.cgi?id=1545155 Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20180403110810.25624-1-kwolf@redhat.com Signed-off-by: Jeff Cody <jcody@redhat.com>
2018-04-03block: handle invalid lseek returns gracefullyJeff Cody1-2/+12
In commit 223a23c198787328ae75bc65d84edf5fde33c0b6, we implemented a workaround in the gluster driver to handle invalid values returned for SEEK_DATA or SEEK_HOLE. In some instances, these same invalid values can be seen in the posix file handler as well - for example, it has been reported on FUSE gluster mounts. Calling assert() for these invalid values is overly harsh; we can safely return -EIO and allow this case to be treated as a "learned nothing" case (e.g., D4 / H4, as commented in the code). This patch does the same thing that 223a23c198787 did for gluster.c, except in file-posix.c Signed-off-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-04-03gluster: Fix blockdev-add with server.N.type=unixKevin Wolf1-4/+17
The legacy command line interface gets the socket path from an option called 'socket'. QAPI in contract uses SocketAddress, where the corresponding option is called 'path'. Fix the gluster block driver to accept both 'socket' and 'path', with 'path' being the preferred syntax. https://bugzilla.redhat.com/show_bug.cgi?id=1545155 Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2018-04-02nbd: Fix 32-bit compilation on BLOCK_STATUSEric Blake1-1/+1
iotests 123 and 209 fail on 32-bit platforms. The culprit: sizeof(extent) is wrong; we want sizeof(*extent). But since the struct is 8 bytes, it happened to work on 64-bit platforms where the pointer is also 8 bytes (nasty). Fixes: 78a33ab58 Reported-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20180327210517.1804242-1-eblake@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2018-03-27Merge remote-tracking branch 'remotes/ericb/tags/pull-qapi-2018-03-27-v2' ↵Peter Maydell1-6/+5
into staging qapi patches for 2018-03-27, 2.12-rc1 - Marc-André Lureau: qmp-test: fix response leak - Eric Blake: tests: Silence false positive warning on generated test name - Laurent Vivier: 0/4 (partial) coccinelle: re-run scripts from scripst/coccinelle - Peter Xu: 0/8 Monitor: some oob related patches (fixes, new param, tests) - Satheesh Rajendran: hmp.c: Revert hmp_info_cpus output format change # gpg: Signature made Tue 27 Mar 2018 16:18:36 BST # gpg: using RSA key A7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" # gpg: aka "[jpeg image of size 6874]" # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-qapi-2018-03-27-v2: hmp.c: Revert hmp_info_cpus output format change tests: qmp-test: add test for new "x-oob" tests: Add parameter to qtest_init_without_qmp_handshake monitor: new parameter "x-oob" qmp: cleanup qmp queues properly tests: add oob-test for qapi-schema tests: let qapi-schema tests detect oob qapi: restrict allow-oob value to be "true" qmp: fix qmp_capabilities error regression qdict: remove useless cast error: Remove NULL checks on error_propagate() calls error: Strip trailing '\n' from error string arguments (again again) tests: Silence false positive warning on generated test name qmp-test: fix response leak Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-27Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into ↵Peter Maydell1-2/+1
staging # gpg: Signature made Tue 27 Mar 2018 15:41:11 BST # gpg: using RSA key 9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/block-pull-request: MAINTAINERS: add include/block/aio-wait.h coroutine: add test-aio coroutine queue chaining test case coroutine: avoid co_queue_wakeup recursion queue: add QSIMPLEQ_PREPEND() Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-27qdict: remove useless castLaurent Vivier1-6/+5
Re-run Coccinelle script scripts/coccinelle/qobject.cocci Signed-off-by: Laurent Vivier <lvivier@redhat.com> Message-Id: <20180323143202.28879-5-lvivier@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Acked-by: Fam Zheng <famz@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-27Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2018-03-26' ↵Peter Maydell4-2/+38
into staging A fix for dirty bitmap migration through shared storage, and a VMDK patch keeping us from creating too large extents. # gpg: Signature made Mon 26 Mar 2018 21:17:05 BST # gpg: using RSA key F407DB0061D5CF40 # gpg: Good signature from "Max Reitz <mreitz@redhat.com>" # Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40 * remotes/maxreitz/tags/pull-block-2018-03-26: vmdk: return ERROR when cluster sector is larger than vmdk limitation iotests: enable shared migration cases in 169 qcow2: fix bitmaps loading when bitmaps already exist qcow2-bitmap: add qcow2_reopen_bitmaps_rw_hint() Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-27coroutine: avoid co_queue_wakeup recursionStefan Hajnoczi1-2/+1
qemu_aio_coroutine_enter() is (indirectly) called recursively when processing co_queue_wakeup. This can lead to stack exhaustion. This patch rewrites co_queue_wakeup in an iterative fashion (instead of recursive) with bounded memory usage to prevent stack exhaustion. qemu_co_queue_run_restart() is inlined into qemu_aio_coroutine_enter() and the qemu_coroutine_enter() call is turned into a loop to avoid recursion. There is one change that is worth mentioning: Previously, when coroutine A queued coroutine B, qemu_co_queue_run_restart() entered coroutine B from coroutine A. If A was terminating then it would still stay alive until B yielded. After this patch B is entered by A's parent so that a A can be deleted immediately if it is terminating. It is safe to make this change since B could never interact with A if it was terminating anyway. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20180322152834.12656-3-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-03-26vmdk: return ERROR when cluster sector is larger than vmdk limitationyuchenlin1-0/+6
VMDK has a hard limitation of extent size, which is due to the size of grain table entry is 32 bits. It means it can only point to a grain located at offset = 2^32. To avoid writing the user data beyond limitation and record a useless offset in grain table. We should return ERROR here. Signed-off-by: yuchenlin <yuchenlin@synology.com> Message-id: 20180322133337.28024-1-yuchenlin@synology.com Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-03-26qcow2: fix bitmaps loading when bitmaps already existVladimir Sementsov-Ogievskiy1-1/+16
On reopen with existing bitmaps, instead of loading bitmaps, lets reopen them if needed. This also fixes bitmaps migration through shared storage. Consider the case. Persistent bitmaps are stored on bdrv_inactivate. Then, on destination process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps are already loaded on destination start. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20180320170521.32152-3-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-03-26qcow2-bitmap: add qcow2_reopen_bitmaps_rw_hint()Vladimir Sementsov-Ogievskiy2-1/+16
Add version of qcow2_reopen_bitmaps_rw, which do the same work but also return a hint about was header updated or not. This will be used in the following fix for bitmaps reloading after migration. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20180320170521.32152-2-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-03-26vhdx: Check for 4 GB maximum log size on creationKevin Wolf1-0/+4
It's unclear what the real maximum is, but we use an uint32_t to store the log size in vhdx_co_create(), so we should check that the given value fits in 32 bits. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-03-26vhdx: Don't use error_setg_errno() with constant errnoKevin Wolf1-5/+4
error_setg_errno() is meant for cases where we got an errno from the OS that can add useful extra information to an error message. It's pointless if we pass a constant errno, these cases should use plain error_setg(). Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-03-26vhdx: Require power-of-two block size on createKevin Wolf1-0/+4
Images with a non-power-of-two block size are invalid and cannot be opened. Reject such block sizes when creating an image. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com>