aboutsummaryrefslogtreecommitdiff
path: root/block
AgeCommit message (Collapse)AuthorFilesLines
2023-09-22nbd: Replace bool structured_reply with mode enumEric Blake1-3/+5
The upcoming patches for 64-bit extensions requires various points in the protocol to make decisions based on what was negotiated. While we could easily add a 'bool extended_headers' alongside the existing 'bool structured_reply', this does not scale well if more modes are added in the future. Better is to expose the mode enum added in the recent commit bfe04d0a7d out to a wider use in the code base. Where the code previously checked for structured_reply being set or clear, it now prefers checking for an inequality; this works because the nodes are in a continuum of increasing abilities, and allows us to touch fewer places if we ever insert other modes in the middle of the enum. There should be no semantic change in this patch. Signed-off-by: Eric Blake <eblake@redhat.com> Message-ID: <20230829175826.377251-20-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
2023-09-21Merge tag 'pull-parallels-2023-09-20-v2' of ↵Stefan Hajnoczi2-89/+303
https://src.openvz.org/scm/~den/qemu into staging Parallels format driver: * regular calculation of cluster used bitmap of the image file * cluster allocation on the base of that bitmap (effectively allocation of new clusters could be done inside the image if that offset space is unused) * support of DISCARD and WRITE_ZEROES operations * image check bugfixes * unit tests fixes * unit tests covering new functionality # -----BEGIN PGP SIGNATURE----- # # iQHDBAABCgAtFiEE9vE2f3B8+RUZInytPzClrpN3nJ8FAmUL7u4PHGRlbkBvcGVu # dnoub3JnAAoJED8wpa6Td5yfdaUL/RW+nOYlFNXlrjOVeasgGLkAKrKBja8O3/As # aRo0DLZKITK8qbLEBAeTDyCpN9LLwy7WdUR1uT4V54FzE5zZP6HAdBEoj9AsaW/9 # wsTF+oyKeqmXw2y348t+lclp8eREHySecwiVhaxTpG9J2TQfDP/D2yhzRU88P7nH # rbVZjOF2yOthzW6Y8h8e/LMd8rfODO053tYaMEBngjirBZnhESH3mAm1WB5mYs+q # 2++4XQZcFFKWFp952MaEDphpwYdh80E65g4vth80JrDTyyMH0KZE9cQqbFb5UgZv # aV1/DCaH0WTSDbjCaI/SrmqKXrO0Mkd/y/ShoQpTu7qJO/FbaClA58f+KfGE7VBd # Fa5pM+JN12UVNxnNIF/Oe+wAiVUJYKtLaDMKibj+MUjM5sE/ZRLqzFLktDbQT0kS # Qvs1u8HTvirJpvxOkJv4cEuNw07JERCzpl/qPF6XkS9rcKeIormhftaaRmjILxS/ # KEmDVNj63g1D0XDY3WTF7LHLNjtXpw== # =FUWj # -----END PGP SIGNATURE----- # gpg: Signature made Thu 21 Sep 2023 03:21:18 EDT # gpg: using RSA key F6F1367F707CF91519227CAD3F30A5AE93779C9F # gpg: issuer "den@openvz.org" # gpg: Good signature from "Denis V. Lunev <den@openvz.org>" [unknown] # gpg: WARNING: The key's User ID is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: F6F1 367F 707C F915 1922 7CAD 3F30 A5AE 9377 9C9F * tag 'pull-parallels-2023-09-20-v2' of https://src.openvz.org/scm/~den/qemu: (22 commits) tests: extend test 131 to cover availability of the write-zeroes parallels: naive implementation of parallels_co_pwrite_zeroes tests: extend test 131 to cover availability of the discard operation parallels: naive implementation of parallels_co_pdiscard parallels: improve readability of allocate_clusters parallels: naive implementation of allocate_clusters with used bitmap parallels: update used bitmap in allocate_cluster parallels: accept multiple clusters in mark_used() tests: test self-cure of parallels image with duplicated clusters tests: fix broken deduplication check in parallels format test parallels: collect bitmap of used clusters at open parallels: add test which will validate data_off fixes through repair parallels: fix broken parallels_check_data_off() tests: ensure that image validation will not cure the corruption parallels: create mark_used() helper which sets bit in used bitmap parallels: refactor path when we need to re-check image in parallels_open parallels: return earlier from parallels_open() function on error parallels: return earler in fail_format branch in parallels_open() parallels: invent parallels_opts_prealloc() helper to parse prealloc opts parallels: fix memory leak in parallels_open() ... Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-09-21Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into stagingStefan Hajnoczi16-125/+232
Block layer patches - Graph locking part 4 (node management) - qemu-img map: report compressed data blocks - block-backend: process I/O in the current AioContext # -----BEGIN PGP SIGNATURE----- # # iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmULHnURHGt3b2xmQHJl # ZGhhdC5jb20ACgkQfwmycsiPL9aB5hAAqH8To7WIUtg1rj1PY809ck78ghm18PKg # TNdN7IbrXQghX5foh2VgPwVVl+JaW2CSrJYWQcAO6AbvFduNIi9iKzI6RT0xKXpb # b8oQXS7zntFzwBv8ohOU5NSVJOgVmNP4h5qJIMmXgB9ZcLFG40zggVH2qQT7guUf # 9MAc81kI/d5vvSHY0ZjdHjNOgwG4q1j8yytL7OFqWUfB8sXloUCA9lT7w4jIYD8L # v2StUOLWB01Zts2o8SCNaFxuajs6wUee8b/DM1cyPyLy4KtOdXvLKhq2NlXpLo2i # aZFr4PtizTVwrQZIJttA9jqM+QCsDOsiSat3BLNNsKUaCWHZB0rOGLCzMCtisyOo # 4PzuL4UI21ik2zieO1qVM+Thqvw16kHtp6dD9pGk4X4ogGreGYEIxzBl79luR+AV # NCRizoeFWTHKymS1tSoKrWT9ZNHcLmwemO6Tt1rMYk9jV3T4uY5e1NwxaUavEfsX # f8dLfQjhNiySOoDknT1OSerBOVdTXURS2ri5H3GZxrxvJ4jOeFkn52C8r3YlZ3Wp # Cr9LCUJZeXgwY+Q1JQ3D4VLY8aZ83txpw6XKEy0eTEv5wxkBj5LWhXx7hNb5F3lg # bqaRYijVJn+P82wVxlftIzMfNeVBFHzFE90taPV5grJjr8lgrGBFmD7Puc97kfDX # oTDBwRxJeew= # =qTNA # -----END PGP SIGNATURE----- # gpg: Signature made Wed 20 Sep 2023 12:31:49 EDT # gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6 # gpg: issuer "kwolf@redhat.com" # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (28 commits) block: mark aio_poll as non-coroutine block-backend: process zoned requests in the current AioContext block-backend: process I/O in the current AioContext test-bdrv-drain: avoid race with BH in IOThread drain test block: remove AIOCBInfo->get_aio_context() qemu-img: map: report compressed data blocks block: add BDRV_BLOCK_COMPRESSED flag for bdrv_block_status() block: Mark bdrv_add/del_child() and caller GRAPH_WRLOCK block: Mark bdrv_unref_child() GRAPH_WRLOCK block: Mark bdrv_root_unref_child() GRAPH_WRLOCK block: Take graph rdlock in bdrv_change_aio_context() block: Take graph rdlock in bdrv_drop_intermediate() block: Mark bdrv_parent_cb_change_media() GRAPH_RDLOCK block: Mark bdrv_child_perm() GRAPH_RDLOCK block: Mark bdrv_get_cumulative_perm() and callers GRAPH_RDLOCK block: Mark bdrv_parent_perms_conflict() and callers GRAPH_RDLOCK block: Mark bdrv_attach_child() GRAPH_WRLOCK block: Call transaction callbacks with lock held block: Mark bdrv_attach_child_common() GRAPH_WRLOCK block: Mark bdrv_replace_child_tran() GRAPH_WRLOCK ... Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-09-21Merge tag 'pull-block-2023-09-01' of https://gitlab.com/hreitz/qemu into stagingStefan Hajnoczi4-108/+109
Block patches - Fix for file-posix's zoning code crashing on I/O errors - Throttling refactoring # -----BEGIN PGP SIGNATURE----- # # iQJGBAABCAAwFiEEy2LXoO44KeRfAE00ofpA0JgBnN8FAmTxnMISHGhyZWl0ekBy # ZWRoYXQuY29tAAoJEKH6QNCYAZzfYkUP+gMG9hhzvgjj/tw9rEBQjciihzcQmqQJ # 2Mm37RH2jj5bnnTdaTbMkcRRwVhncYSCwK9q5EYVbZmU9C/v4YJmsSEQlcl7wVou # hbPUv6NHaBrJZX9nxNSa2RHui6pZMLKa/D0rJVB7NjYBrrRtiPo7kiLVQYjYXa2g # kcCCfY4t3Z2RxOP31mMXRjYlhJE9bIuZdTEndrKme8KS2JGPZEJ9xjkoW1tj96EX # oc/Cg2vk7AEtsFYA0bcD8fTFkBDJEwyYl3usu7Tk24pvH16jk7wFSqRVSsDMfnER # tG8X3mHLIY0hbSkpzdHJdXINvZ6FWpQb0CGzIKr+pMiuWVdWr1HglBr0m4pVF+Y4 # A6AI6VX2JJgtacypoDyCZC9mzs1jIdeiwq9v5dyuikJ6ivTwEEoeoSLnLTN3AjXn # 0mtQYzgCg5Gd6+rTo7XjSO9SSlbaVrDl/B2eXle6tmIFT5k+86fh0hc+zTmP8Rkw # Knbc+5Le95wlMrOUNx2GhXrTGwX510hLxKboho/LITxtAzqvXnEJKrYbnkm3WPnw # wfHnR5VQH1NKEpiH/p33og6OV/vu9e7vgp0ZNZV136SnzC90C1zMUwg2simJW701 # 34EtN0XBX8XBKrxfe7KscV9kRE8wrWWJVbhp+WOcQEomGI8uraxzWqDIk/v7NZXv # m4XBscaB+Iri # =oKgk # -----END PGP SIGNATURE----- # gpg: Signature made Fri 01 Sep 2023 04:11:46 EDT # gpg: using RSA key CB62D7A0EE3829E45F004D34A1FA40D098019CDF # gpg: issuer "hreitz@redhat.com" # gpg: Good signature from "Hanna Reitz <hreitz@redhat.com>" [unknown] # gpg: WARNING: The key's User ID is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: CB62 D7A0 EE38 29E4 5F00 4D34 A1FA 40D0 9801 9CDF * tag 'pull-block-2023-09-01' of https://gitlab.com/hreitz/qemu: tests/file-io-error: New test file-posix: Simplify raw_co_prw's 'out' zone code file-posix: Fix zone update in I/O error path file-posix: Check bs->bl.zoned for zone info file-posix: Clear bs->bl.zoned on error block/throttle-groups: Use ThrottleDirection instread of bool is_write fsdev: Use ThrottleDirection instread of bool is_write throttle: use THROTTLE_MAX/ARRAY_SIZE for hard code throttle: use enum ThrottleDirection instead of bool is_write cryptodev: use NULL throttle timer cb for read direction test-throttle: test read only and write only throttle: support read-only and write-only test-throttle: use enum ThrottleDirection throttle: introduce enum ThrottleDirection Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-09-21parallels: naive implementation of parallels_co_pwrite_zeroesDenis V. Lunev1-0/+14
The zero flag is missed in the Parallels format specification. We can resort to discard if we have no backing file. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: naive implementation of parallels_co_pdiscardDenis V. Lunev1-0/+46
* Discarding with backing stores is not supported by the format. * There is no buffering/queueing of the discard operation. * Only operations aligned to the cluster are supported. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: improve readability of allocate_clustersDenis V. Lunev1-8/+5
Replace 'space' representing the amount of data to preallocate with 'bytes'. Rationale: * 'space' at each place is converted to bytes * the unit is more close to the variable name Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: naive implementation of allocate_clusters with used bitmapDenis V. Lunev1-12/+39
The access to the bitmap is not optimized completely. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: update used bitmap in allocate_clusterDenis V. Lunev1-0/+14
We should extend the bitmap if the file is extended and set the bit in the image used bitmap once the cluster is allocated. Sanity check at that moment also looks like a good idea. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: accept multiple clusters in mark_used()Denis V. Lunev1-8/+10
This would be useful in the next patch in allocate_clusters(). This change would not imply serious performance drawbacks as usually image is full of data or are at the end of the bitmap. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: collect bitmap of used clusters at openDenis V. Lunev2-0/+76
If the operation is failed, we need to check image consistency if the problem is not about memory allocation. Bitmap adjustments in allocate_cluster are not performed yet. They worth to be separate. This was proven useful during debug of this series. Kept as is for future bissecting. It should be specifically noted that used bitmap must be recalculated if data_off has been fixed during image consistency check. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: fix broken parallels_check_data_off()Denis V. Lunev1-0/+1
Once we have repaired data_off field in the header we should update s->data_start which is calculated on the base of it. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: create mark_used() helper which sets bit in used bitmapDenis V. Lunev1-9/+25
This functionality is used twice already and next patch will add more code with it. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: refactor path when we need to re-check image in parallels_openDenis V. Lunev1-11/+10
More conditions follows thus the check should be more scalable. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: return earlier from parallels_open() function on errorDenis V. Lunev1-9/+5
At the beginning of the function we can return immediately until we really allocate s->header. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: return earler in fail_format branch in parallels_open()Denis V. Lunev1-1/+2
We do not need to perform any deallocation/cleanup if wrong format is detected. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: invent parallels_opts_prealloc() helper to parse prealloc optsDenis V. Lunev1-28/+44
This patch creates above mentioned helper and moves its usage to the beginning of parallels_open(). This simplifies parallels_open() a bit. The patch also ensures that we store prealloc_size on block driver state always in sectors. This makes code cleaner and avoids wrong opinion at the assignment that the value is in bytes. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: fix memory leak in parallels_open()Denis V. Lunev1-0/+1
We should free opts allocated through qemu_opts_create() at the end. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: mark driver as supporting CBTDenis V. Lunev1-0/+6
Parallels driver indeed support Parallels Dirty Bitmap Feature in read-only mode. The patch adds bdrv_supports_persistent_dirty_bitmap() callback which always return 1 to indicate that. This will allow to copy CBT from Parallels image with qemu-img. Note: read-write support is signalled through bdrv_co_can_store_new_dirty_bitmap() and is different. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-21parallels: fix formatting in bdrv_parallels initializationDenis V. Lunev1-17/+19
Old code is ugly and contains tabulations. There are no functional changes in this patch. Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
2023-09-20block-backend: process zoned requests in the current AioContextStefan Hajnoczi1-6/+6
Process zoned requests in the current thread's AioContext instead of in the BlockBackend's AioContext. There is no need to use the BlockBackend's AioContext thanks to CoMutex bs->wps->colock, which protects zone metadata. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230912231037.826804-5-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block-backend: process I/O in the current AioContextStefan Hajnoczi1-3/+3
Switch blk_aio_*() APIs over to multi-queue by using qemu_get_current_aio_context() instead of blk_get_aio_context(). This change will allow devices to process I/O in multiple IOThreads in the future. I audited existing blk_aio_*() callers: - migration/block.c: blk_mig_lock() protects the data accessed by the completion callback. - The remaining emulated devices and exports run with qemu_get_aio_context() == blk_get_aio_context(). Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230912231037.826804-4-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: remove AIOCBInfo->get_aio_context()Stefan Hajnoczi2-32/+8
The synchronous bdrv_aio_cancel() function needs the acb's AioContext so it can call aio_poll() to wait for cancellation. It turns out that all users run under the BQL in the main AioContext, so this callback is not needed. Remove the callback, mark bdrv_aio_cancel() GLOBAL_STATE_CODE just like its blk_aio_cancel() caller, and poll the main loop AioContext. The purpose of this cleanup is to identify bdrv_aio_cancel() as an API that does not work with the multi-queue block layer. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230912231037.826804-2-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Klaus Jensen <k.jensen@samsung.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: add BDRV_BLOCK_COMPRESSED flag for bdrv_block_status()Andrey Drobyshev via3-1/+9
Functions qcow2_get_host_offset(), get_cluster_offset(), vmdk_co_block_status() explicitly report compressed cluster types when data is compressed. However, this information is never passed further. Let's make use of it by adding new BDRV_BLOCK_COMPRESSED flag for bdrv_block_status(), so that caller may know that the data range is compressed. In particular, we're going to use this flag to tweak "qemu-img map" output. This new flag is only being utilized by qcow, qcow2 and vmdk formats, as only those support compression. Reviewed-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Hanna Czenczek <hreitz@redhat.com> Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> Message-ID: <20230907210226.953821-2-andrey.drobyshev@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: Mark bdrv_add/del_child() and caller GRAPH_WRLOCKKevin Wolf1-17/+6
The functions read the parents list in the generic block layer, so we need to hold the graph lock already there. The BlockDriver implementations actually modify the graph, so it has to be a writer lock. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-22-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: Mark bdrv_unref_child() GRAPH_WRLOCKKevin Wolf7-1/+31
Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_unref_child(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-21-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: Mark bdrv_root_unref_child() GRAPH_WRLOCKKevin Wolf1-0/+3
Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_root_unref_child(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-20-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: Mark bdrv_child_perm() GRAPH_RDLOCKKevin Wolf1-5/+5
This adds GRAPH_RDLOCK annotations to declare that callers of bdrv_child_perm() need to hold a reader lock for the graph because some implementations access the children list of a node. The callers of bdrv_child_perm() conveniently already hold the lock. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-16-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: Mark bdrv_parent_perms_conflict() and callers GRAPH_RDLOCKKevin Wolf4-8/+34
The function reads the parents list, so it needs to hold the graph lock. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-14-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: Mark bdrv_attach_child() GRAPH_WRLOCKKevin Wolf2-0/+8
Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_attach_child_common(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-13-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: Mark bdrv_attach_child_common() GRAPH_WRLOCKKevin Wolf1-5/+15
Instead of taking the writer lock internally, require callers to already hold it when calling bdrv_attach_child_common(). These callers will typically already hold the graph lock once the locking work is completed, which means that they can't call functions that take it internally. Note that the transaction callbacks still take the lock internally, so tran_finalize() must be called without the lock held. This is because bdrv_append() also calls bdrv_replace_node_noperm(), which currently requires the transaction callbacks to be called unlocked. In the next step, both of them can be switched to locked tran_finalize() calls together. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-11-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20block: Introduce bdrv_schedule_unref()Kevin Wolf1-7/+19
bdrv_unref() is called by a lot of places that need to hold the graph lock (it naturally happens in the context of operations that change the graph). However, bdrv_unref() takes the graph writer lock internally, so it can't actually be called while already holding a graph lock without causing a deadlock. bdrv_unref() also can't just become GRAPH_WRLOCK because it drains the node before closing it, and draining requires that the graph is unlocked. The solution is to defer deleting the node until we don't hold the lock any more and draining is possible again. Note that keeping images open for longer than necessary can create problems, too: You can't open an image again before it is really closed (if image locking didn't prevent it, it would cause corruption). Reopening an image immediately happens at least during bdrv_open() and bdrv_co_create(). In order to solve this problem, make sure to run the deferred unref in bdrv_graph_wrunlock(), i.e. the first possible place where we can drain again. This is also why bdrv_schedule_unref() is marked GRAPH_WRLOCK. The output of iotest 051 is updated because the additional polling changes the order of HMP output, resulting in a new "(qemu)" prompt in the test output that was previously on a separate line and filtered out. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-ID: <20230911094620.45040-6-kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20preallocate: Don't poll during permission updatesKevin Wolf1-20/+69
When the permission related BlockDriver callbacks are called, we are in the middle of an operation traversing the block graph. Polling in such a place is a very bad idea because the graph could change in unexpected ways. In the future, callers will also hold the graph lock, which is likely to turn polling into a deadlock. So we need to get rid of calls to functions like bdrv_getlength() or bdrv_truncate() there as these functions poll internally. They are currently used so that when no parent has write/resize permissions on the image any more, the preallocate filter drops the extra preallocated area in the image file and gives up write/resize permissions itself. In order to achieve this without polling in .bdrv_check_perm, don't immediately truncate the image, but only schedule a BH to do so. The filter keeps the write/resize permissions a bit longer now until the BH has executed. There is one case in which delaying doesn't work: Reopening the image read-only. In this case, bs->file will likely be reopened read-only, too, so keeping write permissions a bit longer on it doesn't work. But we can already cover this case in preallocate_reopen_prepare() and not rely on the permission updates for it. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-4-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-20preallocate: Factor out preallocate_truncate_to_real_size()Kevin Wolf1-26/+22
It's essentially the same code in preallocate_check_perm() and preallocate_close(), except that the latter ignores errors. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230911094620.45040-3-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-11Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into stagingStefan Hajnoczi9-54/+33
Block layer patches - Optimise reqs_lock to make multiqueue actually scale - virtio: Drop out of coroutine context in virtio_load() - iotests: Fix reference output for some tests after recent changes - vpc: Avoid dynamic stack allocation - Code cleanup, improved documentation # -----BEGIN PGP SIGNATURE----- # # iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmT7VYgRHGt3b2xmQHJl # ZGhhdC5jb20ACgkQfwmycsiPL9YfOg/7BoYF6lkB7DF/jH3XLY6f8zoI+OVM7dg1 # QFEjyVO+uZiJVh0CeBNI9WgnBe7f5vXMbiStyGbWKo3BLUsjnwoQcW/Sxpw61bR2 # jZYK6UHe0RhFqTQpbt8G1iCmlpRS+sX+Cy+lxcVcbqxcnLRXCOjT6ivyA4bGbYIC # q9BHg/9hBmjuM05NTV6Axy8qjqBGVaIWE9ALTnw8H//waBr4/ydJPTl7EWHe3+tO # Stm73evgPG7aLHM6W4qdFW4gwAQ8f+f42Q+0NH1YavB/pN3LTN1B6sLQY/51du+0 # d/JCsXex0IZQXmNPhqv1h01vhOyU9WBmlwpPG2iZv3a06SXk1ys3rQt/L7uIcsZg # Z58CpcUJ517FERnkl0BWXzYhsdcW2K+RdlaiL5PX6H1A2B9LT05ouZfD47hh7kKv # oX+Ulk05PFr3JRCKQF6QDEejRKXt169bGzInTlns/wXinD/V4sCkUnr9aWQuhoWk # KhQm7WMscTTIyHP2FznO4x9kq0ALsoX/NKqBW2wgJUtqRzsd4XxPp5CXEsAir8Vt # dpne/DaV5iDI1mGFJrvkctJN545tEoezBtUzC8/9rZGE0cxHAkhvQVZUDo7xVmrq # PlGQ1ko9cNui/Gf9B6qDqaJJwSyw0S6vHurGVQJRwbyly57Fi5aisWkr4w7Rc4eA # 7u9B1RvwF/Q= # =2wGD # -----END PGP SIGNATURE----- # gpg: Signature made Fri 08 Sep 2023 13:10:32 EDT # gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6 # gpg: issuer "kwolf@redhat.com" # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * tag 'for-upstream' of https://repo.or.cz/qemu/kevin: virtio: Drop out of coroutine context in virtio_load() vmstate: Mark VMStateInfo.get/put() coroutine_mixed_fn block: Make more BlockDriver definitions static block/meson.build: Restore alphabetical order of files block: Remove unnecessary variable in bdrv_block_device_info block: Remove bdrv_query_block_node_info vmdk: Clean up bdrv_open_child() return value check qemu-img: Update documentation for compressed images block: Be more verbose in create fallback block/iscsi: Document why we use raw malloc() qemu-img: omit errno value in error message block: change reqs_lock to QemuMutex block: minimize bs->reqs_lock section in tracked_request_end() iotests: adapt test output for new qemu_cleanup() behavior block/vpc: Avoid dynamic stack allocation Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-09-08block: Make more BlockDriver definitions staticKevin Wolf3-3/+3
Most block driver implementations don't have any reason for their BlockDriver to be public. The only exceptions are bdrv_file, bdrv_raw and bdrv_qcow2, which are actually used in other source files. Make all other BlockDriver definitions static if they aren't yet. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-ID: <20230905130607.35134-3-kwolf@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-08block/meson.build: Restore alphabetical order of filesKevin Wolf1-6/+6
When commit 5e5733e5999 created block/meson.build, the list of unconditionally added files was in alphabetical order. Later commits added new files in random places. Reorder the list to be alphabetical again. (As for ordering foo.c against foo-*.c, there are both ways used currently; standardise on having foo.c first, even though this is different from the original commit 5e5733e5999.) Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-ID: <20230905130607.35134-2-kwolf@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-08block: Remove unnecessary variable in bdrv_block_device_infoFabiano Rosas1-3/+2
The commit 5d8813593f ("block/qapi: Let bdrv_query_image_info() recurse") removed the loop where we set the 'bs0' variable, so now it is just the same as 'bs'. Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-ID: <20230901184605.32260-3-farosas@suse.de> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-08block: Remove bdrv_query_block_node_infoFabiano Rosas1-27/+0
The last call site of this function has been removed by commit c04d0ab026 ("qemu-img: Let info print block graph"). Reviewed-by: Claudio Fontana <cfontana@suse.de> Signed-off-by: Fabiano Rosas <farosas@suse.de> Message-ID: <20230901184605.32260-2-farosas@suse.de> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-08vmdk: Clean up bdrv_open_child() return value checkDmitry Frolov1-1/+1
bdrv_open_child() may return NULL. Usually return value is checked for this function. Check for return value is more reliable. Fixes: 24bc15d1f6 ("vmdk: Use BdrvChild instead of BDS for references to extents") Signed-off-by: Dmitry Frolov <frolov@swemel.ru> Message-ID: <20230831125926.796205-1-frolov@swemel.ru> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-08block/iscsi: Document why we use raw malloc()Peter Maydell1-0/+1
In block/iscsi.c we use a raw malloc() call, which is unusual given the project standard is to use the glib memory allocation functions. Document why we do so, to avoid it being converted to g_malloc() by mistake. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-ID: <20230727150705.2664464-1-peter.maydell@linaro.org> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-08block: change reqs_lock to QemuMutexStefan Hajnoczi1-12/+12
CoMutex has poor performance when lock contention is high. The tracked requests list is accessed frequently and performance suffers in QEMU multi-queue block layer scenarios. It is not necessary to use CoMutex for the requests lock. The lock is always released across coroutine yield operations. It is held for relatively short periods of time and it is not beneficial to yield when the lock is held by another coroutine. Change the lock type from CoMutex to QemuMutex to improve multi-queue block layer performance. fio randread bs=4k iodepth=64 with 4 IOThreads handling a virtio-blk device with 8 virtqueues improves from 254k to 517k IOPS (+203%). Full benchmark results and configuration details are available here: https://gitlab.com/stefanha/virt-playbooks/-/commit/980c40845d540e3669add1528739503c2e817b57 In the future we may wish to introduce thread-local tracked requests lists to avoid lock contention completely. That would be much more involved though. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230808155852.2745350-3-stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-08block: minimize bs->reqs_lock section in tracked_request_end()Stefan Hajnoczi1-1/+7
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230808155852.2745350-2-stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-08block/vpc: Avoid dynamic stack allocationPhilippe Mathieu-Daudé1-2/+2
Use autofree heap allocation instead of variable-length array on the stack. Here we don't expect the bitmap size to be enormous, and since we're about to read/write it to disk the overhead of the allocation should be fine. The codebase has very few VLAs, and if we can get rid of them all we can make the compiler error on new additions. This is a defensive measure against security bugs where an on-stack dynamic allocation isn't correctly size-checked (e.g. CVE-2021-3527). Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> [PMM: expanded commit message] Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-ID: <20230811175229.808139-1-peter.maydell@linaro.org> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Francisco Iglesias <frasse.iglesias@gmail.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2023-09-08Merge tag 'pull-trivial-patches' of https://gitlab.com/mjt0k/qemu into stagingStefan Hajnoczi12-17/+17
trivial patches for 2023-09-08 # -----BEGIN PGP SIGNATURE----- # # iQFDBAABCAAtFiEEe3O61ovnosKJMUsicBtPaxppPlkFAmT68tMPHG1qdEB0bHMu # bXNrLnJ1AAoJEHAbT2saaT5ZbEwH/2XcX1f4KcEJbgUn0JVhGQ5GH2c2jepZlkTZ # 2dhvdEECbOPMg73hty0fyyWlyuLWdJ9cMpONfMtzmHTH8RKEOAbpn/zusyo3H+48 # 6cunyUpBqbmb7MHPchrN+JmvtvaSPSazsj2Zdkh+Y4WlfEYj+yVysQ4zQlBlRyHv # iOTi6OdjxXg1QcbtJxAUhp+tKaRJzagiCpLkoyW2m8DIuV9cLVHMJsE3OMgfKNgK # /S+O1fLcaDhuSCrHAbZzArF3Tr4bfLqSwDtGCJfQpqKeIQDJuI+41GLIlm1nYY70 # IFJzEWMOrX/rcMG1CQnUFZOOyDSO+NfILwNnU+eyM49MUekmY54= # =mmPS # -----END PGP SIGNATURE----- # gpg: Signature made Fri 08 Sep 2023 06:09:23 EDT # gpg: using RSA key 7B73BAD68BE7A2C289314B22701B4F6B1A693E59 # gpg: issuer "mjt@tls.msk.ru" # gpg: Good signature from "Michael Tokarev <mjt@tls.msk.ru>" [full] # gpg: aka "Michael Tokarev <mjt@corpit.ru>" [full] # gpg: aka "Michael Tokarev <mjt@debian.org>" [full] # Primary key fingerprint: 6EE1 95D1 886E 8FFB 810D 4324 457C E0A0 8044 65C5 # Subkey fingerprint: 7B73 BAD6 8BE7 A2C2 8931 4B22 701B 4F6B 1A69 3E59 * tag 'pull-trivial-patches' of https://gitlab.com/mjt0k/qemu: (22 commits) qxl: don't assert() if device isn't yet initialized hw/net/vmxnet3: Fix guest-triggerable assert() tests/qtest/usb-hcd: Remove the empty "init" tests target/ppc: use g_free() in test_opcode_table() hw/ppc: use g_free() in spapr_tce_table_post_load() trivial: Simplify the spots that use TARGET_BIG_ENDIAN as a numeric value accel/tcg: Fix typo in translator_io_start() description tests/qtest/test-hmp: Fix migrate_set_parameter xbzrle-cache-size test docs tests: Fix use of migrate_set_parameter qemu-options.hx: Rephrase the descriptions of the -hd* and -cdrom options hw/display/xlnx_dp: update comments block: spelling fixes misc/other: spelling fixes qga/: spelling fixes tests/: spelling fixes scripts/: spelling fixes include/: spelling fixes audio: spelling fixes xen: spelling fix riscv: spelling fixes ... Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2023-09-08block: spelling fixesMichael Tokarev12-17/+17
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru> Reviewed-by: Eric Blake <eblake@redhat.com>
2023-09-07io: follow coroutine AioContext in qio_channel_yield()Stefan Hajnoczi1-10/+1
The ongoing QEMU multi-queue block layer effort makes it possible for multiple threads to process I/O in parallel. The nbd block driver is not compatible with the multi-queue block layer yet because QIOChannel cannot be used easily from coroutines running in multiple threads. This series changes the QIOChannel API to make that possible. In the current API, calling qio_channel_attach_aio_context() sets the AioContext where qio_channel_yield() installs an fd handler prior to yielding: qio_channel_attach_aio_context(ioc, my_ctx); ... qio_channel_yield(ioc); // my_ctx is used here ... qio_channel_detach_aio_context(ioc); This API design has limitations: reading and writing must be done in the same AioContext and moving between AioContexts involves a cumbersome sequence of API calls that is not suitable for doing on a per-request basis. There is no fundamental reason why a QIOChannel needs to run within the same AioContext every time qio_channel_yield() is called. QIOChannel only uses the AioContext while inside qio_channel_yield(). The rest of the time, QIOChannel is independent of any AioContext. In the new API, qio_channel_yield() queries the AioContext from the current coroutine using qemu_coroutine_get_aio_context(). There is no need to explicitly attach/detach AioContexts anymore and qio_channel_attach_aio_context() and qio_channel_detach_aio_context() are gone. One coroutine can read from the QIOChannel while another coroutine writes from a different AioContext. This API change allows the nbd block driver to use QIOChannel from any thread. It's important to keep in mind that the block driver already synchronizes QIOChannel access and ensures that two coroutines never read simultaneously or write simultaneously. This patch updates all users of qio_channel_attach_aio_context() to the new API. Most conversions are simple, but vhost-user-server requires a new qemu_coroutine_yield() call to quiesce the vu_client_trip() coroutine when not attached to any AioContext. While the API is has become simpler, there is one wart: QIOChannel has a special case for the iohandler AioContext (used for handlers that must not run in nested event loops). I didn't find an elegant way preserve that behavior, so I added a new API called qio_channel_set_follow_coroutine_ctx(ioc, true|false) for opting in to the new AioContext model. By default QIOChannel uses the iohandler AioHandler. Code that formerly called qio_channel_attach_aio_context() now calls qio_channel_set_follow_coroutine_ctx(ioc, true) once after the QIOChannel is created. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Acked-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20230830224802.493686-5-stefanha@redhat.com> [eblake: also fix migration/rdma.c] Signed-off-by: Eric Blake <eblake@redhat.com>
2023-09-06parallels: Add data_off repairing to parallels_open()Alexander Ivanov1-13/+16
Place data_start/data_end calculation after reading the image header to s->header. Set s->data_start to the offset calculated in parallels_test_data_off(). Call bdrv_check() if data_off is incorrect. Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com> Reviewed-by: Denis V. Lunev <den@openvz.org> Signed-off-by: Denis V. Lunev <den@openvz.org>
2023-09-06parallels: Add data_off checkAlexander Ivanov1-0/+80
data_off field of the parallels image header can be corrupted. Check if this field greater than the header + BAT size and less than file size. Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com> Reviewed-by: Denis V. Lunev <den@openvz.org> Signed-off-by: Denis V. Lunev <den@openvz.org>
2023-09-06parallels: Use bdrv_co_getlength() in parallels_check_outside_image()Alexander Ivanov1-1/+1
bdrv_co_getlength() should be used in coroutine context. Replace bdrv_getlength() by bdrv_co_getlength() in parallels_check_outside_image(). Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com> Reviewed-by: Denis V. Lunev <den@openvz.org> Signed-off-by: Denis V. Lunev <den@openvz.org>
generated by cgit v1.1 at 2025-01-15 10:18:44 +0000