aboutsummaryrefslogtreecommitdiff
path: root/migration
AgeCommit message (Collapse)AuthorFilesLines
2024-04-24Merge tag 'pull-error-2024-04-24' of https://repo.or.cz/qemu/armbru into stagingRichard Henderson3-7/+6
Error reporting patches for 2024-04-24 # -----BEGIN PGP SIGNATURE----- # # iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmYouloSHGFybWJydUBy # ZWRoYXQuY29tAAoJEDhwtADrkYZTzLwP+wQjCWJHpTB+uQ3+U5Tb77BUJxuEjDMj # txNIJBXHOo7erxTSCieLuQICm8e30z62QAK4nVStyMDcyGh1KfwdSDAxBFnuLpA2 # 7X5bXbvCrm4vXVASRTV1zKCYDlIXFfrMWLvN5KgM90RsodLcy0szlXg+qYyoIM3Z # 8zp0Ug0fQPFHiOAQJi9ZTOsCYJBhZc2sbzgQEmf/g6q9bJaZHzPEHvVT4AQhTAtn # 7BIJY+vGDZNZwbP/0obWy2lai3kbGak8OXpwq/bewdrxeRmvqmM7sk+V/P2tXQD+ # kZe0/HWuDoO5J8L3KHiJnBJ0KCk8fbo4I0T6v9vf55Sj8K0r7O9sykgXXWv8q0lO # GrQa0YcyWAckI41stYQpwEpIlRanuZv/p8OZFJIqsTAfaw7RlbIBYA9xZCUnTton # FbHO/t2BLfo8eO9/xRD4r1u6vMbVozImPETuUMPyLHzlrdw2thxddKQNInHYYZ2U # SvvaByceEP2UywOnOflZhVL2dIhhnrBztiW2Vqod1fQHpfBAcJn909PZIlPZyMkr # gUnABI/rtC/lW3pBee6HmfzJ6Fah0e0XCpCY20qFe27Bi/z3xKi5NWYuyAUG5csp # CuTsc4pXfPVj5Z+Mk4pyY8PK5k4jSa7vAVLCLTNzXJLZlJTb6yuf0HsJ7768nHDc # hSEIjLwQWYtw # =r8Rv # -----END PGP SIGNATURE----- # gpg: Signature made Wed 24 Apr 2024 12:52:58 AM PDT # gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653 # gpg: issuer "armbru@redhat.com" # gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [undefined] # gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [undefined] # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653 * tag 'pull-error-2024-04-24' of https://repo.or.cz/qemu/armbru: qapi: Inline and remove QERR_PROPERTY_VALUE_BAD definition qapi: Inline and remove QERR_MIGRATION_ACTIVE definition qapi: Correct error message for 'vcpu_dirty_limit' parameter qapi: Inline and remove QERR_INVALID_PARAMETER_TYPE definition qapi: Inline QERR_INVALID_PARAMETER_TYPE definition (constant value) qapi: Inline and remove QERR_INVALID_PARAMETER definition qapi: Inline and remove QERR_DEVICE_NO_HOTPLUG definition qapi: Inline and remove QERR_DEVICE_HAS_NO_MEDIUM definition qapi: Inline and remove QERR_BUS_NO_HOTPLUG definition error: Drop superfluous #include "qapi/qmp/qerror.h" Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2024-04-24qapi: Inline and remove QERR_MIGRATION_ACTIVE definitionPhilippe Mathieu-Daudé3-4/+4
Address the comment added in commit 4629ed1e98 ("qerror: Finally unused, clean up"), from 2015: /* * These macros will go away, please don't use * in new code, and do not add new ones! */ Mechanical transformation using sed, manually removing the definition in include/qapi/qmp/qerror.h. Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240312141343.3168265-10-armbru@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> [Straightforward conflict with commit aeaafb1e59f (migration: export migration_is_running) resolved]
2024-04-24qapi: Correct error message for 'vcpu_dirty_limit' parameterPhilippe Mathieu-Daudé1-3/+2
QERR_INVALID_PARAMETER_VALUE is defined as: #define QERR_INVALID_PARAMETER_VALUE \ "Parameter '%s' expects %s" The current error is formatted as: "Parameter 'vcpu_dirty_limit' expects is invalid, it must greater then 1 MB/s" Replace by: "Parameter 'vcpu_dirty_limit' must be greater than 1 MB/s" Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240312141343.3168265-9-armbru@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> [New error message corrected, commit message updated accordingly]
2024-04-23Merge tag 'migration-20240423-pull-request' of ↵Richard Henderson13-94/+198
https://gitlab.com/peterx/qemu into staging Migration pull for 9.1 - Het's new test cases for "channels" - Het's fix for a typo for vsock parsing - Cedric's VFIO error report series - Cedric's one more patch for dirty-bitmap error reports - Zhijian's rdma deprecation patch - Yuan's zeropage optimization to fix double faults on anon mem - Zhijian's COLO fix on a crash # -----BEGIN PGP SIGNATURE----- # # iIgEABYKADAWIQS5GE3CDMRX2s990ak7X8zN86vXBgUCZig4HxIccGV0ZXJ4QHJl # ZGhhdC5jb20ACgkQO1/MzfOr1wbQiwD/V5nSJzSuAG4Ra1Fjo+LRG2TT6qk8eNCi # fIytehSw6cYA/0wqarxOF0tr7ikeyhtG3w4xFf44kk6KcPkoVSl1tqoL # =pJmQ # -----END PGP SIGNATURE----- # gpg: Signature made Tue 23 Apr 2024 03:37:19 PM PDT # gpg: using EDDSA key B9184DC20CC457DACF7DD1A93B5FCCCDF3ABD706 # gpg: issuer "peterx@redhat.com" # gpg: Good signature from "Peter Xu <xzpeter@gmail.com>" [unknown] # gpg: aka "Peter Xu <peterx@redhat.com>" [unknown] # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: B918 4DC2 0CC4 57DA CF7D D1A9 3B5F CCCD F3AB D706 * tag 'migration-20240423-pull-request' of https://gitlab.com/peterx/qemu: (26 commits) migration/colo: Fix bdrv_graph_rdlock_main_loop: Assertion `!qemu_in_coroutine()' failed. migration/multifd: solve zero page causing multiple page faults migration: Add Error** argument to add_bitmaps_to_list() migration: Modify ram_init_bitmaps() to report dirty tracking errors migration: Add Error** argument to xbzrle_init() migration: Add Error** argument to ram_state_init() memory: Add Error** argument to the global_dirty_log routines migration: Introduce ram_bitmaps_destroy() memory: Add Error** argument to .log_global_start() handler migration: Add Error** argument to .load_setup() handler migration: Add Error** argument to .save_setup() handler migration: Add Error** argument to qemu_savevm_state_setup() migration: Add Error** argument to vmstate_save() migration: Always report an error in ram_save_setup() migration: Always report an error in block_save_setup() vfio: Always report an error in vfio_save_setup() s390/stattrib: Add Error** argument to set_migrationmode() handler tests/qtest/migration: Fix typo for vsock in SocketAddress_to_str tests/qtest/migration: Add negative tests to validate migration QAPIs tests/qtest/migration: Add multifd_tcp_plain test using list of channels instead of uri ... Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2024-04-23migration/colo: Fix bdrv_graph_rdlock_main_loop: Assertion ↵Li Zhijian1-8/+10
`!qemu_in_coroutine()' failed. bdrv_activate_all() should not be called from the coroutine context, move it to the QEMU thread colo_process_incoming_thread() with the bql_lock protected. The backtrace is as follows: #4 0x0000561af7948362 in bdrv_graph_rdlock_main_loop () at ../block/graph-lock.c:260 #5 0x0000561af7907a68 in graph_lockable_auto_lock_mainloop (x=0x7fd29810be7b) at /patch/to/qemu/include/block/graph-lock.h:259 #6 0x0000561af79167d1 in bdrv_activate_all (errp=0x7fd29810bed0) at ../block.c:6906 #7 0x0000561af762b4af in colo_incoming_co () at ../migration/colo.c:935 #8 0x0000561af7607e57 in process_incoming_migration_co (opaque=0x0) at ../migration/migration.c:793 #9 0x0000561af7adbeeb in coroutine_trampoline (i0=-106876144, i1=22042) at ../util/coroutine-ucontext.c:175 #10 0x00007fd2a5cf21c0 in () at /lib64/libc.so.6 Cc: qemu-stable@nongnu.org Cc: Fabiano Rosas <farosas@suse.de> Closes: https://gitlab.com/qemu-project/qemu/-/issues/2277 Fixes: 2b3912f135 ("block: Mark bdrv_first_blk() and bdrv_is_root_node() GRAPH_RDLOCK") Signed-off-by: Li Zhijian <lizhijian@fujitsu.com> Reviewed-by: Zhang Chen <chen.zhang@intel.com> Tested-by: Zhang Chen <chen.zhang@intel.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240417025634.1014582-1-lizhijian@fujitsu.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration/multifd: solve zero page causing multiple page faultsYuan Liu6-1/+11
Implemented recvbitmap tracking of received pages in multifd. If the zero page appears for the first time in the recvbitmap, this page is not checked and set. If the zero page has already appeared in the recvbitmap, there is no need to check the data but directly set the data to 0, because it is unlikely that the zero page will be migrated multiple times. Signed-off-by: Yuan Liu <yuan1.liu@intel.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240401154110.2028453-2-yuan1.liu@intel.com [peterx: touch up the comment, as the bitmap is used outside postcopy now] Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Add Error** argument to add_bitmaps_to_list()Cédric Le Goater1-18/+16
This allows to report more precise errors in the migration handler dirty_bitmap_save_setup(). Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Signed-off-by: Cédric Le Goater <clg@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Link: https://lore.kernel.org/r/20240329105627.311227-1-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Modify ram_init_bitmaps() to report dirty tracking errorsCédric Le Goater1-16/+11
The .save_setup() handler has now an Error** argument that we can use to propagate errors reported by the .log_global_start() handler. Do that for the RAM. The caller qemu_savevm_state_setup() will store the error under the migration stream for later detection in the migration sequence. Signed-off-by: Cédric Le Goater <clg@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240320064911.545001-15-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Add Error** argument to xbzrle_init()Cédric Le Goater1-12/+10
Since the return value (-ENOMEM) is not exploited, follow the recommendations of qapi/error.h and change it to a bool Signed-off-by: Cédric Le Goater <clg@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240320064911.545001-14-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Add Error** argument to ram_state_init()Cédric Le Goater1-6/+13
Since the return value not exploited, follow the recommendations of qapi/error.h and change it to a bool Signed-off-by: Cédric Le Goater <clg@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240320064911.545001-13-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23memory: Add Error** argument to the global_dirty_log routinesCédric Le Goater2-4/+32
Now that the log_global*() handlers take an Error** parameter and return a bool, do the same for memory_global_dirty_log_start() and memory_global_dirty_log_stop(). The error is reported in the callers for now and it will be propagated in the call stack in the next changes. To be noted a functional change in ram_init_bitmaps(), if the dirty pages logger fails to start, there is no need to synchronize the dirty pages bitmaps. colo_incoming_start_dirty_log() could be modified in a similar way. Cc: Stefano Stabellini <sstabellini@kernel.org> Cc: Anthony Perard <anthony.perard@citrix.com> Cc: Paul Durrant <paul@xen.org> Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: David Hildenbrand <david@redhat.com> Cc: Hyman Huang <yong.huang@smartx.com> Signed-off-by: Cédric Le Goater <clg@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Acked-by: Peter Xu <peterx@redhat.com> Link: https://lore.kernel.org/r/20240320064911.545001-12-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Introduce ram_bitmaps_destroy()Cédric Le Goater1-7/+15
We will use it in ram_init_bitmaps() to clear the allocated bitmaps when support for error reporting is added to memory_global_dirty_log_start(). Signed-off-by: Cédric Le Goater <clg@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240320064911.545001-11-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Add Error** argument to .load_setup() handlerCédric Le Goater2-5/+9
This will be useful to report errors at a higher level, mostly in VFIO today. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Cédric Le Goater <clg@redhat.com> Link: https://lore.kernel.org/r/20240320064911.545001-9-clg@redhat.com [peterx: drop comment for ERRP_GUARD, per Markus] Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Add Error** argument to .save_setup() handlerCédric Le Goater4-20/+16
The purpose is to record a potential error in the migration stream if qemu_savevm_state_setup() fails. Most of the current .save_setup() handlers can be modified to use the Error argument instead of managing their own and calling locally error_report(). Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Harsh Prateek Bora <harshpb@linux.ibm.com> Cc: Halil Pasic <pasic@linux.ibm.com> Cc: Thomas Huth <thuth@redhat.com> Cc: Eric Blake <eblake@redhat.com> Cc: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Cc: John Snow <jsnow@redhat.com> Cc: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Signed-off-by: Cédric Le Goater <clg@redhat.com> Link: https://lore.kernel.org/r/20240320064911.545001-8-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Add Error** argument to qemu_savevm_state_setup()Cédric Le Goater3-14/+47
This prepares ground for the changes coming next which add an Error** argument to the .save_setup() handler. Callers of qemu_savevm_state_setup() now handle the error and fail earlier setting the migration state from MIGRATION_STATUS_SETUP to MIGRATION_STATUS_FAILED. In qemu_savevm_state(), move the cleanup to preserve the error reported by .save_setup() handlers. Since the previous behavior was to ignore errors at this step of migration, this change should be examined closely to check that cleanups are still correctly done. Signed-off-by: Cédric Le Goater <clg@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Link: https://lore.kernel.org/r/20240320064911.545001-7-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Add Error** argument to vmstate_save()Cédric Le Goater1-10/+16
This will prepare ground for future changes adding an Error** argument to qemu_savevm_state_setup(). Reviewed-by: Prasad Pandit <pjp@fedoraproject.org> Signed-off-by: Cédric Le Goater <clg@redhat.com> Link: https://lore.kernel.org/r/20240320064911.545001-6-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Always report an error in ram_save_setup()Cédric Le Goater1-1/+10
This will prepare ground for future changes adding an Error** argument to the save_setup() handler. We need to make sure that on failure, ram_save_setup() sets a new error. Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Cédric Le Goater <clg@redhat.com> Link: https://lore.kernel.org/r/20240320064911.545001-5-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-23migration: Always report an error in block_save_setup()Cédric Le Goater1-5/+15
This will prepare ground for future changes adding an Error** argument to the save_setup() handler. We need to make sure that on failure, block_save_setup() always sets a new error. Cc: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Cédric Le Goater <clg@redhat.com> Link: https://lore.kernel.org/r/20240320064911.545001-4-clg@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-04-18colo: move stubs out of stubs/Paolo Bonzini2-0/+38
Since the colo stubs are needed exactly when the build options are not enabled, move them together with the code they stub. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-ID: <20240408155330.522792-16-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-04-07migration/postcopy: ensure preempt channel is ready before loading statesWei Wang1-0/+21
Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event hasn't been dispatched. Cc: qemu-stable <qemu-stable@nongnu.org> Fixes: 9358982744 ("migration: Send requested page directly in rp-return thread") Originally-by: Lei Wang <lei4.wang@intel.com> Link: https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@intel.com/T/ Signed-off-by: Lei Wang <lei4.wang@intel.com> Signed-off-by: Wei Wang <wei.w.wang@intel.com> Link: https://lore.kernel.org/r/20240405034056.23933-1-wei.w.wang@intel.com [peterx: add a todo section, add Fixes and copy stable for 8.0+] Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-31migration/postcopy: Ensure postcopy_start() sets errp if it failsAvihai Horon1-0/+8
There are several places where postcopy_start() fails without setting errp. This can cause a null pointer de-reference, as in case of error, the caller of postcopy_start() copies/prints the error set in errp. Fix it by setting errp in all of postcopy_start() error paths. Cc: qemu-stable <qemu-stable@nongnu.org> Fixes: 908927db28ea ("migration: Update error description whenever migration fails") Signed-off-by: Avihai Horon <avihaih@nvidia.com> Reviewed-by: Cédric Le Goater <clg@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Link: https://lore.kernel.org/r/20240328140252.16756-3-avihaih@nvidia.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-31migration: Set migration error in migration_completion()Avihai Horon1-0/+10
After commit 9425ef3f990a ("migration: Use migrate_has_error() in close_return_path_on_source()"), close_return_path_on_source() assumes that migration error is set if an error occurs during migration. This may not be true if migration errors in migration_completion(). For example, if qemu_savevm_state_complete_precopy() errors, migration error will not be set. This in turn, will cause a migration hang bug, similar to the bug that was fixed by commit 22b04245f0d5 ("migration: Join the return path thread before releasing to_dst_file"), as shutdown() will not be issued for the return-path channel. Fix it by ensuring migration error is set in case of error in migration_completion(). Signed-off-by: Avihai Horon <avihaih@nvidia.com> Reviewed-by: Peter Xu <peterx@redhat.com> Fixes: 9425ef3f990a ("migration: Use migrate_has_error() in close_return_path_on_source()") Acked-by: Cédric Le Goater <clg@redhat.com> Link: https://lore.kernel.org/r/20240328140252.16756-2-avihaih@nvidia.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-22migration/multifd: Fix clearing of mapped-ram zero pagesFabiano Rosas1-2/+1
When the zero page detection is done in the multifd threads, we need to iterate the second part of the pages->offset array and clear the file bitmap for each zero page. The piece of code we merged to do that is wrong. The reason this has passed all the tests is because the bitmap is initialized with zeroes already, so clearing the bits only really has an effect during live migration and when a data page goes from having data to no data. Fixes: 303e6f54f9 ("migration/multifd: Implement zero page transmission on the multifd thread.") Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240321201242.6009-1-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-22migration/postcopy: Fix high frequency syncPeter Xu1-4/+3
With current code base I can observe extremely high sync count during precopy, as long as one enables postcopy-ram=on before switchover to postcopy. To provide some context of when QEMU decides to do a full sync: it checks must_precopy (which implies "data must be sent during precopy phase"), and as long as it is lower than the threshold size we calculated (out of bandwidth and expected downtime) QEMU will kick off the slow/exact sync. However, when postcopy is enabled (even if still during precopy phase), RAM only reports all pages as can_postcopy, and report must_precopy==0. Then "must_precopy <= threshold_size" mostly always triggers and enforces a slow sync for every call to migration_iteration_run() when postcopy is enabled even if not used. That is insane. It turns out it was a regress bug introduced in the previous refactoring in 8.0 as reported by Nina [1]: (a) c8df4a7aef ("migration: Split save_live_pending() into state_pending_*") Then a workaround patch is applied at the end of release (8.0-rc4) to fix it: (b) 28ef5339c3 ("migration: fix ram_state_pending_exact()") However that "workaround" was overlooked when during the cleanup in this 9.0 release in this commit.. (c) b0504edd40 ("migration: Drop unnecessary check in ram's pending_exact()") Then the issue was re-exposed as reported by Nina [1]. The problem with (b) is that it only fixed the case for RAM, rather than all the rest of iterators. Here a slow sync should only be required if all dirty data (precopy+postcopy) is less than the threshold_size that QEMU calculated. It is even debatable whether a sync is needed when switched to postcopy. Currently ram_state_pending_exact() will be mostly noop if switched to postcopy, and that logic seems to apply too for all the rest of iterators, as sync dirty bitmap during a postcopy doesn't make much sense. However let's leave such change for later, as we're in rc phase. So rather than reusing commit (b), this patch provides the complete fix for all iterators. When at it, cleanup a little bit on the lines around. [1] https://gitlab.com/qemu-project/qemu/-/issues/1565 Reported-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com> Fixes: b0504edd40 ("migration: Drop unnecessary check in ram's pending_exact()") Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240320214453.584374-1-peterx@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-22migration: Revert mapped-ram multifd support to fd: URIFabiano Rosas5-84/+8
This reverts commit decdc76772c453ff1444612e910caa0d45cd8eac in full and also the relevant migration-tests from 7a09f092834641b7a793d50a3a261073bbb404a6. After the addition of the new QAPI-based migration address API in 8.2 we've been converting an "fd:" URI into a SocketAddress, missing the fact that the "fd:" syntax could also be used for a plain file instead of a socket. This is a problem because the SocketAddress is part of the API, so we're effectively asking users to create a "socket" channel to pass in a plain file. The easiest way to fix this situation is to deprecate the usage of both SocketAddress and "fd:" when used with a plain file for migration. Since this has been possible since 8.2, we can wait until 9.1 to deprecate it. For 9.0, however, we should avoid adding further support to migration to a plain file using the old "fd:" syntax or the new SocketAddress API, and instead require the usage of either the old-style "file:" URI or the FileMigrationArgs::filename field of the new API with the "/dev/fdset/NN" syntax, both of which are already supported. Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240319210941.1907-1-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-18Merge tag 'migration-20240317-pull-request' of ↵Peter Maydell5-54/+84
https://gitlab.com/peterx/qemu into staging Migration pull for 9.0-rc0 - Nicholas/Phil's fix on migration corruption / inconsistent for tcg - Cedric's fix on block migration over n_sectors==0 - Steve's CPR reboot documentation page - Fabiano's misc fixes on mapped-ram (IOC leak, dup() errors, fd checks, fd use race, etc.) # -----BEGIN PGP SIGNATURE----- # # iIgEABYKADAWIQS5GE3CDMRX2s990ak7X8zN86vXBgUCZfdZEhIccGV0ZXJ4QHJl # ZGhhdC5jb20ACgkQO1/MzfOr1wa+1AEA0+f7nCssvsILvCY9KifYO+OUJsLodUuQ # JW0JBz+1iPMA+wSiyIVl2Xg78Q97nJxv71UJf+1cDJENA5EMmXMnxmYK # =SLnA # -----END PGP SIGNATURE----- # gpg: Signature made Sun 17 Mar 2024 20:56:50 GMT # gpg: using EDDSA key B9184DC20CC457DACF7DD1A93B5FCCCDF3ABD706 # gpg: issuer "peterx@redhat.com" # gpg: Good signature from "Peter Xu <xzpeter@gmail.com>" [marginal] # gpg: aka "Peter Xu <peterx@redhat.com>" [marginal] # gpg: WARNING: This key is not certified with sufficiently trusted signatures! # gpg: It is not certain that the signature belongs to the owner. # Primary key fingerprint: B918 4DC2 0CC4 57DA CF7D D1A9 3B5F CCCD F3AB D706 * tag 'migration-20240317-pull-request' of https://gitlab.com/peterx/qemu: migration/multifd: Duplicate the fd for the outgoing_args migration/multifd: Ensure we're not given a socket for file migration migration: Fix iocs leaks during file and fd migration migration: cpr-reboot documentation migration: Skip only empty block devices physmem: Fix migration dirty bitmap coherency with TCG memory access physmem: Factor cpu_physical_memory_dirty_bits_cleared() out physmem: Expose tlb_reset_dirty_range_all() migration: Fix error handling after dup in file migration io: Introduce qio_channel_file_new_dupfd Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-03-15migration/multifd: Duplicate the fd for the outgoing_argsFabiano Rosas1-3/+12
We currently store the file descriptor used during the main outgoing channel creation to use it again when creating the multifd channels. Since this fd is used for the first iochannel, there's risk that the QIOChannel gets freed and the fd closed while outgoing_args.fd still has it available. This could lead to an fd-reuse bug. Duplicate the outgoing_args fd to avoid this issue. Suggested-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Peter Xu <peterx@redhat.com> Link: https://lore.kernel.org/r/20240315032040.7974-3-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-15migration/multifd: Ensure we're not given a socket for file migrationFabiano Rosas3-3/+18
When doing migration using the fd: URI, QEMU will fetch the file descriptor passed in via the monitor at fd_start_outgoing|incoming_migration(), which means the checks at migration_channels_and_transport_compatible() happen too soon and we don't know at that point whether the FD refers to a plain file or a socket. For this reason, we've been allowing a migration channel of type SOCKET_ADDRESS_TYPE_FD to pass the initial verifications in scenarios where the socket migration is not supported, such as with fd + multifd. The commit decdc76772 ("migration/multifd: Add mapped-ram support to fd: URI") was supposed to add a second check prior to starting migration to make sure a socket fd is not passed instead of a file fd, but failed to do so. Add the missing verification and update the comment explaining this situation which is currently incorrect. Fixes: decdc76772 ("migration/multifd: Add mapped-ram support to fd: URI") Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Peter Xu <peterx@redhat.com> Link: https://lore.kernel.org/r/20240315032040.7974-2-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-14migration: Fix iocs leaks during file and fd migrationFabiano Rosas3-42/+46
The memory for the io channels is being leaked in three different ways during file migration: 1) if the offset check fails we never drop the ioc reference; 2) we allocate an extra channel for no reason; 3) if multifd is enabled but channel creation fails when calling dup(), we leave the previous channels around along with the glib polling; Fix all issues by restructuring the code to first allocate the channels and only register the watches when all channels have been created. For multifd, the file and fd migrations can share code because both are backed by a QIOChannelFile. For the non-multifd case, the fd needs to be separate because it is backed by a QIOChannelSocket. Fixes: 2dd7ee7a51 ("migration/multifd: Add incoming QIOChannelFile support") Fixes: decdc76772 ("migration/multifd: Add mapped-ram support to fd: URI") Reported-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240313212824.16974-2-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-13migration: Skip only empty block devicesCédric Le Goater1-1/+4
The block .save_setup() handler calls a helper routine init_blk_migration() which builds a list of block devices to take into account for migration. When one device is found to be empty (sectors == 0), the loop exits and all the remaining devices are ignored. This is a regression introduced when bdrv_iterate() was removed. Change that by skipping only empty devices. Cc: Markus Armbruster <armbru@redhat.com> Cc: qemu-stable <qemu-stable@nongnu.org> Suggested-by: Kevin Wolf <kwolf@redhat.com> Fixes: fea68bb6e9fa ("block: Eliminate bdrv_iterate(), use bdrv_next()") Signed-off-by: Cédric Le Goater <clg@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Link: https://lore.kernel.org/r/20240312120431.550054-1-clg@redhat.com [peterx: fix "Suggested-by:"] Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-12migration: Fix error handling after dup in file migrationFabiano Rosas2-12/+11
The file migration code was allowing a possible -1 from a failed call to dup() to propagate into the new QIOFileChannel::fd before checking for validity. Coverity doesn't like that, possibly due to the the lseek(-1, ...) call that would ensue before returning from the channel creation routine. Use the newly introduced qio_channel_file_dupfd() to properly check the return of dup() before proceeding. Fixes: CID 1539961 Fixes: CID 1539965 Fixes: CID 1539960 Fixes: 2dd7ee7a51 ("migration/multifd: Add incoming QIOChannelFile support") Fixes: decdc76772 ("migration/multifd: Add mapped-ram support to fd: URI") Reported-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com> Link: https://lore.kernel.org/r/20240311233335.17299-3-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-12Merge tag 'pull-request-2024-03-12' of https://gitlab.com/thuth/qemu into ↵Peter Maydell3-4/+6
staging * Add missing ERRP_GUARD() statements in functions that need it * Prefer fast cpu_env() over slower CPU QOM cast macro # -----BEGIN PGP SIGNATURE----- # # iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmXwPhYRHHRodXRoQHJl # ZGhhdC5jb20ACgkQLtnXdP5wLbWHvBAAgKx5LHFjz3xREVA+LkDTQ49mz0lK3s32 # SGvNlIHjiaDGVttVYhVC4sinBWUruG4Lyv/2QN72OJBzn6WUsEUQE3KPH1d7Y3/s # wS9X7mj70n4kugWJqeIJP5AXSRasHmWoQ4QJLVQRJd6+Eb9jqwep0x7bYkI1de6D # bL1Q7bIfkFeNQBXaiPWAm2i+hqmT4C1r8HEAGZIjAsMFrjy/hzBEjNV+pnh6ZSq9 # Vp8BsPWRfLU2XHm4WX0o8d89WUMAfUGbVkddEl/XjIHDrUD+Zbd1HAhLyfhsmrnE # jXIwSzm+ML1KX4MoF5ilGtg8Oo0gQDEBy9/xck6G0HCm9lIoLKlgTxK9glr2vdT8 # yxZmrM9Hder7F9hKKxmb127xgU6AmL7rYmVqsoQMNAq22D6Xr4UDpgFRXNk2/wO6 # zZZBkfZ4H4MpZXbd/KJpXvYH5mQA4IpkOy8LJdE+dbcHX7Szy9ksZdPA+Z10hqqf # zqS13qTs3abxymy2Q/tO3hPKSJCk1+vCGUkN60Wm+9VoLWGoU43qMc7gnY/pCS7m # 0rFKtvfwFHhokX1orK0lP/ppVzPv/5oFIeK8YDY9if+N+dU2LCwVZHIuf2/VJPRq # wmgH2vAn3JDoRKPxTGX9ly6AMxuZaeP92qBTOPap0gDhihYzIpaCq9ecEBoTakI7 # tdFhV0iRr08= # =NiP4 # -----END PGP SIGNATURE----- # gpg: Signature made Tue 12 Mar 2024 11:35:50 GMT # gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5 # gpg: issuer "thuth@redhat.com" # gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full] # gpg: aka "Thomas Huth <thuth@redhat.com>" [full] # gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full] # gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown] # Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5 * tag 'pull-request-2024-03-12' of https://gitlab.com/thuth/qemu: (55 commits) user: Prefer fast cpu_env() over slower CPU QOM cast macro target/xtensa: Prefer fast cpu_env() over slower CPU QOM cast macro target/tricore: Prefer fast cpu_env() over slower CPU QOM cast macro target/sparc: Prefer fast cpu_env() over slower CPU QOM cast macro target/sh4: Prefer fast cpu_env() over slower CPU QOM cast macro target/rx: Prefer fast cpu_env() over slower CPU QOM cast macro target/ppc: Prefer fast cpu_env() over slower CPU QOM cast macro target/openrisc: Prefer fast cpu_env() over slower CPU QOM cast macro target/nios2: Prefer fast cpu_env() over slower CPU QOM cast macro target/mips: Prefer fast cpu_env() over slower CPU QOM cast macro target/microblaze: Prefer fast cpu_env() over slower CPU QOM cast macro target/m68k: Prefer fast cpu_env() over slower CPU QOM cast macro target/loongarch: Prefer fast cpu_env() over slower CPU QOM cast macro target/i386/hvf: Use CPUState typedef target/hexagon: Prefer fast cpu_env() over slower CPU QOM cast macro target/cris: Prefer fast cpu_env() over slower CPU QOM cast macro target/avr: Prefer fast cpu_env() over slower CPU QOM cast macro target/alpha: Prefer fast cpu_env() over slower CPU QOM cast macro target: Replace CPU_GET_CLASS(cpu -> obj) in cpu_reset_hold() handler bulk: Call in place single use cpu_env() ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-03-12bulk: Access existing variables initialized to &S->F when availablePhilippe Mathieu-Daudé1-1/+1
When a variable is initialized to &struct->field, use it in place. Rationale: while this makes the code more concise, this also helps static analyzers. Mechanical change using the following Coccinelle spatch script: @@ type S, F; identifier s, m, v; @@ S *s; ... F *v = &s->m; <+... - &s->m + v ...+> Inspired-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-ID: <20240129164514.73104-2-philmd@linaro.org> Acked-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Anthony PERARD <anthony.perard@citrix.com> [thuth: Dropped hunks that need a rebase, and fixed sizeof() in pmu_realize()] Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-03-12error: Move ERRP_GUARD() to the beginning of the functionZhao Liu2-3/+3
Since the commit 05e385d2a9 ("error: Move ERRP_GUARD() to the beginning of the function"), there are new codes that don't put ERRP_GUARD() at the beginning of the functions. As stated in the commit 05e385d2a9: "include/qapi/error.h advises to put ERRP_GUARD() right at the beginning of the function, because only then can it guard the whole function.", so clean up the few spots disregarding the advice. Inspired-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240312060337.3240965-1-zhao1.liu@linux.intel.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-03-12migration/option: Fix missing ERRP_GUARD() for error_prepend()Zhao Liu1-0/+2
As the comment in qapi/error, passing @errp to error_prepend() requires ERRP_GUARD(): * = Why, when and how to use ERRP_GUARD() = * * Without ERRP_GUARD(), use of the @errp parameter is restricted: ... * - It should not be passed to error_prepend(), error_vprepend() or * error_append_hint(), because that doesn't work with &error_fatal. * ERRP_GUARD() lifts these restrictions. * * To use ERRP_GUARD(), add it right at the beginning of the function. * @errp can then be used without worrying about the argument being * NULL or &error_fatal. ERRP_GUARD() could avoid the case when @errp is &error_fatal, the user can't see this additional information, because exit() happens in error_setg earlier than information is added [1]. The migrate_params_check() passes @errp to error_prepend() without ERRP_GUARD(), and it could be called from migration_object_init(), where the passed @errp points to @error_fatal. Therefore, the error message echoed in error_prepend() will be lost because of the above issue. To fix this, add missing ERRP_GUARD() at the beginning of this function. [1]: Issue description in the commit message of commit ae7c80a7bd73 ("error: New macro ERRP_GUARD()"). Cc: Peter Xu <peterx@redhat.com> Cc: Fabiano Rosas <farosas@suse.de> Signed-off-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Acked-by: Peter Xu <peterx@redhat.com> Message-ID: <20240311033822.3142585-28-zhao1.liu@linux.intel.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-03-11migration/multifd: Enable multifd zero page checking by default.Hao Xiang1-1/+1
1. Set default "zero-page-detection" option to "multifd". Now zero page checking can be done in the multifd threads and this becomes the default configuration. 2. Handle migration QEMU9.0 -> QEMU8.2 compatibility. We provide backward compatibility where zero page checking is done from the migration main thread. Signed-off-by: Hao Xiang <hao.xiang@bytedance.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Peter Xu <peterx@redhat.com> Link: https://lore.kernel.org/r/20240311180015.3359271-7-hao.xiang@linux.dev Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration/multifd: Implement ram_save_target_page_multifd to handle multifd ↵Hao Xiang1-9/+29
version of MigrationOps::ram_save_target_page. 1. Add a dedicated handler for MigrationOps::ram_save_target_page in multifd live migration. 2. Refactor ram_save_target_page_legacy so that the legacy and multifd handlers don't have internal functions calling into each other. Signed-off-by: Hao Xiang <hao.xiang@bytedance.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Message-Id: <20240226195654.934709-4-hao.xiang@bytedance.com> Link: https://lore.kernel.org/r/20240311180015.3359271-6-hao.xiang@linux.dev Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration/multifd: Implement zero page transmission on the multifd thread.Hao Xiang8-30/+221
1. Add zero_pages field in MultiFDPacket_t. 2. Implements the zero page detection and handling on the multifd threads for non-compression, zlib and zstd compression backends. 3. Added a new value 'multifd' in ZeroPageDetection enumeration. 4. Adds zero page counters and updates multifd send/receive tracing format to track the newly added counters. Signed-off-by: Hao Xiang <hao.xiang@bytedance.com> Acked-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240311180015.3359271-5-hao.xiang@linux.dev Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration/multifd: Add new migration option zero-page-detection.Hao Xiang4-0/+35
This new parameter controls where the zero page checking is running. 1. If this parameter is set to 'legacy', zero page checking is done in the migration main thread. 2. If this parameter is set to 'none', zero page checking is disabled. Signed-off-by: Hao Xiang <hao.xiang@bytedance.com> Reviewed-by: Peter Xu <peterx@redhat.com> Acked-by: Markus Armbruster <armbru@redhat.com> Link: https://lore.kernel.org/r/20240311180015.3359271-4-hao.xiang@linux.dev Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration/multifd: Allow clearing of the file_bmap from multifdFabiano Rosas3-4/+9
We currently only need to clear the mapped-ram file bitmap from the migration thread during save_zero_page. We're about to add support for zero page detection on the multifd thread, so allow ramblock_set_file_bmap_atomic() to also clear the bits. Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240311180015.3359271-3-hao.xiang@linux.dev Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration/multifd: Allow zero pages in file migrationFabiano Rosas1-1/+1
Currently, it's an error to have no data pages in the multifd file migration because zero page detection is done in the migration thread and zero pages don't reach multifd. This is enforced with the pages->num assert. We're about to add zero page detection on the multifd thread. Fix the file_write_ramblock_iov() to stop considering p->iovs_num=0 an error. Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240311180015.3359271-2-hao.xiang@linux.dev Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration: purge MigrationState from public interfaceSteve Sistare1-0/+6
Move remaining MigrationState references from the public file misc.h to the private file migration.h. Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1710179338-294359-12-git-send-email-steven.sistare@oracle.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration: delete unused accessorsSteve Sistare1-10/+0
Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1710179338-294359-11-git-send-email-steven.sistare@oracle.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration: privatize colo interfacesSteve Sistare1-6/+11
Remove private migration interfaces from net/colo-compare.c and push them to migration/colo.c. Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1710179338-294359-10-git-send-email-steven.sistare@oracle.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration: migration_file_set_errorSteve Sistare1-0/+11
Define and export migration_file_set_error to eliminate a dependency on MigrationState. Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1710179338-294359-9-git-send-email-steven.sistare@oracle.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration: migration_is_deviceSteve Sistare1-0/+7
Define and export migration_is_device to eliminate a dependency on MigrationState. Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1710179338-294359-8-git-send-email-steven.sistare@oracle.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration: migration_thread_is_selfSteve Sistare1-0/+7
Define and export migration_thread_is_self to eliminate a dependency on MigrationState. Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1710179338-294359-7-git-send-email-steven.sistare@oracle.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration: export vcpu_dirty_limit_periodSteve Sistare1-0/+7
Define and export vcpu_dirty_limit_period to eliminate a dependency on MigrationState. Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1710179338-294359-6-git-send-email-steven.sistare@oracle.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration: export migration_is_runningSteve Sistare4-9/+9
Delete the MigrationState parameter from migration_is_running and move it to the public API in misc.h. Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1710179338-294359-5-git-send-email-steven.sistare@oracle.com Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-11migration: export migration_is_activeSteve Sistare1-4/+6
Delete the MigrationState parameter from migration_is_active so it can be exported and used without including migration.h. Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1710179338-294359-4-git-send-email-steven.sistare@oracle.com Signed-off-by: Peter Xu <peterx@redhat.com>
generated by cgit v1.1 at 2025-09-29 09:47:34 +0000