aboutsummaryrefslogtreecommitdiff
path: root/migration
AgeCommit message (Collapse)AuthorFilesLines
2024-09-03migration/multifd: Replace multifd_send_state->pages with client dataFabiano Rosas3-34/+50
Multifd currently has a simple scheduling mechanism that distributes work to the various channels by keeping storage space within each channel and an extra space that is given to the client. Each time the client fills the space with data and calls into multifd, that space is given to the next idle channel and a free storage space is taken from the channel and given to client for the next iteration. This means we always need (#multifd_channels + 1) memory slots to operate multifd. This is fine, except that the presence of this one extra memory slot doesn't allow different types of payloads to be processed at the same time in different channels, i.e. the data type of multifd_send_state->pages needs to be the same as p->pages. For each new data type different from MultiFDPage_t that is to be handled, this logic would need to be duplicated by adding new fields to multifd_send_state, to the channels and to multifd_send_pages(). Fix this situation by moving the extra slot into the client and using only the generic type MultiFDSendData in the multifd core. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Don't send ram data during SYNCFabiano Rosas1-3/+10
Skip saving and loading any ram data in the packet in the case of a SYNC. This fixes a shortcoming of the current code which requires a reset of the MultiFDPages_t fields right after the previous pending_job finishes, otherwise the very next job might be a SYNC and multifd_send_fill_packet() will put the stale values in the packet. By not calling multifd_ram_fill_packet(), we can stop resetting MultiFDPages_t in the multifd core and leave that to the client code. Actually moving the reset function is not yet done because pages->num==0 is used by the client code to determine whether the MultiFDPages_t needs to be flushed. The subsequent patches will replace that with a generic flag that is not dependent on MultiFDPages_t. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Isolate ram pages packet dataFabiano Rosas2-41/+63
While we cannot yet disentangle the multifd packet from page data, we can make the code a bit cleaner by setting the page-related fields in a separate function. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Remove total pages tracingFabiano Rosas3-20/+4
The total_normal_pages and total_zero_pages elements are used only for the end tracepoints of the multifd threads. These are not super useful since they record per-channel numbers and are just the sum of all the pages that are transmitted per-packet, for which we already have tracepoints. Remove the totals from the tracing. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Move pages accounting into multifd_send_zero_page_detect()Fabiano Rosas2-3/+6
All references to pages are being removed from the multifd worker threads in order to allow multifd to deal with different payload types. multifd_send_zero_page_detect() is called by all multifd migration paths that deal with pages and is the last spot where zero pages and normal page amounts are adjusted. Move the pages accounting into that function. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Replace p->pages with an union pointerFabiano Rosas7-47/+57
We want multifd to be able to handle more types of data than just ram pages. To start decoupling multifd from pages, replace p->pages (MultiFDPages_t) with the new type MultiFDSendData that hides the client payload inside an union. The general idea here is to isolate functions that *need* to handle MultiFDPages_t and move them in the future to multifd-ram.c, while multifd.c will stay with only the core functions that handle MultiFDSendData/MultiFDRecvData. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Make MultiFDPages_t:offset a flexible array memberFabiano Rosas2-9/+14
We're about to use MultiFDPages_t from inside the MultiFDSendData payload union, which means we cannot have pointers to allocated data inside the pages structure, otherwise we'd lose the reference to that memory once another payload type touches the union. Move the offset array into the end of the structure and turn it into a flexible array member, so it is allocated along with the rest of MultiFDSendData in the next patches. Note that other pointers, such as the ramblock pointer are still fine as long as the storage for them is not owned by the migration code and can be correctly released at some point. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Introduce MultiFDSendDataFabiano Rosas1-0/+26
Add a new data structure to replace p->pages in the multifd channel. This new structure will hide the multifd payload type behind an union, so we don't need to add a new field to the channel each time we want to handle a different data type. This also allow us to keep multifd_send_pages() as is, without needing to complicate the pointer switching. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Pass in MultiFDPages_t to file_write_ramblock_iovFabiano Rosas3-3/+4
We want to stop dereferencing 'pages' so it can be replaced by an opaque pointer in the next patches. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Remove pages->allocatedFabiano Rosas2-6/+2
This value never changes and is always the same as page_count. We don't need a copy of it per-channel plus one in the extra slot. Remove it. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Inline page_size and page_countFabiano Rosas7-55/+71
The MultiFD*Params structures are for per-channel data. Constant values should not be there because that needlessly wastes cycles and storage. The page_size and page_count fall into this category so move them inline in multifd.h. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration/multifd: Reduce access to p->pagesFabiano Rosas5-15/+19
I'm about to replace the p->pages pointer with an opaque pointer, so do a cleanup now to reduce direct accesses to p->page, which makes the next diffs cleaner. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-09-03migration: delete unused parameter misSteve Sistare1-6/+4
Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-08-20migration/multifd: Free MultiFDRecvParams::dataPeter Maydell1-0/+2
In multifd_recv_setup() we allocate (among other things) * a MultiFDRecvData struct to multifd_recv_state::data * a MultiFDRecvData struct to each multfd_recv_state->params[i].data (Then during execution we might swap these pointers around.) But in multifd_recv_cleanup() we free multifd_recv_state->data in multifd_recv_cleanup_state() but we don't ever free the multifd_recv_state->params[i].data. This results in a memory leak reported by LeakSanitizer: (cd build/asan && \ ASAN_OPTIONS="fast_unwind_on_malloc=0:strip_path_prefix=/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/../../" \ QTEST_QEMU_BINARY=./qemu-system-x86_64 \ ./tests/qtest/migration-test --tap -k -p /x86_64/migration/multifd/file/mapped-ram ) [...] Direct leak of 72 byte(s) in 3 object(s) allocated from: #0 0x561cc0afcfd8 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-x86_64+0x218efd8) (BuildId: be72e086d4e47b172b0a72779972213fd9916466) #1 0x7f89d37acc50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x561cc1e9c83c in multifd_recv_setup migration/multifd.c:1606:19 #3 0x561cc1e68618 in migration_ioc_process_incoming migration/migration.c:972:9 #4 0x561cc1e3ac59 in migration_channel_process_incoming migration/channel.c:45:9 #5 0x561cc1e4fa0b in file_accept_incoming_migration migration/file.c:132:5 #6 0x561cc30f2c0c in qio_channel_fd_source_dispatch io/channel-watch.c:84:12 #7 0x7f89d37a3c43 in g_main_dispatch debian/build/deb/../../../glib/gmain.c:3419:28 #8 0x7f89d37a3c43 in g_main_context_dispatch debian/build/deb/../../../glib/gmain.c:4137:7 #9 0x561cc3b21659 in glib_pollfds_poll util/main-loop.c:287:9 #10 0x561cc3b1ff93 in os_host_main_loop_wait util/main-loop.c:310:5 #11 0x561cc3b1fb5c in main_loop_wait util/main-loop.c:589:11 #12 0x561cc1da2917 in qemu_main_loop system/runstate.c:801:9 #13 0x561cc3796c1c in qemu_default_main system/main.c:37:14 #14 0x561cc3796c67 in main system/main.c:48:12 #15 0x7f89d163bd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 #16 0x7f89d163be3f in __libc_start_main csu/../csu/libc-start.c:392:3 #17 0x561cc0a79fa4 in _start (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-x86_64+0x210bfa4) (BuildId: be72e086d4e47b172b0a72779972213fd9916466) Direct leak of 24 byte(s) in 1 object(s) allocated from: #0 0x561cc0afcfd8 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-x86_64+0x218efd8) (BuildId: be72e086d4e47b172b0a72779972213fd9916466) #1 0x7f89d37acc50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x561cc1e9bed9 in multifd_recv_setup migration/multifd.c:1588:32 #3 0x561cc1e68618 in migration_ioc_process_incoming migration/migration.c:972:9 #4 0x561cc1e3ac59 in migration_channel_process_incoming migration/channel.c:45:9 #5 0x561cc1e4fa0b in file_accept_incoming_migration migration/file.c:132:5 #6 0x561cc30f2c0c in qio_channel_fd_source_dispatch io/channel-watch.c:84:12 #7 0x7f89d37a3c43 in g_main_dispatch debian/build/deb/../../../glib/gmain.c:3419:28 #8 0x7f89d37a3c43 in g_main_context_dispatch debian/build/deb/../../../glib/gmain.c:4137:7 #9 0x561cc3b21659 in glib_pollfds_poll util/main-loop.c:287:9 #10 0x561cc3b1ff93 in os_host_main_loop_wait util/main-loop.c:310:5 #11 0x561cc3b1fb5c in main_loop_wait util/main-loop.c:589:11 #12 0x561cc1da2917 in qemu_main_loop system/runstate.c:801:9 #13 0x561cc3796c1c in qemu_default_main system/main.c:37:14 #14 0x561cc3796c67 in main system/main.c:48:12 #15 0x7f89d163bd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 #16 0x7f89d163be3f in __libc_start_main csu/../csu/libc-start.c:392:3 #17 0x561cc0a79fa4 in _start (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qemu-system-x86_64+0x210bfa4) (BuildId: be72e086d4e47b172b0a72779972213fd9916466) SUMMARY: AddressSanitizer: 96 byte(s) leaked in 4 allocation(s). Free the params[i].data too. Cc: qemu-stable@nongnu.org Fixes: d117ed0699d41 ("migration/multifd: Allow receiving pages without packets") Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-08-16savevm: Fix load_snapshot error path crashNicholas Piggin1-0/+1
An error path missed setting *errp, which can cause a NULL deref. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Message-Id: <20240813050638.446172-11-npiggin@gmail.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20240813202329.1237572-19-alex.bennee@linaro.org>
2024-08-02migration/multifd: Fix multifd_send_setup cleanup when channel creation failsFabiano Rosas1-11/+15
When a channel fails to create, the code currently just returns. This is wrong for two reasons: 1) Channel n+1 will not get to initialize it's semaphores, leading to an assert when terminate_threads tries to post to it: qemu-system-x86_64: ../util/qemu-thread-posix.c:92: qemu_mutex_lock_impl: Assertion `mutex->initialized' failed. 2) (theoretical) If channel n-1 already started creation it will defeat the purpose of the channels_created logic which is in place to avoid migrate_fd_cleanup() to run while channels are still being created. This cannot really happen today because the current failure cases for multifd_new_send_channel_create() are all synchronous, resulting from qio_channel_file_new_path() getting a bad filename. This would hit all channels equally. But I don't want to set a trap for future people, so have all channels try to create (even if failing), and only fail after the channels_created semaphore has been posted. While here, remove the error_report_err call. There's one already at migrate_fd_cleanup later on. Cc: qemu-stable@nongnu.org Reported-by: Jim Fehlig <jfehlig@suse.com> Fixes: b7b03eb614 ("migration/multifd: Add outgoing QIOChannelFile support") Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-08-02migration: Fix cleanup of iochannel in file migrationFabiano Rosas1-2/+0
The QIOChannelFile object already has its reference decremented by g_autoptr. Trying to unref an extra time causes: ERROR:../qom/object.c:1241:object_unref: assertion failed: (obj->ref > 0) Fixes: a701c03dec ("migration: Drop reference to QIOChannel if file seeking fails") Fixes: 6d3279655a ("migration: Fix file migration with fdset") Reported-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-08-02migration: Free removed SaveStateEntryAkihiko Odaki1-0/+2
This fixes LeakSanitizer warnings. Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-21migration/postcopy: Add postcopy-recover-setup phasePeter Xu4-8/+45
This patch adds a migration state on src called "postcopy-recover-setup". The new state will describe the intermediate step starting from when the src QEMU received a postcopy recovery request, until the migration channels are properly established, but before the recovery process take place. The request came from Libvirt where Libvirt currently rely on the migration state events to detect migration state changes. That works for most of the migration process but except postcopy recovery failures at the beginning. Currently postcopy recovery only has two major states: - postcopy-paused: this is the state that both sides of QEMU will be in for a long time as long as the migration channel was interrupted. - postcopy-recover: this is the state where both sides of QEMU handshake with each other, preparing for a continuation of postcopy which used to be interrupted. The issue here is when the recovery port is invalid, the src QEMU will take the URI/channels, noticing the ports are not valid, and it'll silently keep in the postcopy-paused state, with no event sent to Libvirt. In this case, the only thing Libvirt can do is to poll the migration status with a proper interval, however that's less optimal. Considering that this is the only case where Libvirt won't get a notification from QEMU on such events, let's add postcopy-recover-setup state to mimic what we have with the "setup" state of a newly initialized migration, describing the phase of connection establishment. With that, postcopy recovery will have two paths to go now, and either path will guarantee an event generated. Now the events will look like this during a recovery process on src QEMU: - Initially when the recovery is initiated on src, QEMU will go from "postcopy-paused" -> "postcopy-recover-setup". Old QEMUs don't have this event. - Depending on whether the channel re-establishment is succeeded: - In succeeded case, src QEMU will move from "postcopy-recover-setup" to "postcopy-recover". Old QEMUs also have this event. - In failure case, src QEMU will move from "postcopy-recover-setup" to "postcopy-paused" again. Old QEMUs don't have this event. This guarantees that Libvirt will always receive a notification for recovery process properly. One thing to mention is, such new status is only needed on src QEMU not both. On dest QEMU, the state machine doesn't change. Hence the events don't change either. It's done like so because dest QEMU may not have an explicit point of setup start. E.g., it can happen that when dest QEMUs doesn't use migrate-recover command to use a new URI/channel, but the old URI/channels can be reused in recovery, in which case the old ports simply can work again after the network routes are fixed up. Add a new helper postcopy_is_paused() detecting whether postcopy is still paused, taking RECOVER_SETUP into account too. When using it on both src/dst, a slight change is done altogether to always wait for the semaphore before checking the status, because for both sides a sem_post() will be required for a recovery. Cc: Jiri Denemark <jdenemar@redhat.com> Cc: Prasad Pandit <ppandit@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Buglink: https://issues.redhat.com/browse/RHEL-38485 Signed-off-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-21migration: Cleanup incoming migration setup state changePeter Xu1-2/+26
Destination QEMU can setup incoming ports for two purposes: either a fresh new incoming migration, in which QEMU will switch to SETUP for channel establishment, or a paused postcopy migration, in which QEMU will stay in POSTCOPY_PAUSED until kicking off the RECOVER phase. Now the state machine worked on dest node for the latter, only because migrate_set_state() implicitly will become a noop if the current state check failed. It wasn't clear at all. Clean it up by providing a helper migration_incoming_state_setup() doing proper checks over current status. Postcopy-paused will be explicitly checked now, and then we can bail out for unknown states. Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-21migration: Use MigrationStatus instead of intPeter Xu2-21/+12
QEMU uses "int" in most cases even if it stores MigrationStatus. I don't know why, so let's try to do that right and see what blows up.. Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-21migration: Rename thread debug namesPeter Xu5-10/+10
The postcopy thread names on dest QEMU are slightly confusing, partly I'll need to blame myself on 36f62f11e4 ("migration: Postcopy preemption preparation on channel creation"). E.g., "fault-fast" reads like a fast version of "fault-default", but it's actually the fast version of "postcopy/listen". Taking this chance, rename all the migration threads with proper rules. Considering we only have 15 chars usable, prefix all threads with "mig/", meanwhile identify src/dst threads properly this time. So now most thread names will look like "mig/DIR/xxx", where DIR will be "src"/"dst", except the bg-snapshot thread which doesn't have a direction. For multifd threads, making them "mig/{src|dst}/{send|recv}_%d". We used to have "live_migration" thread for a very long time, now it's called "mig/src/main". We may hope to have "mig/dst/main" soon but not yet. Reviewed-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Zhijian Li (Fujitsu) <lizhijian@fujitsu.com> Signed-off-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-21migration/multifd: Avoid the final FLUSH in complete()Peter Xu1-4/+0
We always do the flush when finishing one round of scan, and during complete() phase we should scan one more round making sure no dirty page existed. In that case we shouldn't need one explicit FLUSH at the end of complete(), as when reaching there all pages should have been flushed. Reviewed-by: Fabiano Rosas <farosas@suse.de> Tested-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-21migration/multifd: Add direct-io supportFabiano Rosas3-6/+51
When multifd is used along with mapped-ram, we can take benefit of a filesystem that supports the O_DIRECT flag and perform direct I/O in the multifd threads. This brings a significant performance improvement because direct-io writes bypass the page cache which would otherwise be thrashed by the multifd data which is unlikely to be needed again in a short period of time. To be able to use a multifd channel opened with O_DIRECT, we must ensure that a certain aligment is used. Filesystems usually require a block-size alignment for direct I/O. The way to achieve this is by enabling the mapped-ram feature, which already aligns its I/O properly (see MAPPED_RAM_FILE_OFFSET_ALIGNMENT at ram.c). By setting O_DIRECT on the multifd channels, all writes to the same file descriptor need to be aligned as well, even the ones that come from outside multifd, such as the QEMUFile I/O from the main migration code. This makes it impossible to use the same file descriptor for the QEMUFile and for the multifd channels. The various flags and metadata written by the main migration code will always be unaligned by virtue of their small size. To workaround this issue, we'll require a second file descriptor to be used exclusively for direct I/O. The second file descriptor can be obtained by QEMU by re-opening the migration file (already possible), or by being provided by the user or management application (support to be added in future patches). Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-21migration: Add direct-io parameterFabiano Rosas3-0/+47
Add the direct-io migration parameter that tells the migration code to use O_DIRECT when opening the migration stream file whenever possible. This is currently only used with the mapped-ram migration that has a clear window guaranteed to perform aligned writes. Acked-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-20migration: Fix file migration with fdsetFabiano Rosas1-2/+9
When the "file:" migration support was added we missed the special case in the qemu_open_old implementation that allows for a particular file name format to be used to refer to a set of file descriptors that have been previously provided to QEMU via the add-fd QMP command. When using this fdset feature, we should not truncate the migration file because being given an fd means that the management layer is in control of the file and will likely already have some data written to it. This is further indicated by the presence of the 'offset' argument, which indicates the start of the region where QEMU is allowed to write. Fix the issue by replacing the O_TRUNC flag on open by an ftruncate call, which will take the offset into consideration. Fixes: 385f510df5 ("migration: file URI offset") Suggested-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Prasad Pandit <pjp@fedoraproject.org> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-20migration: Drop reference to QIOChannel if file seeking failsFabiano Rosas1-0/+1
We forgot to drop the reference to the QIOChannel in the error path of the offset adjustment. Do it now. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-14migration/multifd: Switch to no compression when no hardware supportShameer Kolothum1-39/+53
Send raw packets over if UADK hardware support is not available. This is to satisfy  Qemu qtest CI which may run on platforms that don't have UADK hardware support. Subsequent patch will add support for uadk migration qtest. Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-14migration/multifd: Add UADK based compression and decompressionShameer Kolothum1-2/+130
Uses UADK wd_do_comp_sync() API to (de)compress a normal page using hardware accelerator. Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-14migration/multifd: Add UADK initializationShameer Kolothum1-1/+208
Initialize UADK session and allocate buffers required. The actual compression/decompression will only be done in a subsequent patch. Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-14migration/multifd: add uadk compression frameworkShameer Kolothum3-2/+24
Adds the skeleton to support uadk compression method. Complete functionality will be added in subsequent patches. Acked-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-14migration/multifd: implement qpl compression and decompressionYuan Liu1-4/+420
QPL compression and decompression will use IAA hardware path if the IAA hardware is available. Otherwise the QPL library software path is used. The hardware path will automatically fall back to QPL software path if the IAA queues are busy. In some scenarios, this may happen frequently, such as configuring 4 channels but only one IAA device is available. In the case of insufficient IAA hardware resources, retry and fallback can help optimize performance: 1. Retry + SW fallback: total time: 14649 ms downtime: 25 ms throughput: 17666.57 mbps pages-per-second: 1509647 2. No fallback, always wait for work queues to become available total time: 18381 ms downtime: 25 ms throughput: 13698.65 mbps pages-per-second: 859607 If both the hardware and software paths fail, the uncompressed page is sent directly. Signed-off-by: Yuan Liu <yuan1.liu@intel.com> Reviewed-by: Nanhai Zou <nanhai.zou@intel.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-14migration/multifd: implement initialization of qpl compressionYuan Liu1-1/+327
during initialization, a software job is allocated to each channel for software path fallabck when the IAA hardware is unavailable or the hardware job submission fails. If the IAA hardware is available, multiple hardware jobs are allocated for batch processing. Signed-off-by: Yuan Liu <yuan1.liu@intel.com> Reviewed-by: Nanhai Zou <nanhai.zou@intel.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-14migration/multifd: add qpl compression methodYuan Liu3-0/+22
add the Query Processing Library (QPL) compression method Introduce the qpl as a new multifd migration compression method, it can use In-Memory Analytics Accelerator(IAA) to accelerate compression and decompression, which can not only reduce network bandwidth requirement but also reduce host compression and decompression CPU overhead. How to enable qpl compression during migration: migrate_set_parameter multifd-compression qpl There is no qpl compression level parameter added since it only supports level one, users do not need to specify the qpl compression level. Signed-off-by: Yuan Liu <yuan1.liu@intel.com> Reviewed-by: Nanhai Zou <nanhai.zou@intel.com> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> [fixed docs spacing in migration.json] Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-06-14migration/multifd: put IOV initialization into compression methodYuan Liu3-11/+26
Different compression methods may require different numbers of IOVs. Based on streaming compression of zlib and zstd, all pages will be compressed to a data block, so two IOVs are needed for packet header and compressed data block. Signed-off-by: Yuan Liu <yuan1.liu@intel.com> Reviewed-by: Nanhai Zou <nanhai.zou@intel.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-27migration: Rephrase message on failure to save / load Xen device stateMarkus Armbruster1-3/+2
Functions that use an Error **errp parameter to return errors should not also report them to the user, because reporting is the caller's job. When the caller does, the error is reported twice. When it doesn't (because it recovered from the error), there is no error to report, i.e. the report is bogus. qmp_xen_save_devices_state() and qmp_xen_load_devices_state() violate this principle: they call qemu_save_device_state() and qemu_loadvm_state(), which call error_report_err(). I wish I could clean this up now, but migration's error reporting is too complicated (confused?) for me to mess with it. Instead, I'm merely improving the error reported by qmp_xen_load_devices_state() and qmp_xen_load_devices_state() to the QMP core from An IO error has occurred to saving Xen device state failed and loading Xen device state failed respectively. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240513141703.549874-6-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Acked-by: Fabiano Rosas <farosas@suse.de> Acked-by: Peter Xu <peterx@redhat.com>
2024-05-25migration: remove unnecessary zlib dependencyPaolo Bonzini3-3/+1
zlib code is only used by the emulators, not by the tests. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-22Merge tag 'migration-20240522-pull-request' of ↵Richard Henderson5-21/+15
https://gitlab.com/farosas/qemu into staging Migration pull request - Li Zhijian's COLO minor fixes - Marc-André's virtio-gpu fix - Fiona's virtio-net USO fix - A couple of migration-test fixes from Thomas # -----BEGIN PGP SIGNATURE----- # # iQJEBAABCAAuFiEEqhtIsKIjJqWkw2TPx5jcdBvsMZ0FAmZObggQHGZhcm9zYXNA # c3VzZS5kZQAKCRDHmNx0G+wxnWE8D/49RGE+g29qyk9aKx3lU8mSq+ZzmX5GncBt # 5+Mx5qoHDsBCQTE+dQpEVIoeMJ2HIbgbOML4qsnp6Hw/4/TWkfwC/R6+ZmHBevRk # fVLkVh2JMHVg8Tq+0FO1X1QnMU03uJ7EAuWdDa8HqlJ5dQY/K3gDaku8oQBXk96X # 13pChSbMob76tdb+wiwbdEakabigH7XfrPdI6lzI8MCGTIcPKc/UKTFYuoj/OsNx # raqy+uBtvKtfHxiaYnIgHIPNAF/1f4tP3iAOcPoZWIMXWxFkE8+ANDJAbWo6xIcL # DGg/wEzZO/OnXLjOhjvLBUHK/fx4wQ5bsqA09BVxoRyBGblkXr+bcwBLYjgiEqzT # aniPiAx5W/Db+T7HqZPIWesFYj3cmcwvYUTrx/RPMdC0epG+ZczDMtescHdZbxvt # Pjs3nFeCLhyYcVhlTI72eXRCxdd/26+r6/OmrBC2+GaZrybM61TvNo+3XvO0Pfhi # UmwF2EN27XmSMelLvH/MnflUVgBHKDs3CCQzDlxreHq2jMVR0SL7LU5wMJJ58Iok # M3u74izQM25bwYxiASH+4iRn0puH1mOwgOx28W0uiQfZY/678/lCnwa1Tul15BRE # fIQZJhyIGzhSpwLqEXmdXdlLQs1isqIgpd/mzKgZ285nLr7kz+4gxCUqiXgVbrl7 # P45Dym1u4g== # =DDrh # -----END PGP SIGNATURE----- # gpg: Signature made Wed 22 May 2024 03:13:28 PM PDT # gpg: using RSA key AA1B48B0A22326A5A4C364CFC798DC741BEC319D # gpg: issuer "farosas@suse.de" # gpg: Good signature from "Fabiano Rosas <farosas@suse.de>" [unknown] # gpg: aka "Fabiano Almeida Rosas <fabiano.rosas@suse.com>" [unknown] # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: AA1B 48B0 A223 26A5 A4C3 64CF C798 DC74 1BEC 319D * tag 'migration-20240522-pull-request' of https://gitlab.com/farosas/qemu: tests/qtest/migration-test: Fix the check for a successful run of analyze-migration.py tests/qtest/migration-test: Run some basic tests on s390x and ppc64 with TCG, too hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1 virtio-gpu: fix v2 migration migration: fix a typo migration: add "exists" info to load-state-field trace migration/colo: Tidy up bql_unlock() around bdrv_activate_all() migration/colo: make colo_incoming_co() return void migration/colo: Minor fix for colo error message Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2024-05-22migration: fix a typoMarc-André Lureau1-1/+1
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Fiona Ebner <f.ebner@proxmox.com> Tested-by: Fiona Ebner <f.ebner@proxmox.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-22migration: add "exists" info to load-state-field traceMarc-André Lureau2-3/+4
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Fiona Ebner <f.ebner@proxmox.com> Tested-by: Fiona Ebner <f.ebner@proxmox.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-22migration/colo: Tidy up bql_unlock() around bdrv_activate_all()Li Zhijian1-2/+1
Make the code more tight. Suggested-by: Michael Tokarev <mjt@tls.msk.ru> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Zhang Chen <chen.zhang@intel.com> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com> Signed-off-by: Michael Tokarev <mjt@tls.msk.ru> [fixed mangled author email address] Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-22migration/colo: make colo_incoming_co() return voidLi Zhijian3-12/+6
Currently, it always returns 0, no need to check the return value at all. In addition, enter colo coroutine only if migration_incoming_colo_enabled() is true. Once the destination side enters the COLO* state, the COLO process will take over the remaining processes until COLO exits. Cc: Fabiano Rosas <farosas@suse.de> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Zhang Chen <chen.zhang@intel.com> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com> [fixed mangled author email address] Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-22migration/colo: Minor fix for colo error messageLi Zhijian1-3/+3
- Explicitly show the missing module name: replication - Fix capability name to x-colo Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Zhang Chen <chen.zhang@intel.com> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com> Suggested-by: Michael Tokarev <mjt@tls.msk.ru> [fixed mangled author email address] Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-16migration: Extend migration_file_set_error() with Error* argumentCédric Le Goater1-2/+4
Use it to update the current error of the migration stream if available and if not, simply print out the error. Next changes will update with an error to report. Reviewed-by: Avihai Horon <avihaih@nvidia.com> Acked-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Cédric Le Goater <clg@redhat.com>
2024-05-08migration: Deprecate fd: for file migrationFabiano Rosas1-0/+12
The fd: URI can currently trigger two different types of migration, a TCP migration using sockets and a file migration using a plain file. This is in conflict with the recently introduced (8.2) QMP migrate API that takes structured data as JSON-like format. We cannot keep the same backend for both types of migration because with the new API the code is more tightly coupled to the type of transport. This means a TCP migration must use the 'socket' transport and a file migration must use the 'file' transport. If we keep allowing fd: when using a file, this creates an issue when the user converts the old-style (fd:) to the new style ("transport": "socket") invocation because the file descriptor in question has previously been allowed to be either a plain file or a socket. To avoid creating too much confusion, we can simply deprecate the fd: + file usage, which is thought to be rarely used currently and instead establish a 1:1 correspondence between fd: URI and socket transport, and file: URI and file transport. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-08migration: Remove non-multifd compressionFabiano Rosas11-1105/+9
The 'compress' migration capability enables the old compression code which has shown issues over the years and is thought to be less stable and tested than the more recent multifd-based compression. The old compression code has been deprecated in 8.2 and now is time to remove it. Deprecation commit 864128df46 ("migration: Deprecate old compression method"). Acked-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-08migration: Remove block migrationFabiano Rosas10-1160/+0
The block migration has been considered obsolete since QEMU 8.2 in favor of the more flexible storage migration provided by the blockdev-mirror driver. Two releases have passed so now it's time to remove it. Deprecation commit 66db46ca83 ("migration: Deprecate block migration"). Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-08migration: Remove 'blk/-b' option from migrate commandsFabiano Rosas5-54/+6
The block migration is considered obsolete and has been deprecated in 8.2. Remove the migrate command option that enables it. This only affects the QMP and HMP commands, the feature can still be accessed by setting the migration 'block' capability. The whole feature will be removed in a future patch. Deprecation commit 8846b5bfca ("migration: migrate 'blk' command option is deprecated."). Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-08migration: Remove 'inc' option from migrate commandFabiano Rosas5-68/+10
The block incremental option for block migration has been deprecated in 8.2 in favor of using the block-mirror feature. Remove it now. Deprecation commit 40101f320d ("migration: migrate 'inc' command option is deprecated."). Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
2024-05-08migration: Remove 'skipped' field from MigrationStatsFabiano Rosas2-4/+0
The 'skipped' field of the MigrationStats struct has been deprecated in 8.1. Time to remove it. Deprecation commit 7b24d32634 ("migration: skipped field is really obsolete."). Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de>
generated by cgit v1.1 at 2025-02-03 15:11:52 +0000