aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2024-03-04migration: simplify exec migration functionsSteve Sistare1-49/+8
Simplify the exec migration code by using list utility functions. As a side effect, this also fixes a minor memory leak. On function return, "g_auto(GStrv) argv" frees argv and each element, which is wrong, because the function does not own the individual elements. To compensate, the code uses g_steal_pointer which NULLs argv and prevents the destructor from running, but argv is leaked. Fixes: cbab4face57b ("migration: convert exec backend ...") Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Message-ID: <20240227153321.467343-4-armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-03-04qapi: New strv_from_str_list()Steve Sistare2-0/+22
Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-ID: <20240227153321.467343-3-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-03-04qapi: New QAPI_LIST_LENGTH()Steve Sistare1-0/+13
Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-ID: <20240227153321.467343-2-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-03-04docs/devel/writing-monitor-commands: Minor improvementsMarkus Armbruster1-17/+15
Avoid "JSON" when talking about the QAPI schema syntax. Capitalize QEMU. Don't claim all HMP commands live in monitor/hmp-cmds.c (this was never true). Fix punctuation and drop inappropriate "the" here and there. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227115617.237875-3-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2024-03-04docs/devel/writing-monitor-commands: Repair a decade of rotMarkus Armbruster1-279/+181
The tutorial doesn't match reality since at least 2013. Repairing it involves fixing the following issues: * Update for commit 6d327171551 (aio / timers: Remove alarm timers): replace the broken examples. Instead of having one for returning a struct and another for returning a list of structs, do just one for the latter. This resolves the FIXME added in commit e218052f928 (aio / timers: De-document -clock) back in 2014. * Update for commit 895a2a80e0e (qapi: Use 'struct' instead of 'type' in schema). * Update for commit 3313b6124b5 (qapi: add qapi2texi script): add required documentation to the schema snippets, and drop section "Command Documentation". * Update for commit a3c45b3e629 (qapi: New special feature flag "unstable"): supply the required feature, deemphasize the x- prefix. * Update for commit dd98234c059 (qapi: introduce x-query-roms QMP command): rephrase from "add new command" to "examine existing command". * Update for commit 9492718b7c0 (qapi misc: Elide redundant has_FOO in generated C): hello-world's message argument no longer comes with a has_message, add a second argument that does. * Update for moved and renamed files. While there, update QMP version output to current output. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227115617.237875-2-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [Whitespace tidied up, typo fixed]
2024-03-04qapi: Reject "Returns" section when command doesn't return anythingMarkus Armbruster5-1/+15
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-14-armbru@redhat.com>
2024-03-04qga/qapi-schema: Fix guest-set-memory-blocks documentationMarkus Armbruster1-3/+3
Documentation claims the command can "return NULL". "NULL" doesn't exist in JSON. "null" does, but the command returns lists, and null isn't. Correct documentation to "return an empty list". Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-13-armbru@redhat.com>
2024-03-04qga/qapi-schema: Tweak documentation of fsfreeze commandsMarkus Armbruster1-7/+9
"Returns:" sections of guest-fsfreeze-freeze and guest-fsfreeze-freeze-list describe both command behavior and success response. Move behavior out, so "Returns:" is only about success response. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-12-armbru@redhat.com>
2024-03-04qga/qapi-schema: Clean up "Returns" sectionsMarkus Armbruster1-8/+8
Drop "on success" where it is redundant with "Returns:". Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-11-armbru@redhat.com>
2024-03-04qga/qapi-schema: Delete useless "Returns" sectionsMarkus Armbruster1-12/+0
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-10-armbru@redhat.com>
2024-03-04qga/qapi-schema: Move error documentation to new "Errors" sectionsMarkus Armbruster1-12/+10
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-9-armbru@redhat.com>
2024-03-04qapi/yank: Tweak @yank's error description for consistencyMarkus Armbruster1-1/+1
Phrase it like "If <condition>, <error>", like we do elsewhere. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-8-armbru@redhat.com>
2024-03-04qapi: Clean up "Returns" sectionsMarkus Armbruster3-5/+5
Drop list markup, since there's just one item left. Drop "on success" where it is redundant with "Returns:". Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-7-armbru@redhat.com>
2024-03-04qapi: Delete useless "Returns" sectionsMarkus Armbruster17-183/+1
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-6-armbru@redhat.com>
2024-03-04qapi: Move error documentation to new "Errors" sectionsMarkus Armbruster12-31/+126
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-5-armbru@redhat.com>
2024-03-04qapi: New documentation section tag "Errors"Markus Armbruster5-7/+32
We use section "Returns" for documenting both success and error response of commands. I intend to generate better command success response documentation. Easier when "Returns" documents just he success response. Create new section tag "Errors". The next two commits will move error response documentation from "Returns" sections to "Errors" sections. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-4-armbru@redhat.com>
2024-03-04qapi: Slightly clearer error message for invalid "Returns" sectionMarkus Armbruster2-3/+4
Change "'Returns:' is only valid for commands" to "'Returns' section is only valid for commands". Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-3-armbru@redhat.com>
2024-03-04qapi: Memorize since & returns sectionsMarkus Armbruster1-14/+17
This is chiefly to make code that looks up these sections easier to read. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-2-armbru@redhat.com>
2024-03-04migration/multifd: Document two places for mapped-ramPeter Xu2-1/+19
Add two documentations for mapped-ram migration on two spots that may not be extremely clear. Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240301091524.39900-1-peterx@redhat.com Cc: Prasad Pandit <ppandit@redhat.com> [peterx: fix two English errors per Prasad] Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-03roms/hppa: Add build rules for hppa-firmwareHelge Deller1-0/+7
Signed-off-by: Helge Deller <deller@gmx.de> Suggested-by: Michael Tokarev <mjt@tls.msk.ru> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2024-03-03pc-bios/README: Add information about hppa-firmwareHelge Deller1-0/+6
Signed-off-by: Helge Deller <deller@gmx.de> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2024-03-03pc-bios/meson: Add hppa-firmware64.img blobHelge Deller1-0/+1
Add the missing 64-bit hppa firmware blob so that it gets installed. Signed-off-by: Helge Deller <deller@gmx.de> Fixes: 7c0dfcf9395e ("target/hppa: Update SeaBIOS-hppa to version 16") Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2024-03-03target/hppa: Restore unwind_breg before calculating iorHelge Deller3-3/+6
When calculating the IOR for the exception handlers, the current unwind_breg value is needed on 64-bit hppa machines. Restore that value by calling cpu_restore_state() earlier, which in turn calls hppa_restore_state_to_opc() which restores the unwind_breg for the current instruction. Signed-off-by: Helge Deller <deller@gmx.de> Fixes: 3824e0d643f3 ("target/hppa: Export function hppa_set_ior_and_isr()") Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2024-03-03target: hppa: Fix unaligned double word accesses for hppa64Guenter Roeck1-1/+2
Unaligned 64-bit accesses were found in Linux to clobber carry bits, resulting in bad results if an arithmetic operation involving a carry bit was executed after an unaligned 64-bit operation. hppa 2.0 defines additional carry bits in PSW register bits 32..39. When restoring PSW after executing an unaligned instruction trap, those bits were not cleared and ended up to be active all the time. Since there are no bits other than the upper carry bits needed in the upper 32 bit of env->psw and since those are stored in env->psw_cb, just clear the entire upper 32 bit when storing psw to solve the problem unconditionally. Fixes: 931adff31478 ("target/hppa: Update cpu_hppa_get/put_psw for hppa64") Cc: Richard Henderson <richard.henderson@linaro.org> Cc: Charlie Jenkins <charlie@rivosinc.com> Cc: Helge Deller <deller@gmx.de> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Helge Deller <deller@gmx.de>
2024-03-01tests/tcg: Check that shmat() does not break /proc/self/mapsIlya Leoshkevich1-0/+55
Add a regression test for a recently fixed issue, where shmat() desynced the guest and the host view of the address space and caused open("/proc/self/maps") to SEGV. Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Message-Id: <jwyuvao4apydvykmsnvacwshdgy3ixv7qvkh4dbxm3jkwgnttw@k4wpaayou7oq> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2024-03-01linux-user: Rewrite target_shmatRichard Henderson1-33/+133
Handle combined host and guest alignment requirements. Handle host and guest page size differences. Handle SHM_EXEC. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/115 Tested-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2024-03-01linux-user: Add strace for shmatRichard Henderson2-1/+25
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2024-03-01Merge tag 'pull-request-2024-03-01' of https://gitlab.com/thuth/qemu into ↵Peter Maydell5-18/+33
staging * Fix some bugs in the vring setup of libqos * Fix GIC settings when using --without-default-devices * Fix USB PCAP streams on Windows * Remove temporary files from test-util-sockets * Fix TLS io channels sending too much data to the backend # -----BEGIN PGP SIGNATURE----- # # iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmXhjBcRHHRodXRoQHJl # ZGhhdC5jb20ACgkQLtnXdP5wLbWzZw/+OTSsKg6JTX0z3fjL6If+Ns/EnFiniHHI # y1jR7fPub3ybFokgHIWrRVG+9ol+1tJuPlO5Bsx1SKFzNC8++l1iyK7E25xnmp2P # Ry/Z0ZSWp3JWCtLm6km5pKl3wWI9p0XwfUOJbCWYM5dOsd2a4C4DSvjcwNXR7pTB # 4AxXFj5G5UwXuffgPnBzeerm5baNweSqa/uczUY6Od+iJwmHpaSk0SJ4NIlIYvnS # z3BH81trQAoKYZyywp0aF+jH9w3YNwD1XDp2Bcf3qSWDdokN55V13yyiSllTfUwX # Lq3fotoLqFJL0A4m8a3TdCKMP8ReoJATSriPHJmGMgqWGTnvtRBIsx1pmMpYRB6U # EltaKW0WXeQAad0ZYGl4odx7kQUQjdc8HTQRTevQD9g3XpZZN0o9MfTPdM2eR5h6 # xaS8UEc61YJAcmD27Ir92JGGGMLBTYz4OGLx3Rj7l7S759GQ67qih9TGcE9nqSj+ # +wAHfiOjUZnSWnp7Hq4M/TIjNh0BuyBW4oRBASaikjjQ82Hih5bPTdRu1J8lYuXC # PhN8dLwdk0NMrnnHuFDKq/sXjDeXF3Hg0AmtVvOGiB2z2OeVSPtMxl/YgniGVTbj # CHXDqDrBAa7i6WxAo+TtUgE20iVz5uYzGqHmxTNpq6FzIRb1uvHksbjFlin79Pkw # eF0Dj80zxAY= # =9JAI # -----END PGP SIGNATURE----- # gpg: Signature made Fri 01 Mar 2024 08:04:39 GMT # gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5 # gpg: issuer "thuth@redhat.com" # gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full] # gpg: aka "Thomas Huth <thuth@redhat.com>" [full] # gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full] # gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown] # Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5 * tag 'pull-request-2024-03-01' of https://gitlab.com/thuth/qemu: chardev/char-socket: Fix TLS io channels sending too much data to the backend tests/unit/test-util-sockets: Remove temporary file after test hw/usb/bus.c: PCAP adding 0xA in Windows version hw/intc/Kconfig: Fix GIC settings when using "--without-default-devices" libqos/virtio.c: fix 'avail_event' offset in qvring_init() libqos/virtio.c: init all elems in qvring_indirect_desc_setup() Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-03-01Merge tag 'pull-loongarch-20240229' of https://gitlab.com/gaosong/qemu into ↵Peter Maydell3-33/+107
staging pull-loongarch-20240229 V2: fix build error on mipsel # -----BEGIN PGP SIGNATURE----- # # iLMEAAEKAB0WIQS4/x2g0v3LLaCcbCxAov/yOSY+3wUCZeBrwAAKCRBAov/yOSY+ # 33YXA/4+A5Bpe/3+mSAWZSUlluGTqUi0ILBYRMyX1RXovMx4uCRGr7PXzAf03yKS # MZzlVzTuOK69WmTm/iTdYWOxkXisC3gzxL/wm8hP4lzh4c0dHrHRsKHqq6gR3+t2 # ojdZn7TefeflnNqIhxXxgxb1OETofhBNnBJ74pvqxO7XV5SWnA== # =J2Kb # -----END PGP SIGNATURE----- # gpg: Signature made Thu 29 Feb 2024 11:34:24 GMT # gpg: using RSA key B8FF1DA0D2FDCB2DA09C6C2C40A2FFF239263EDF # gpg: Good signature from "Song Gao <m17746591750@163.com>" [unknown] # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: B8FF 1DA0 D2FD CB2D A09C 6C2C 40A2 FFF2 3926 3EDF * tag 'pull-loongarch-20240229' of https://gitlab.com/gaosong/qemu: loongarch: Change the UEFI loading mode to loongarch Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-03-01tests/qtest/migration: Add a multifd + mapped-ram migration testFabiano Rosas1-0/+68
Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-24-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Add mapped-ram support to fd: URIFabiano Rosas5-6/+64
If we receive a file descriptor that points to a regular file, there's nothing stopping us from doing multifd migration with mapped-ram to that file. Enable the fd: URI to work with multifd + mapped-ram. Note that the fds passed into multifd are duplicated because we want to avoid cross-thread effects when doing cleanup (i.e. close(fd)). The original fd doesn't need to be duplicated because monitor_get_fd() transfers ownership to the caller. Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Peter Xu <peterx@redhat.com> Link: https://lore.kernel.org/r/20240229153017.2221-23-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Support incoming mapped-ram stream formatFabiano Rosas5-6/+73
For the incoming mapped-ram migration we need to read the ramblock headers, get the pages bitmap and send the host address of each non-zero page to the multifd channel thread for writing. Usage on HMP is: (qemu) migrate_set_capability multifd on (qemu) migrate_set_capability mapped-ram on (qemu) migrate_incoming file:migfile (the ram.h include needs to move because we've been previously relying on it being included from migration.c. Now file.h will start including multifd.h before migration.o is processed) Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-22-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Support outgoing mapped-ram stream formatFabiano Rosas8-16/+125
The new mapped-ram stream format uses a file transport and puts ram pages in the migration file at their respective offsets and can be done in parallel by using the pwritev system call which takes iovecs and an offset. Add support to enabling the new format along with multifd to make use of the threading and page handling already in place. This requires multifd to stop sending headers and leaving the stream format to the mapped-ram code. When it comes time to write the data, we need to call a version of qio_channel_write that can take an offset. Usage on HMP is: (qemu) stop (qemu) migrate_set_capability multifd on (qemu) migrate_set_capability mapped-ram on (qemu) migrate_set_parameter max-bandwidth 0 (qemu) migrate_set_parameter multifd-channels 8 (qemu) migrate file:migfile Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-21-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Prepare multifd sync for mapped-ram migrationFabiano Rosas1-7/+31
The mapped-ram migration can be performed live or non-live, but it is always asynchronous, i.e. the source machine and the destination machine are not migrating at the same time. We only need some pieces of the multifd sync operations. multifd_send_sync_main() ------------------------ Issued by the ram migration code on the migration thread, causes the multifd send channels to synchronize with the migration thread and makes the sending side emit a packet with the MULTIFD_FLUSH flag. With mapped-ram we want to maintain the sync on the sending side because that provides ordering between the rounds of dirty pages when migrating live. MULTIFD_FLUSH ------------- On the receiving side, the presence of the MULTIFD_FLUSH flag on a packet causes the receiving channels to start synchronizing with the main thread. We're not using packets with mapped-ram, so there's no MULTIFD_FLUSH flag and therefore no channel sync on the receiving side. multifd_recv_sync_main() ------------------------ Issued by the migration thread when the ram migration flag RAM_SAVE_FLAG_MULTIFD_FLUSH is received, causes the migration thread on the receiving side to start synchronizing with the recv channels. Due to compatibility, this is also issued when RAM_SAVE_FLAG_EOS is received. For mapped-ram we only need to synchronize the channels at the end of migration to avoid doing cleanup before the channels have finished their IO. Make sure the multifd syncs are only issued at the appropriate times. Note that due to pre-existing backward compatibility issues, we have the multifd_flush_after_each_section property that can cause a sync to happen at EOS. Since the EOS flag is needed on the stream, allow mapped-ram to just ignore it. Also emit an error if any other unexpected flags are found on the stream. Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Peter Xu <peterx@redhat.com> Link: https://lore.kernel.org/r/20240229153017.2221-20-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Add incoming QIOChannelFile supportFabiano Rosas3-11/+30
On the receiving side we don't need to differentiate between main channel and threads, so whichever channel is defined first gets to be the main one. And since there are no packets, use the atomic channel count to index into the params array. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-19-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Add outgoing QIOChannelFile supportFabiano Rosas4-3/+57
Allow multifd to open file-backed channels. This will be used when enabling the mapped-ram migration stream format which expects a seekable transport. The QIOChannel read and write methods will use the preadv/pwritev versions which don't update the file offset at each call so we can reuse the fd without re-opening for every channel. Contrary to the socket migration, the file migration doesn't need an asynchronous channel creation process, so expose multifd_channel_connect() and call it directly. Note that this is just setup code and multifd cannot yet make use of the file channels. Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Peter Xu <peterx@redhat.com> Link: https://lore.kernel.org/r/20240229153017.2221-18-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Add a wrapper for channels_createdFabiano Rosas2-1/+7
We'll need to access multifd_send_state->channels_created from outside multifd.c, so introduce a helper for that. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-17-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Allow receiving pages without packetsFabiano Rosas3-7/+138
Currently multifd does not need to have knowledge of pages on the receiving side because all the information needed is within the packets that come in the stream. We're about to add support to mapped-ram migration, which cannot use packets because it expects the ramblock section in the migration file to contain only the guest pages data. Add a data structure to transfer pages between the ram migration code and the multifd receiving threads. We don't want to reuse MultiFDPages_t for two reasons: a) multifd threads don't really need to know about the data they're receiving. b) the receiving side has to be stopped to load the pages, which means we can experiment with larger granularities than page size when transferring data. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-16-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Allow multifd without packetsFabiano Rosas1-61/+114
For the upcoming support to the new 'mapped-ram' migration stream format, we cannot use multifd packets because each write into the ramblock section in the migration file is expected to contain only the guest pages. They are written at their respective offsets relative to the ramblock section header. There is no space for the packet information and the expected gains from the new approach come partly from being able to write the pages sequentially without extraneous data in between. The new format also simply doesn't need the packets and all necessary information can be taken from the standard migration headers with some (future) changes to multifd code. Use the presence of the mapped-ram capability to decide whether to send packets. This only moves code under multifd_use_packets(), it has no effect for now as mapped-ram cannot yet be enabled with multifd. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-15-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Decouple recv method from pagesFabiano Rosas4-13/+16
Next patches will abstract the type of data being received by the channels, so do some cleanup now to remove references to pages and dependency on 'normal_num'. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-14-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/multifd: Rename MultiFDSend|RecvParams::data to compress_dataFabiano Rosas3-22/+22
Use a more specific name for the compression data so we can use the generic for the multifd core code. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-13-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01tests/qtest/migration: Add tests for mapped-ram file-based migrationFabiano Rosas1-0/+59
Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-12-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/ram: Add incoming 'mapped-ram' migrationFabiano Rosas1-2/+141
Add the necessary code to parse the format changes for the 'mapped-ram' capability. One of the more notable changes in behavior is that in the 'mapped-ram' case ram pages are restored in one go rather than constantly looping through the migration stream. Signed-off-by: Nikolay Borisov <nborisov@suse.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-11-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/ram: Add outgoing 'mapped-ram' migrationFabiano Rosas2-9/+135
Implement the outgoing migration side for the 'mapped-ram' capability. A bitmap is introduced to track which pages have been written in the migration file. Pages are written at a fixed location for every ramblock. Zero pages are ignored as they'd be zero in the destination migration as well. The migration stream is altered to put the dirty pages for a ramblock after its header instead of having a sequential stream of pages that follow the ramblock headers. Without mapped-ram (current): With mapped-ram (new): --------------------- -------------------------------- | ramblock 1 header | | ramblock 1 header | --------------------- -------------------------------- | ramblock 2 header | | ramblock 1 mapped-ram header | --------------------- -------------------------------- | ... | | padding to next 1MB boundary | --------------------- | ... | | ramblock n header | -------------------------------- --------------------- | ramblock 1 pages | | RAM_SAVE_FLAG_EOS | | ... | --------------------- -------------------------------- | stream of pages | | ramblock 2 header | | (iter 1) | -------------------------------- | ... | | ramblock 2 mapped-ram header | --------------------- -------------------------------- | RAM_SAVE_FLAG_EOS | | padding to next 1MB boundary | --------------------- | ... | | stream of pages | -------------------------------- | (iter 2) | | ramblock 2 pages | | ... | | ... | --------------------- -------------------------------- | ... | | ... | --------------------- -------------------------------- | RAM_SAVE_FLAG_EOS | -------------------------------- | ... | -------------------------------- where: - ramblock header: the generic information for a ramblock, such as idstr, used_len, etc. - ramblock mapped-ram header: the new information added by this feature: bitmap of pages written, bitmap size and offset of pages in the migration file. Signed-off-by: Nikolay Borisov <nborisov@suse.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-10-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration: Add mapped-ram URI compatibility checkFabiano Rosas1-0/+29
The mapped-ram migration format needs a channel that supports seeking to be able to write each page to an arbitrary offset in the migration stream. Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-9-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/ram: Introduce 'mapped-ram' migration capabilityFabiano Rosas7-1/+187
Add a new migration capability 'mapped-ram'. The core of the feature is to ensure that RAM pages are mapped directly to offsets in the resulting migration file instead of being streamed at arbitrary points. The reasons why we'd want such behavior are: - The resulting file will have a bounded size, since pages which are dirtied multiple times will always go to a fixed location in the file, rather than constantly being added to a sequential stream. This eliminates cases where a VM with, say, 1G of RAM can result in a migration file that's 10s of GBs, provided that the workload constantly redirties memory. - It paves the way to implement O_DIRECT-enabled save/restore of the migration stream as the pages are ensured to be written at aligned offsets. - It allows the usage of multifd so we can write RAM pages to the migration file in parallel. For now, enabling the capability has no effect. The next couple of patches implement the core functionality. Acked-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-8-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01migration/qemu-file: add utility methods for working with seekable channelsFabiano Rosas3-0/+114
Add utility methods that will be needed when implementing 'mapped-ram' migration capability. Signed-off-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com> Link: https://lore.kernel.org/r/20240229153017.2221-7-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01io: fsync before closing a file channelFabiano Rosas1-0/+5
Make sure the data is flushed to disk before closing file channels. This is to ensure data is on disk and not lost in the event of a host crash. This is currently being implemented to affect the migration code when migrating to a file, but all QIOChannelFile users should benefit from the change. Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com> Acked-by: "Daniel P. Berrangé" <berrange@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-6-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01io: implement io_pwritev/preadv for QIOChannelFileNikolay Borisov1-0/+56
The upcoming 'mapped-ram' feature will require qemu to write data to (and restore from) specific offsets of the migration file. Add a minimal implementation of pwritev/preadv and expose them via the io_pwritev and io_preadv interfaces. Signed-off-by: Nikolay Borisov <nborisov@suse.com> Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-5-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
2024-03-01io: Add generic pwritev/preadv interfaceNikolay Borisov2-0/+140
Introduce basic pwritev/preadv support in the generic channel layer. Specific implementation will follow for the file channel as this is required in order to support migration streams with fixed location of each ram page. Signed-off-by: Nikolay Borisov <nborisov@suse.com> Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-4-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
generated by cgit v1.1 at 2025-07-27 02:49:35 +0000