aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2019-03-12qemu-iotests: Test the x-blockdev-reopen QMP commandAlberto Garcia3-0/+997
This patch adds several tests for the x-blockdev-reopen QMP command. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Add an 'x-blockdev-reopen' QMP commandAlberto Garcia2-0/+89
This command allows reopening an arbitrary BlockDriverState with a new set of options. Some options (e.g node-name) cannot be changed and some block drivers don't allow reopening, but otherwise this command is modelled after 'blockdev-add' and the state of the reopened BlockDriverState should generally be the same as if it had just been added by 'blockdev-add' with the same set of options. One notable exception is the 'backing' option: 'x-blockdev-reopen' requires that it is always present unless the BlockDriverState in question doesn't have a current or default backing file. This command allows reconfiguring the graph by using the appropriate options to change the children of a node. At the moment it's possible to change a backing file by setting the 'backing' option to the name of the new node, but it should also be possible to add a similar functionality to other block drivers (e.g. Quorum, blkverify). Although the API is unlikely to change, this command is marked experimental for the time being so there's room to see if the semantics need changes. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Remove the AioContext parameter from bdrv_reopen_multiple()Alberto Garcia4-6/+5
This parameter has been unused since 1a63a907507fbbcfaee3f622907ec244b Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Add bdrv_reset_options_allowed()Alberto Garcia1-0/+58
bdrv_reopen_prepare() receives a BDRVReopenState with (among other things) a new set of options to be applied to that BlockDriverState. If an option is missing then it means that we want to reset it to its default value rather than keep the previous one. This way the state of the block device after being reopened is comparable to that of a device added with "blockdev-add" using the same set of options. Not all options from all drivers can be changed this way, however. If the user attempts to reset an immutable option to its default value using this method then we must forbid it. This new function takes a BlockDriverState and a new set of options and checks if there's any option that was previously set but is missing from the new set of options. If the option is present in both sets we don't need to check that they have the same value. The loop at the end of bdrv_reopen_prepare() already takes care of that. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Add a 'mutable_opts' field to BlockDriverAlberto Garcia4-0/+42
If we reopen a BlockDriverState and there is an option that is present in bs->options but missing from the new set of options then we have to return an error unless the driver is able to reset it to its default value. This patch adds a new 'mutable_opts' field to BlockDriver. This is a list of runtime options that can be modified during reopen. If an option in this list is unspecified on reopen then it must be reset (or return an error). Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Allow changing the backing file on reopenAlberto Garcia2-0/+168
This patch allows the user to change the backing file of an image that is being reopened. Here's what it does: - In bdrv_reopen_prepare(): check that the value of 'backing' points to an existing node or is null. If it points to an existing node it also needs to make sure that replacing the backing file will not create a cycle in the node graph (i.e. you cannot reach the parent from the new backing file). - In bdrv_reopen_commit(): perform the actual node replacement by calling bdrv_set_backing_hd(). There may be temporary implicit nodes between a BDS and its backing file (e.g. a commit filter node). In these cases bdrv_reopen_prepare() looks for the real (non-implicit) backing file and requires that the 'backing' option points to it. Replacing or detaching a backing file is forbidden if there are implicit nodes in the middle. Although x-blockdev-reopen is meant to be used like blockdev-add, there's an important thing that must be taken into account: the only way to set a new backing file is by using a reference to an existing node (previously added with e.g. blockdev-add). If 'backing' contains a dictionary with a new set of options ({"driver": "qcow2", "file": { ... }}) then it is interpreted that the _existing_ backing file must be reopened with those options. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Allow omitting the 'backing' option in certain casesAlberto Garcia1-1/+7
Of all options of type BlockdevRef used to specify children in BlockdevOptions, 'backing' is the only one that is optional. For "x-blockdev-reopen" we want that if an option is omitted then it must be reset to its default value. The default value of 'backing' means that QEMU opens the backing file specified in the image metadata, but this is not something that we want to support for the reopen operation. Because of this the 'backing' option has to be specified during reopen, pointing to the existing backing file if we want to keep it, or pointing to a different one (or NULL) if we want to replace it (to be implemented in a subsequent patch). In order to simplify things a bit and not to require that the user passes the 'backing' option to every single block device even when it's clearly not necessary, this patch allows omitting this option if the block device being reopened doesn't have a backing file attached _and_ no default backing file is specified in the image metadata. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Handle child references in bdrv_reopen_queue()Alberto Garcia2-7/+46
Children in QMP are specified with BlockdevRef / BlockdevRefOrNull, which can contain a set of child options, a child reference, or NULL. In optional attributes like "backing" it can also be missing. Only the first case (set of child options) is being handled properly by bdrv_reopen_queue(). This patch deals with all the others. Here's how these cases should be handled when bdrv_reopen_queue() is deciding what to do with each child of a BlockDriverState: 1) Set of child options: if the child was implicitly created (i.e inherits_from points to the parent) then the options are removed from the parent's options QDict and are passed to the child with a recursive bdrv_reopen_queue() call. This case was already working fine. 2) Child reference: there's two possibilites here. 2a) Reference to the current child: if the child was implicitly created then it is put in the reopen queue, keeping its current set of options (since this was a child reference there was no way to specify a different set of options). If the child is not implicit then it keeps its current set of options but it is not reopened (and therefore does not inherit any new option from the parent). 2b) Reference to a different BDS: the current child is not put in the reopen queue at all. Passing a reference to a different BDS can be used to replace a child, although at the moment no driver implements this, so it results in an error. In any case, the current child is not going to be reopened (and might in fact disappear if it's replaced) 3) NULL: This is similar to (2b). Although no driver allows this yet it can be used to detach the current child so it should not be put in the reopen queue. 4) Missing option: at the moment "backing" is the only case where this can happen. With "blockdev-add", leaving "backing" out means that the default backing file is opened. We don't want to open a new image during reopen, so we require that "backing" is always present. We'll relax this requirement a bit in the next patch. If keep_old_opts is true and "backing" is missing then this behaves like 2a (the current child is reopened). Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Add 'keep_old_opts' parameter to bdrv_reopen_queue()Alberto Garcia4-19/+24
The bdrv_reopen_queue() function is used to create a queue with the BDSs that are going to be reopened and their new options. Once the queue is ready bdrv_reopen_multiple() is called to perform the operation. The original options from each one of the BDSs are kept, with the new options passed to bdrv_reopen_queue() applied on top of them. For "x-blockdev-reopen" we want a function that behaves much like "blockdev-add". We want to ignore the previous set of options so that only the ones actually specified by the user are applied, with the rest having their default values. One of the things that we need is a way to tell bdrv_reopen_queue() whether we want to keep the old set of options or not, and that's what this patch does. All current callers are setting this new parameter to true and x-blockdev-reopen will set it to false. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Freeze the backing chain for the duration of the stream jobAlberto Garcia1-0/+21
Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Freeze the backing chain for the duration of the mirror jobAlberto Garcia1-0/+8
Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Freeze the backing chain for the duration of the commit jobAlberto Garcia1-0/+16
Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Allow freezing BdrvChild linksAlberto Garcia3-0/+87
Our permission system is useful to define what operations are allowed on a certain block node and includes things like BLK_PERM_WRITE or BLK_PERM_RESIZE among others. One of the permissions is BLK_PERM_GRAPH_MOD which allows "changing the node that this BdrvChild points to". The exact meaning of this has never been very clear, but it can be understood as "change any of the links connected to the node". This can be used to prevent changing a backing link, but it's too coarse. This patch adds a new 'frozen' attribute to BdrvChild, which forbids detaching the link from the node it points to, and new API to freeze and unfreeze a backing chain. After this change a few functions can fail, so they need additional checks. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12nvme: fix write zeroes offset and countKeith Busch1-3/+3
The implementation used blocks units rather than the expected bytes. Fixes: c03e7ef12a9 ("nvme: Implement Write Zeroes") Reported-by: Ming Lei <ming.lei@redhat.com> Signed-off-by: Keith Busch <keith.busch@intel.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12file-posix: Make auto-read-only dynamicKevin Wolf2-25/+23
Until now, with auto-read-only=on we tried to open the file read-write first and if that failed, read-only was tried. This is actually not good enough for libvirt, which gives QEMU SELinux permissions for read-write only as soon as it actually intends to write to the image. So we need to be able to switch between read-only and read-write at runtime. This patch makes auto-read-only dynamic, i.e. the file is opened read-only as long as no user of the node has requested write permissions, but it is automatically reopened read-write as soon as the first writer is attached. Conversely, if the last writer goes away, the file is reopened read-only again. bs->read_only is no longer set for auto-read-only=on files even if the file descriptor is opened read-only because it will be transparently upgraded as soon as a writer is attached. This changes the output of qemu-iotests 232. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12file-posix: Prepare permission code for fd switchingKevin Wolf1-11/+85
In order to be able to dynamically reopen the file read-only or read-write, depending on the users that are attached, we need to be able to switch to a different file descriptor during the permission change. This interacts with reopen, which also creates a new file descriptor and performs permission changes internally. In this case, the permission change code must reuse the reopen file descriptor instead of creating a third one. In turn, reopen can drop its code to copy file locks to the new file descriptor because that is now done when applying the new permissions. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12file-posix: Lock new fd in raw_reopen_prepare()Kevin Wolf1-11/+16
There is no reason why we can take locks on the new file descriptor only in raw_reopen_commit() where error handling isn't possible any more. Instead, we can already do this in raw_reopen_prepare(). Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12file-posix: Store BDRVRawState.reopen_state during reopenKevin Wolf1-0/+10
We'll want to access the file descriptor in the reopen_state while processing permission changes in the context of the repoen. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12file-posix: Factor out raw_reconfigure_getfd()Kevin Wolf1-45/+62
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12file-posix: Fix bdrv_open_flags() for snapshot=onKevin Wolf4-7/+25
Using a different read-only setting for bs->open_flags than for the flags to the driver's open function is just inconsistent and a bad idea. After this patch, the temporary snapshot keeps being opened read-only if read-only=on,snapshot=on is passed. If we wanted to change this behaviour to make only the orginal image file read-only, but the temporary overlay read-write (as the comment in the removed code suggests), that change would have to be made in bdrv_temp_snapshot_options() (where the comment suggests otherwise). Addressing this inconsistency before introducing dynamic auto-read-only is important because otherwise we would immediately try to reopen the temporary overlay even though the file is already unlinked. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Make permission changes in reopen less wrongKevin Wolf1-11/+24
The way that reopen interacts with permission changes has one big problem: Both operations are recursive, and the permissions are changes for each node in the reopen queue. For a simple graph that consists just of parent and child, .bdrv_check_perm will be called twice for the child, once recursively when adjusting the permissions of parent, and once again when the child itself is reopened. Even worse, the first .bdrv_check_perm call happens before .bdrv_reopen_prepare was called for the child and the second one is called afterwards. Making sure that .bdrv_check_perm (and the other permission callbacks) are called only once is hard. We can cope with multiple calls right now, but as soon as file-posix gets a dynamic auto-read-only that may need to open a new file descriptor, we get the additional requirement that all of them are after the .bdrv_reopen_prepare call. So reorder things in bdrv_reopen_multiple() to first call .bdrv_reopen_prepare for all involved nodes and only then adjust permissions. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12block: Avoid useless local_errKevin Wolf1-3/+1
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-12qemu-iotests: commit to backing file with auto-read-onlyKevin Wolf2-0/+51
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-12tests/virtio-blk-test: Disable auto-read-onlyKevin Wolf1-1/+1
tests/virtio-blk-test uses a temporary image file that it deletes while QEMU is still running, so it can't be reopened when writers are attached or detached. Disable auto-read-only to keep it always writable. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-12qapi: drop x- from x-block-latency-histogram-setVladimir Sementsov-Ogievskiy3-17/+17
Drop x- and x_ prefixes for latency histograms and update version to 4.0 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-12tests/qemu-iotests: add bitmap resize test 246John Snow3-0/+410
Test that we can actually resize qcow2 images with persistent bitmaps correctly. Throw some other goofy stuff at the test while we're at it, like adding bitmaps of different granularities and at different times. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Tested-by: Eric Blake <eblake@redhat.com> Message-id: 20190311185147.52309-5-vsementsov@virtuozzo.com [vsmentsov: drop \n from the end of test output, test output changed a bit: some bitmaps goes in other order int the output] Signed-off-by: John Snow <jsnow@redhat.com>
2019-03-12block/qcow2-bitmap: Allow resizes with persistent bitmapsJohn Snow3-3/+48
Since we now load all bitmaps into memory anyway, we can just truncate them in-memory and then flush them back to disk. Just in case, we will still check and enforce that this shortcut is valid -- i.e. that any bitmap described on-disk is indeed in-memory and can be modified. If there are any inconsistent bitmaps, we refuse to allow the truncate as we do not actually load these bitmaps into memory, and it isn't safe or reasonable to attempt to truncate corrupted data. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20190311185147.52309-4-vsementsov@virtuozzo.com [vsementsov: drop bitmap flushing, fix block comments style] Signed-off-by: John Snow <jsnow@redhat.com>
2019-03-12block/qcow2-bitmap: Don't check size for IN_USE bitmapVladimir Sementsov-Ogievskiy1-3/+18
We are going to allow image resize when there are persistent bitmaps. It may lead to appearing of inconsistent bitmaps (IN_USE=1) with inconsistent size. But we still want to load them as inconsistent. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20190311185147.52309-3-vsementsov@virtuozzo.com Signed-off-by: John Snow <jsnow@redhat.com>
2019-03-12Merge remote-tracking branch ↵Peter Maydell1-2/+2
'remotes/ehabkost/tags/python-next-pull-request' into staging qmp-shell: fix nested json regression One small bug fix. # gpg: Signature made Tue 12 Mar 2019 02:16:24 GMT # gpg: using RSA key 2807936F984DC5A6 # gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full] # Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6 * remotes/ehabkost/tags/python-next-pull-request: qmp-shell: fix nested json regression Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-03-12.travis.yml: add softmmu check-tcg testsAlex Bennée1-0/+6
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12.travis.yml: separate softfloat from check-tcgAlex Bennée2-1/+7
While used by TCG it is not explicitly part of TCG and the tests can be run standalone in a minimal build. Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg/arm: account for pauth randomnessAlex Bennée1-6/+20
Pointer authentication isn't guaranteed to always detect a clash between different keys. Take this into account in the test by running several times and checking the percentage hit rate of the test. Cc: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg/i386: add memory test to exercise softmmuAlex Bennée1-0/+243
This is a simple test to check various access patterns to memory including unaligned access. Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg/i386: add system mode Hello World testAlex Bennée4-0/+255
This introduces the build framework for simple i386 system tests. The first test is the eponymous "Hello World" which simply outputs the text on the serial port and then exits. I've included the framework for x86_64 but it is not in this series as it is a work in progress. Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg: provide a minilib for system testsAlex Bennée4-0/+180
We will likely want a few common functions to make up for the fact we don't have a libc and we don't want to feel like we are programming by banging rocks together. I've purloined the printf function from: https://git.virtualopensystems.com/dev/tcg_baremetal_tests Although I have tweaked the names to avoid confusing GCC about clashing with builtins. Cc: Alexander Spyridakis <a.spyridakis@virtualopensystems.com> Cc: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg: enable cris base user-mode testsAlex Bennée4-168/+65
This converts the existing Makefile into a Makefile.target and updates it so it can be called by the tcg build system. The original Makefile didn't set -cpu except for the v17 tests however that has broken (I assume because linux-user is a "max" cpu) so here I force it to be crisv17. I've also replicated the GNU simulator targets (run-FOO-on-sim). Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg/cris: align mul operationsAlex Bennée1-0/+11
To avoid: Error: dangerous MULS/MULU location; give it higher alignment Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg/cris: comment out the ccs testAlex Bennée1-7/+7
Evidently upstream gcc doesn't like this opcode. Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg: split cris tests into bare and libc directoriesAlex Bennée102-0/+0
Bare tests are standalone assembly tests that don't require linking to any libc and hence can be built with kernel only compilers. The libc tests need a compiler capable of building properly linked userspace binaries. As we don't have such a cross compiler at the moment we won't be building those tests. Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg/cris: cleanup sys.cAlex Bennée1-48/+52
This is a mini library which provides helper functions to the tests which are all currently written in assembly. A bunch of minor changes: - removed libc related headers (fedora-cris-cross is a system compiler) - re-organised the functions to avoid forward declarations - cleaned up brace usage - restored exit for _fail case - removed tabs and fixed indentation Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/docker: add fedora-cris-cross compilersAlex Bennée1-0/+8
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg/arm: add ARMv6-M UNDEFINED 32-bit instruction testStefan Hajnoczi3-0/+204
Test that 32-bit instructions declared UNDEFINED in the ARMv6-M Reference Manual really do raise an exception. Also test that the 6 32-bit instructions defined in the ARMv6-M Reference Manual do not raise an exception. Based-on: <20181029194519.15628-1-stefanha@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20181129185113.30353-1-stefanha@redhat.com> [AJB: integrated into system tests] Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg/xtensa: enable system testsAlex Bennée2-102/+42
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/docker: add debian-xtensa-cross imagePhilippe Mathieu-Daudé2-0/+42
Xtensa cpu supported: - dc232b - dc233c - csp Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg/mips: fix hello-mips compilationAlex Bennée2-8/+5
The compilation flags for proper building are in the source tree. We also fix exit to 0 so the result is counted as a success. Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg: add gdb runner variantAlex Bennée1-0/+3
With this you can launch a test in gdb with: cd $(BUILD)/tests make -f $(SRC)/tests/tcg/Makefile gdb-$(TEST_NAME) Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg: split run-test into user and system variantsAlex Bennée1-2/+17
We can't rely on shell redirect magic to get things right so lets setup a common output chardev that is expecting to write to files. As we have split run-test up we might as well move the default monitor bits into the call. Finally a little make sophistry is required to correctly quote $(COMMA) and as we don't inherit common rules we have our own little copy here. Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12tests/tcg: add QEMU_OPT option for test runnerAlex Bennée1-1/+4
This will allow tests to modify the QEMU invocation with for example different -cpu stazas without having to define a whole new set of runner types. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12tests/tcg: enable tcg tests for softmmuAlex Bennée2-6/+3
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-03-12tests/tcg: add softmmu awareness to MakefileAlex Bennée2-2/+17
Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>