aboutsummaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)AuthorFilesLines
2021-11-22scripts/device-crash-test: Use a QMP timeoutJohn Snow1-1/+1
Despite all the previous fixes, it's still possible for device-crash-test to wedge itself in the case that QEMU terminates *so quickly* that it doesn't even begin a connection attempt to our QMP client. Python will just joyfully wait ad infinitum for a connection that will now never arrive. The real fix is to use asyncio to simultaneously poll both the health of the launched process AND the connection attempt. That's quite a bit more invasive than just setting a connection timeout, though. Do the very simplest thing for now. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20211118204620.1897674-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-17Revert "device-crash-test: Ignore errors about a bus not being available"Thomas Huth1-1/+0
This reverts commit ca89d15f8e42f2e5eac5bd200af38fdbfb32e875. There is already an entry for this kind of messages earlier in the ERROR_RULE_LIST - when I added this patch, I just got fooled by the other errors that occur due to a race between QMP connection and QEMU terminating early (which still spit out the 'No bus found' messages in their backtrace), but these other problems have now fortunately been tackled by John Snow, so we certainly don't need this duplicated entry here anymore. Message-Id: <20211112072220.108580-1-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-11-17Merge tag 'python-pull-request' of https://gitlab.com/jsnow/qemu into stagingRichard Henderson1-8/+25
Pull request # gpg: Signature made Wed 17 Nov 2021 01:33:06 AM CET # gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E # gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full] * tag 'python-pull-request' of https://gitlab.com/jsnow/qemu: scripts/device-crash-test: hide tracebacks for QMP connect errors scripts/device-crash-test: don't emit AQMP connection errors to stdout scripts/device-crash-test: simplify Exception handling python/aqmp: fix ConnectError string method python/aqmp: Fix disconnect during capabilities negotiation Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-11-16scripts/device-crash-test: hide tracebacks for QMP connect errorsJohn Snow1-3/+18
Generally, the traceback for a connection failure is uninteresting and all we need to know is that the connection attempt failed. Reduce the verbosity in these cases, except when debugging. Signed-off-by: John Snow <jsnow@redhat.com> Reported-by: Thomas Huth <thuth@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com> Message-id: 20211111143719.2162525-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-16scripts/device-crash-test: don't emit AQMP connection errors to stdoutJohn Snow1-0/+6
These errors are expected, so they shouldn't clog up terminal output. In the event that they're *not* expected, we'll be seeing an awful lot more output concerning the nature of the failure. Reported-by: Thomas Huth <thuth@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com> Message-id: 20211111143719.2162525-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-16scripts/device-crash-test: simplify Exception handlingJohn Snow1-6/+2
We don't need to handle KeyboardInterruptError specifically; we can instead tighten the scope of the broad Exception handlers to only catch "Exception", which has the effect of allowing all BaseException classes that do not inherit from Exception to be raised through. KeyboardInterruptError and a few other important ones are BaseExceptions, so this does the same thing with less code. Signed-off-by: John Snow <jsnow@redhat.com> Reported-by: Thomas Huth <thuth@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com> Message-id: 20211111143719.2162525-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-16Merge tag 'pull-nbd-2021-11-16' of https://repo.or.cz/qemu/ericb into stagingRichard Henderson1-0/+3
nbd patches for 2021-11-16 - Rich Jones: Add 'qemu-nbd --selinux-label' option for running Unix socket with appropriate SELinux labeling - Eric Blake: Address clang sanitizer warning # gpg: Signature made Tue 16 Nov 2021 05:32:26 PM CET # gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full] # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full] # gpg: aka "[jpeg image of size 6874]" [full] * tag 'pull-nbd-2021-11-16' of https://repo.or.cz/qemu/ericb: nbd/server: Add --selinux-label option nbd/server: Silence clang sanitizer warning Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-11-16Jobs based on custom runners: add CentOS Stream 8Cleber Rosa5-0/+384
This introduces three different parts of a job designed to run on a custom runner managed by Red Hat. The goals include: a) propose a model for other organizations that want to onboard their own runners, with their specific platforms, build configuration and tests. b) bring awareness to the differences between upstream QEMU and the version available under CentOS Stream, which is "A preview of upcoming Red Hat Enterprise Linux minor and major releases". c) because of b), it should be easier to identify and reduce the gap between Red Hat's downstream and upstream QEMU. The components of this custom job are: I) OS build environment setup code: - additions to the existing "build-environment.yml" playbook that can be used to set up CentOS/EL 8 systems. - a CentOS Stream 8 specific "build-environment.yml" playbook that adds to the generic one. II) QEMU build configuration: a script that will produce binaries with features as similar as possible to the ones built and packaged on CentOS stream 8. III) Scripts that define the minimum amount of testing that the binaries built with the given configuration (point II) under the given OS build environment (point I) should be subjected to. IV) Job definition: GitLab CI jobs that will dispatch the build/test jobs (see points #II and #III) to the machine specifically configured according to #I. Signed-off-by: Cleber Rosa <crosa@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Tested-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20211111160501.862396-2-crosa@redhat.com> Message-Id: <20211115142915.3797652-6-alex.bennee@linaro.org>
2021-11-16nbd/server: Add --selinux-label optionRichard W.M. Jones1-0/+3
Under SELinux, Unix domain sockets have two labels. One is on the disk and can be set with commands such as chcon(1). There is a different label stored in memory (called the process label). This can only be set by the process creating the socket. When using SELinux + SVirt and wanting qemu to be able to connect to a qemu-nbd instance, you must set both labels correctly first. For qemu-nbd the options to set the second label are awkward. You can create the socket in a wrapper program and then exec into qemu-nbd. Or you could try something with LD_PRELOAD. This commit adds the ability to set the label straightforwardly on the command line, via the new --selinux-label flag. (The name of the flag is the same as the equivalent nbdkit option.) A worked example showing how to use the new option can be found in this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1984938 Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1984938 Signed-off-by: Richard W.M. Jones <rjones@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> [eblake: rebase to configure changes, reject --selinux-label if it is not compiled in or not used on a Unix socket] Note that we may relax some of these restrictions at a later date, such as making it possible to label a TCP socket, although it may be smarter to do so as a generic QMP action rather than more one-off command lines in qemu-nbd. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20211115202944.615966-1-eblake@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> [eblake: adjust meson output as suggested by thuth] Signed-off-by: Eric Blake <eblake@redhat.com>
2021-11-11tcg: Remove TCI experimental statusPhilippe Mathieu-Daudé1-2/+1
The following commits (released in v6.0.0) made raised the quality of the TCI backend to the other TCG architectures, thus is is not considerated experimental anymore: - c6fbea47664..2f74f45e32b - dc09f047edd..9e9acb7b348 - b6139eb0578..2fc6f16ca5e - dbcbda2cd84..5e8892db93f Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20211106111457.517546-1-f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-11-02Move the l2tpv3 test from configure to meson.buildThomas Huth1-0/+3
And while we're at it, also provide a proper entry for this feature in meson_options.txt, so that people who don't need it have a knob to disable this feature. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20211028185910.1729744-3-thuth@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-11-02meson: bump submodule to 0.59.3Paolo Bonzini1-16/+0
This gains some bugfixes, especially: - it fixes the introspection of array options. While technically we still support Meson 0.58.2, this issue only appears when adding a new option and not if the user is just building QEMU. In the relatively rare case of a contributor using --meson to point to a 0.58 version, review can catch spurious changes to scripts/meson-buildoptions.sh easily. - it fixes "meson test" when it is not the process group leader. Make is the process group leader when "make check" invokes "meson test", so this is a requirement for using it as a test harness. Tested-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-11-01iotests: Accommodate async QMP Exception classesJohn Snow1-1/+2
(But continue to support the old ones for now, too.) There are very few cases of any user of QEMUMachine or a subclass thereof relying on a QMP Exception type. If you'd like to check for yourself, you want to grep for all of the derivatives of QMPError, excluding 'AQMPError' and its derivatives. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20211026175612.4127598-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-10-29qapi: Extend -compat to set policy for unstable interfacesMarkus Armbruster2-8/+12
New option parameters unstable-input and unstable-output set policy for unstable interfaces just like deprecated-input and deprecated-output set policy for deprecated interfaces (see commit 6dd75472d5 "qemu-options: New -compat to set policy for deprecated interfaces"). This is intended for testing users of the management interfaces. It is experimental. For now, this covers only syntactic aspects of QMP, i.e. stuff tagged with feature 'unstable'. We may want to extend it to cover semantic aspects, or the command line. Note that there is no good way for management application to detect presence of these new option parameters: they are not visible output of query-qmp-schema or query-command-line-options. Tolerable, because it's meant for testing. If running with -compat fails, skip the test. Signed-off-by: Markus Armbruster <armbru@redhat.com> Acked-by: John Snow <jsnow@redhat.com> Message-Id: <20211028102520.747396-10-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [Doc comments fixed up]
2021-10-29qapi: Generalize enum member policy checkingMarkus Armbruster1-9/+11
The code to check enumeration value policy can see special feature flag 'deprecated' in QEnumLookup member flags[value]. I want to make feature flag 'unstable' visible there as well, so I can add policy for it. Instead of extending flags[], replace it by @special_features (a bitset of QapiSpecialFeature), because that's how special features get passed around elsewhere. Signed-off-by: Markus Armbruster <armbru@redhat.com> Acked-by: John Snow <jsnow@redhat.com> Message-Id: <20211028102520.747396-8-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-10-29qapi: Generalize command policy checkingMarkus Armbruster1-5/+4
The code to check command policy can see special feature flag 'deprecated' as command flag QCO_DEPRECATED. I want to make feature flag 'unstable' visible there as well, so I can add policy for it. To let me make it visible, add member @special_features (a bitset of QapiSpecialFeature) to QmpCommand, and adjust the generator to pass it through qmp_register_command(). Then replace "QCO_DEPRECATED in @flags" by QAPI_DEPRECATED in @special_features", and drop QCO_DEPRECATED. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: John Snow <jsnow@redhat.com> Message-Id: <20211028102520.747396-7-armbru@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2021-10-29qapi: Generalize struct member policy checkingMarkus Armbruster1-7/+7
The generated visitor functions call visit_deprecated_accept() and visit_deprecated() when visiting a struct member with special feature flag 'deprecated'. This makes the feature flag visible to the actual visitors. I want to make feature flag 'unstable' visible there as well, so I can add policy for it. To let me make it visible, replace these functions by visit_policy_reject() and visit_policy_skip(), which take the member's special features as an argument. Note that the new functions have the opposite sense, i.e. the return value flips. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20211028102520.747396-6-armbru@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> [Unbreak forward visitor]
2021-10-29qapi: Tools for sets of special feature flags in generated codeMarkus Armbruster2-0/+11
New enum QapiSpecialFeature enumerates the special feature flags. New helper gen_special_features() returns code to represent a collection of special feature flags as a bitset. The next few commits will put them to use. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-Id: <20211028102520.747396-5-armbru@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-10-29qapi: Eliminate QCO_NO_OPTIONS for a slight simplificationMarkus Armbruster1-4/+1
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-Id: <20211028102520.747396-4-armbru@redhat.com>
2021-10-27qapi: Implement deprecated-input={reject,crash} for enum valuesMarkus Armbruster1-1/+16
This copies the code implementing the policy from qapi/qmp-dispatch.c to qapi/qobject-input-visitor.c. Tolerable, but if we acquire more copies, we should look into factoring them out. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Peter Krempa <pkrempa@redhat.com> Acked-by: Peter Krempa <pkrempa@redhat.com> Message-Id: <20211025042405.3762351-5-armbru@redhat.com>
2021-10-27qapi: Add feature flags to enum membersMarkus Armbruster3-5/+25
This is quite similar to commit 84ab008687 "qapi: Add feature flags to struct members", only for enums instead of structs. Special feature flag 'deprecated' is silently ignored there. This is okay only because it will be implemented shortly. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20211025042405.3762351-3-armbru@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
2021-10-27qapi: Enable enum member introspection to show more than nameMarkus Armbruster1-4/+14
The next commit will add feature flags to enum members. There's a problem, though: query-qmp-schema shows an enum type's members as an array of member names (SchemaInfoEnum member @values). If it showed an array of objects with a name member, we could simply add more members to these objects. Since it's just strings, we can't. I can see three ways to correct this design mistake: 1. Do it the way we should have done it, plus compatibility goo. We want a ['SchemaInfoEnumMember'] member in SchemaInfoEnum. Since changing @values would be a compatibility break, add a new member @members instead. @values is now redundant. In my testing, output of qemu-system-x86_64's query-qmp-schema grows by 11% (18.5KiB). We can deprecate @values now and drop it later. This will break outmoded clients. Well-behaved clients such as libvirt are expected to break cleanly. 2. Like 1, but omit "boring" elements of @member, and empty @member. @values does not become redundant. @members augments it. Somewhat cumbersome, but output of query-qmp-schema grows only as we make enum members non-boring. There is nothing to deprecate here. 3. Versioned query-qmp-schema. query-qmp-schema provides either @values or @members. The QMP client can select which version it wants. There is no redundant output. We can deprecate old versions and eventually drop them. This will break outmoded clients. Breaking cleanly is easier than for 1. While 1 and 2 operate within the common rules for compatible evolution apply (section "Compatibility considerations" in docs/devel/qapi-code-gen.rst), 3 bypasses them. Attractive when operating within the rules is just too awkward. Not the case here. This commit implements 1. Libvirt developers prefer it. Deprecate @values in favour of @members. Since query-qmp-schema compatibility is pretty fundamental for management applications, an extended grace period is advised. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Peter Krempa <pkrempa@redhat.com> Acked-by: Peter Krempa <pkrempa@redhat.com> Message-Id: <20211025042405.3762351-2-armbru@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
2021-10-23analyze-migration.py: fix extract contents ('-x') errorsLaurent Vivier1-2/+2
When we try to use 'analyze-migration.py -x' with python3, we have the following errors: Traceback (most recent call last): File "scripts/analyze-migration.py", line 593, in <module> f.write(jsonenc.encode(dump.vmsd_desc)) TypeError: a bytes-like object is required, not 'str' Traceback (most recent call last): File "scripts/analyze-migration.py", line 601, in <module> f.write(jsonenc.encode(dict)) TypeError: a bytes-like object is required, not 'str' This happens because the file 'f' is open in binary mode while jsonenc.encode() returns a string. The results are human-readable files, 'desc.json' and 'state.json', so there is no reason to use the binary mode. Signed-off-by: Laurent Vivier <lvivier@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20211015131645.501281-3-lvivier@redhat.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-10-23analyze-migration.py: fix a long standing typoLaurent Vivier1-1/+1
The parameters of '-d' can be either 'state' or 'desc', not 'dump' as it is reported in the error message. Fixes: b17425701d66 ("Add migration stream analyzation script") Signed-off-by: Laurent Vivier <lvivier@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20211015131645.501281-2-lvivier@redhat.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-10-20roms/edk2: Only initialize required submodulesPhilippe Mathieu-Daudé1-1/+6
The EDK2 firmware images built to test QEMU do not require the following submodules: - MdeModulePkg/Universal/RegularExpressionDxe/oniguruma - UnitTestFrameworkPkg/Library/CmockaLib/cmocka The only submodules required are: - ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 - BaseTools/Source/C/BrotliCompress/brotli - CryptoPkg/Library/OpensslLib/openssl - MdeModulePkg/Library/BrotliCustomDecompressLib/brotli Adapt the buildsys machinery to only initialize the required submodules. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Gerd Hoffmann <kraxel@redhat.com> Message-Id: <20211018105816.2663195-3-philmd@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-10-15block-coroutine-wrapper.py: support BlockBackend first argumentVladimir Sementsov-Ogievskiy1-2/+10
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211006131718.214235-7-vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2021-10-14configure: automatically parse command line for meson -D optionsPaolo Bonzini2-0/+349
Right now meson_options.txt lists about 90 options. Each option needs code in configure to parse it and pass the option down to Meson as a -D command-line argument; in addition the default must be duplicated between configure and meson_options.txt. This series tries to remove the code duplication by generating the case statement for those --enable and --disable options, as well as the corresponding help text. About 80% of the options can be handled completely by the new mechanism. Eight meson options are not of the --enable/--disable kind. Six more need to be parsed in configure for various reasons documented in the patch, but they still have their help automatically generated. The advantages are: - less code in configure - parsing and help is more consistent (for example --enable-blobs was not supported) - options are described entirely in one place, meson_options.txt. This make it more attractive to use Meson options instead of hand-crafted configure options and config-host.mak A few options change name: --enable-tcmalloc and --enable-jemalloc become --enable-malloc={tcmalloc,jemalloc}; --disable-blobs becomes --disable-install-blobs; --enable-trace-backend becomes --enable-trace-backends. However, the old names are allowed for backwards compatibility. Message-Id: <20211007130829.632254-19-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-14meson-buildoptions: include list of tracing backendsPaolo Bonzini1-0/+16
Manually patch the introspection data to include the tracing backends. This works around a deficiency in Meson that will be fixed by https://github.com/mesonbuild/meson/pull/9395. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-14configure: prepare for auto-generated option parsingPaolo Bonzini2-0/+77
Prepare the configure script and Makefile for automatically generated help and parsing. Because we need to run the script to generate the full help, we cannot rely on the user supplying the path to a Python interpreter with --python; therefore, the introspection output is parsed into shell functions and stored in scripts/. The converter is written in Python as standard for QEMU, and this commit contains a stub. Tested-by: Thomas Huth <thuth@redhat.com> Message-Id: <20211007130829.632254-18-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-14trace: move configuration from configure to MesonPaolo Bonzini1-1/+1
Cc: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20211007130829.632254-4-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-05meson: switch minimum meson version to 0.58.2, minimum recommended to 0.59.2Paolo Bonzini1-5/+2
Meson 0.58.2 does not need b_staticpic=$pie anymore, and has stabilized the keyval module. Remove the workaround and use a few replacements for features deprecated in the 0.57.0 release cycle. One feature that we would like to use is passing dependencies to summary. However, that was broken in 0.59.0 and 0.59.1. Therefore, use the embedded Meson if the host has anything older than 0.59.2, but allow --meson= to use 0.58.2. Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-02qapi/parser: enable pylint checksJohn Snow1-2/+1
Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-14-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/parser: Silence too-few-public-methods warningJohn Snow1-0/+3
Eh. Not worth the fuss today. There are bigger fish to fry. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-13-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/parser: enable mypy checksJohn Snow1-5/+0
Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-12-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/parser: Add FIXME for consolidating JSON-related typesJohn Snow1-0/+4
The fix for this comment is forthcoming in a future commit, but this will keep me honest. The linting configuration in ./python/setup.cfg prohibits 'FIXME' comments. A goal of this long-running series is to move ./scripts/qapi to ./python/qemu/qapi so that the QAPI generator is regularly type-checked by GitLab CI. This comment is a time-bomb to force me to address this issue prior to that step. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-11-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/parser: add type hint annotations (QAPIDoc)John Snow1-28/+39
Annotations do not change runtime behavior. This commit consists of only annotations. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-10-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/parser: add import cycle workaroundJohn Snow1-4/+11
Adding static types causes a cycle in the QAPI generator: [schema -> expr -> parser -> schema]. It exists because the QAPIDoc class needs the names of types defined by the schema module, but the schema module needs to import both expr.py/parser.py to do its actual parsing. Ultimately, the layering violation is that parser.py should not have any knowledge of specifics of the Schema. QAPIDoc performs double-duty here both as a parser *and* as a finalized object that is part of the schema. In this patch, add the offending type hints alongside the workaround to avoid the cycle becoming a problem at runtime. See https://mypy.readthedocs.io/en/latest/runtime_troubles.html#import-cycles for more information on this workaround technique. I see three ultimate resolutions here: (1) Just keep this patch and use the TYPE_CHECKING trick to eliminate the cycle which is only present during static analysis. (2) Don't bother to annotate connect_member() et al, give them 'object' or 'Any'. I don't particularly like this, because it diminishes the usefulness of type hints for documentation purposes. Still, it's an extremely quick fix. (3) Reimplement doc <--> definition correlation directly in schema.py, integrating doc fields directly into QAPISchemaMember and relieving the QAPIDoc class of the responsibility. Users of the information would instead visit the members first and retrieve their documentation instead of the inverse operation -- visiting the documentation and retrieving their members. My preference is (3), but in the short-term (1) is the easiest way to have my cake (strong type hints) and eat it too (Not have import cycles). Do (1) for now, but plan for (3). Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-9-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/parser: Introduce NullSectionJohn Snow1-11/+16
Here's the weird bit. QAPIDoc generally expects -- virtually everywhere -- that it will always have a current section. The sole exception to this is in the case that end_comment() is called, which leaves us with *no* section. However, in this case, we also don't expect to actually ever mutate the comment contents ever again. NullSection is just a Null-object that allows us to maintain the invariant that we *always* have a current section, enforced by static typing -- allowing us to type that field as QAPIDoc.Section instead of the more ambiguous Optional[QAPIDoc.Section]. end_section is renamed to switch_section and now accepts as an argument the new section to activate, clarifying that no callers ever just unilaterally end a section; they only do so when starting a new section. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-8-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/parser: clarify _end_section() logicJohn Snow1-7/+15
The "if self._section" clause in end_section is mysterious: In which circumstances might we end a section when we don't have one? QAPIDoc always expects there to be a "current section", only except after a call to end_comment(). This actually *shouldn't* ever be 'None', so let's remove that logic so I don't wonder why it's like this again in three months. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-7-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/parser: remove FIXME comment from _append_body_lineJohn Snow1-2/+4
True, we do not check the validity of this symbol -- but we don't check the validity of definition names during parse, either -- that happens later, during the expr check. I don't want to introduce a dependency on expr.py:check_name_str here and introduce a cycle. Instead, rest assured that a documentation block is required for each definition. This requirement uses the names of each section to ensure that we fulfilled this requirement. e.g., let's say that block-core.json has a comment block for "Snapshot!Info" by accident. We'll see this error message: In file included from ../../qapi/block.json:8: ../../qapi/block-core.json: In struct 'SnapshotInfo': ../../qapi/block-core.json:38: documentation comment is for 'Snapshot!Info' That's a pretty decent error message. Now, let's say that we actually mangle it twice, identically: ../../qapi/block-core.json: In struct 'Snapshot!Info': ../../qapi/block-core.json:38: struct has an invalid name That's also pretty decent. If we forget to fix it in both places, we'll just be back to the first error. Therefore, let's just drop this FIXME and adjust the error message to not imply a more thorough check than is actually performed. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-6-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/parser: fix unused check_args_section argumentsJohn Snow1-8/+10
Pylint informs us we're not using these arguments. Oops, it's right. Correct the error message and remove the remaining unused parameter. Fix test output now that the error message is improved. Fixes: e151941d1b Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-4-jsnow@redhat.com> [Commit message formatting tweaked] Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/gen: use dict.items() to iterate over _modulesJohn Snow1-2/+1
New pylint warning. I could silence it, but this is the only occurrence in the entire tree, including everything in iotests/ and python/. Easier to just change this one instance. (The warning is emitted in cases where you are fetching the values anyway, so you may as well just take advantage of the iterator to avoid redundant lookups.) Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-3-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02qapi/pylintrc: ignore 'consider-using-f-string' warningJohn Snow1-0/+1
Pylint 2.11.x adds this warning. We're not yet ready to pursue that conversion, so silence it for now. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20210930205716.1148693-2-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-09-27qapi: Drop simple unionsMarkus Armbruster2-94/+28
Simple unions predate flat unions. Having both complicates the QAPI schema language and the QAPI generator. We haven't been using simple unions in new code for a long time, because they are less flexible and somewhat awkward on the wire. The previous commits eliminated simple union from the tree. Now drop them from the QAPI schema language entirely, and update mentions of "flat union" to just "union". Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210917143134.412106-22-armbru@redhat.com>
2021-09-25qapi: Stop enforcing "type name should not end in 'Kind'Markus Armbruster1-3/+3
I'm about to convert simple unions to flat unions, then drop simple union support. The conversion involves making the implict enum types explicit. To reduce churn, I'd like to name them exactly like the implicit types they replace. However, these names are reserved for the generator's use. They won't be once simple unions are gone. Stop enforcing this naming rule now rather than then. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210917143134.412106-3-armbru@redhat.com>
2021-09-15simplebench: add img_bench_templater.pyVladimir Sementsov-Ogievskiy2-0/+157
Add simple grammar-parsing template benchmark. New tool consume test template written in bash with some special grammar injections and produces multiple tests, run them and finally print a performance comparison table of different tests produced from one template. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210824101517.59802-2-vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-08qapi: Fix bogus error for 'if': { 'not': '' }Markus Armbruster1-8/+13
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20210908045428.2689093-6-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> [check_infix()'s type hint fixed]
2021-09-08qapi: Bury some unused code in class IndentationMarkus Armbruster1-6/+1
.__int__() has never been used. Drop it. .decrease() raises ArithmeticError when asked to decrease indentation level below zero. Nothing catches it. It's a programming error. Dumb down to assert. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20210908045428.2689093-4-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-08qapi: Drop Indentation.__bool__()Markus Armbruster1-6/+3
Intentation.__bool__() is not worth its keep: it has just one user, which can just as well check .__str__() instead. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20210908045428.2689093-3-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-08qapi: Fix a botched type annotationMarkus Armbruster1-1/+2
Mypy is unhappy: $ mypy --config-file=scripts/qapi/mypy.ini `git-ls-files scripts/qapi/\*py` scripts/qapi/common.py:208: error: Function is missing a return type annotation scripts/qapi/common.py:227: error: Returning Any from function declared to return "str" Messed up in commit ccea6a8637 "qapi: Factor common recursion out of cgen_ifcond(), docgen_ifcond()". Tidy up. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20210908045428.2689093-2-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
generated by cgit v1.1 at 2024-10-13 05:25:56 +0000