aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-06-01python: add devel package requirements to setuptoolsJohn Snow5-9/+27
setuptools doesn't have a formal understanding of development requires, but it has an optional feataures section. Fine; add a "devel" feature and add the requirements to it. To avoid duplication, we can modify pipenv to install qemu[devel] instead. This enables us to run invocations like "pip install -e .[devel]" and test the package on bleeding-edge packages beyond those specified in Pipfile.lock. Importantly, this also allows us to install the qemu development packages in a non-networked mode: `pip3 install --no-index -e .[devel]` will now fail if the proper development dependencies are not already met. This can be useful for automated build scripts where fetching network packages may be undesirable. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-27-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/qemu: add qemu package itself to pipenvJohn Snow2-2/+8
This adds the python qemu packages themselves to the pipenv manifest. 'pipenv sync' will create a virtual environment sufficient to use the SDK. 'pipenv sync --dev' will create a virtual environment sufficient to use and test the SDK (with pylint, mypy, isort, flake8, etc.) The qemu packages are installed in 'editable' mode; all changes made to the python package inside the git tree will be reflected in the installed package without reinstallation. This includes changes made via git pull and so on. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-26-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/qemu: add isort to pipenvJohn Snow2-2/+3
isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret certain "from ..." clauses that are not related to imports. isort < 5.1.1 has a bug where it does not handle comments near import statements correctly. Require 5.1.2 or greater. isort can be run (in "check" mode) with 'isort -c qemu' from the python root. isort can also be used to fix/rewrite import order automatically by using 'isort qemu'. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-25-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: move .isort.cfg into setup.cfgJohn Snow2-7/+8
Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-24-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add mypy to pipenvJohn Snow3-1/+38
0.730 appears to be about the oldest version that works with the features we want, including nice human readable output (to make sure iotest 297 passes), and type-parameterized Popen generics. 0.770, however, supports adding 'strict' to the config file, so require at least 0.770. Now that we are checking a namespace package, we need to tell mypy to allow PEP420 namespaces, so modify the mypy config as part of the move. mypy can now be run from the python root by typing 'mypy -p qemu'. A note on mypy invocation: Running it as "mypy qemu/" changes the import path detection mechanisms in mypy slightly, and it will fail. See https://github.com/python/mypy/issues/8584 for a decent entry point with more breadcrumbs on the various behaviors that contribute to this subtle difference. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-23-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: move mypy.ini into setup.cfgJohn Snow2-4/+5
mypy supports reading its configuration values from a central project configuration file; do so. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-22-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: Add flake8 to pipenvJohn Snow2-1/+51
flake8 3.5.x does not support the --extend-ignore syntax used in the .flake8 file to gracefully extend default ignores, so 3.6.x is our minimum requirement. There is no known upper bound. flake8 can be run from the python/ directory with no arguments. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-21-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add excluded dirs to flake8 configJohn Snow1-0/+2
Instruct flake8 to avoid certain well-known directories created by python tooling that it ought not check. Note that at-present, nothing actually creates a ".venv" directory; but it is in such widespread usage as a de-facto location for a developer's virtual environment that it should be excluded anyway. A forthcoming commit canonizes this with a "make venv" command. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-20-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: move flake8 config to setup.cfgJohn Snow2-2/+3
Update the comment concerning the flake8 exception to match commit 42c0dd12, whose commit message stated: A note on the flake8 exception: flake8 will warn on *any* bare except, but pylint's is context-aware and will suppress the warning if you re-raise the exception. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-19-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add pylint to pipenvJohn Snow2-0/+131
We are specifying >= pylint 2.8.x for several reasons: 1. For setup.cfg support, added in pylint 2.5.x 2. To specify a version that has incompatibly dropped bad-whitespace checks (2.6.x) 3. 2.7.x fixes "unsubscriptable" warnings in Python 3.9 4. 2.8.x adds a new, incompatible 'consider-using-with' warning that must be disabled in some cases. These pragmas cause warnings themselves in 2.7.x. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-18-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: move pylintrc into setup.cfgJohn Snow2-58/+29
Delete the empty settings now that it's sharing a home with settings for other tools. pylint can now be run from this folder as "pylint qemu". Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-17-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add pylint import exceptionsJohn Snow3-2/+5
Pylint 2.5.x - 2.7.x have regressions that make import checking inconsistent, see: https://github.com/PyCQA/pylint/issues/3609 https://github.com/PyCQA/pylint/issues/3624 https://github.com/PyCQA/pylint/issues/3651 Pinning to 2.4.4 is worse, because it mandates versions of shared dependencies that are too old for features we want in isort and mypy. Oh well. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-16-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: Add pipenv supportJohn Snow2-0/+14
pipenv is a tool used for managing virtual environments with pinned, explicit dependencies. It is used for precisely recreating python virtual environments. pipenv uses two files to do this: (1) Pipfile, which is similar in purpose and scope to what setup.cfg lists. It specifies the requisite minimum to get a functional environment for using this package. (2) Pipfile.lock, which is similar in purpose to `pip freeze > requirements.txt`. It specifies a canonical virtual environment used for deployment or testing. This ensures that all users have repeatable results. The primary benefit of using this tool is to ensure *rock solid* repeatable CI results with a known set of packages. Although I endeavor to support as many versions as I can, the fluid nature of the Python toolchain often means tailoring code for fairly specific versions. Note that pipenv is *not* required to install or use this module; this is purely for the sake of repeatable testing by CI or developers. Here, a "blank" pipfile is added with no dependencies, but specifies Python 3.6 for the virtual environment. Pipfile will specify our version minimums, while Pipfile.lock specifies an exact loadout of packages that were known to operate correctly. This latter file provides the real value for easy setup of container images and CI environments. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-15-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add MANIFEST.inJohn Snow2-0/+5
When creating a source or binary distribution via 'python3 setup.py <sdist|bdist>', the VERSION and PACKAGE.rst files aren't bundled by default. Create a MANIFEST.in file that instructs the build tools to include these so that installation from these files won't fail. This is required by 'tox', as well as by the tooling needed to upload packages to PyPI. Exclude the 'README.rst' file -- that's intended as a guidebook to our source tree, not a file that needs to be distributed. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-14-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add directory structure README.rst filesJohn Snow5-0/+74
Add short readmes to python/, python/qemu/, python/qemu/machine, python/qemu/qmp, and python/qemu/utils that explain the directory hierarchy. These readmes are visible when browsing the source on e.g. gitlab/github and are designed to help new developers/users quickly make sense of the source tree. They are not designed for inclusion in a published manual. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-13-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add VERSION fileJohn Snow2-0/+2
Python infrastructure as it exists today is not capable reliably of single-sourcing a package version from a parent directory. The authors of pip are working to correct this, but as of today this is not possible. The problem is that when using pip to build and install a python package, it copies files over to a temporary directory and performs its build there. This loses access to any information in the parent directory, including git itself. Further, Python versions have a standard (PEP 440) that may or may not follow QEMU's versioning. In general, it does; but naturally QEMU does not follow PEP 440. To avoid any automatically-generated conflict, a manual version file is preferred. I am proposing: - Python tooling follows the QEMU version, indirectly, but with a major version of 0 to indicate that the API is not expected to be stable. This would mean version 0.5.2.0, 0.5.1.1, 0.5.3.0, etc. - In the event that a Python package needs to be updated independently of the QEMU version, a pre-release alpha version should be preferred, but *only* after inclusion to the qemu development or stable branches. e.g. 0.5.2.0a1, 0.5.2.0a2, and so on should be preferred prior to 5.2.0's release. - The Python core tooling makes absolutely no version compatibility checks or constraints. It *may* work with releases of QEMU from the past or future, but it is not required to. i.e., "qemu.machine" will, for now, remain in lock-step with QEMU. - We reserve the right to split the qemu package into independently versioned subpackages at a later date. This might allow for us to begin versioning QMP independently from QEMU at a later date, if we so choose. Implement this versioning scheme by adding a VERSION file and setting it to 0.6.0.0a1. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-12-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add qemu package installerJohn Snow3-0/+78
Add setup.cfg and setup.py, necessary for installing a package via pip. Add a ReST document (PACKAGE.rst) explaining the basics of what this package is for and who to contact for more information. This document will be used as the landing page for the package on PyPI. List the subpackages we intend to package by name instead of using find_namespace because find_namespace will naively also packages tests, things it finds in the dist/ folder, etc. I could not figure out how to modify this behavior; adding allow/deny lists to setuptools kept changing the packaged hierarchy. This works, roll with it. I am not yet using a pyproject.toml style package manifest, because "editable" installs are not defined (yet?) by PEP-517/518. I consider editable installs crucial for development, though they have (apparently) always been somewhat poorly defined. Pip now (19.2 and later) now supports editable installs for projects using pyproject.toml manifests, but might require the use of the --no-use-pep517 flag, which somewhat defeats the point. Full support for setup.py-less editable installs was not introduced until pip 21.1.1: https://github.com/pypa/pip/pull/9547/commits/7a95720e796a5e56481c1cc20b6ce6249c50f357 For now, while the dust settles, stick with the de-facto setup.py/setup.cfg combination supported by setuptools. It will be worth re-evaluating this point again in the future when our supported build platforms all ship a fairly modern pip. Additional reading on this matter: https://github.com/pypa/packaging-problems/issues/256 https://github.com/pypa/pip/issues/6334 https://github.com/pypa/pip/issues/6375 https://github.com/pypa/pip/issues/6434 https://github.com/pypa/pip/issues/6438 Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-11-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: create qemu packagesJohn Snow17-32/+83
move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and update import directives across the tree. This is done to create a PEP420 namespace package, in which we may create subpackages. To do this, the namespace directory ("qemu") should not have any modules in it. Those files will go into new 'machine', 'qmp' and 'utils' subpackages instead. Implement machine/__init__.py making the top-level classes and functions from its various modules available directly inside the package. Change qmp.py to qmp/__init__.py similarly, such that all of the useful QMP library classes are available directly from "qemu.qmp" instead of "qemu.qmp.qmp". Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01iotests/297: add --namespace-packages to mypy argumentsJohn Snow1-0/+1
mypy is kind of weird about how it handles imports. For legacy reasons, it won't load PEP 420 namespaces, because of logic implemented prior to that becoming a standard. So, if you plan on using any, you have to pass --namespace-packages. Alright, fine. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-9-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: Trim line length to below 80 charsJohn Snow1-1/+1
One more little delinting fix that snuck in. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: disable warning for Popen in _launch()John Snow1-0/+3
We handle this resource rather meticulously in shutdown/kill/wait/__exit__ et al, through the laborious mechanisms in _do_shutdown(). Quiet this pylint warning here. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-7-jsnow@redhat.com Message-id: 20210517184808.3562549-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: Disable pylint warning for open() in _pre_launchJohn Snow1-1/+5
Shift the open() call later so that the pylint pragma applies *only* to that one open() call. Add a note that suggests why this is safe: the resource is unconditionally cleaned up in _post_shutdown(). _post_shutdown is called after failed launches (see launch()), and unconditionally after every call to shutdown(), and therefore also on __exit__. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-6-jsnow@redhat.com Message-id: 20210517184808.3562549-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/console_socket: Add a pylint ignoreJohn Snow1-0/+1
We manage cleaning up this resource ourselves. Pylint should shush. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-5-jsnow@redhat.com Message-id: 20210517184808.3562549-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: use subprocess.run instead of subprocess.PopenJohn Snow1-6/+9
use run() instead of Popen() -- to assert to pylint that we are not forgetting to close a long-running program. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-4-jsnow@redhat.com Message-id: 20210517184808.3562549-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: use subprocess.DEVNULL instead of open(os.path.devnull)John Snow1-4/+2
One less file resource to manage, and it helps quiet some pylint >= 2.8.0 warnings about not using a with-context manager for the open call. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-3-jsnow@redhat.com Message-id: 20210517184808.3562549-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/console_socket: avoid one-letter variableJohn Snow1-5/+5
Fixes pylint warnings. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20210527211715.394144-2-jsnow@redhat.com Message-id: 20210517184808.3562549-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01acceptance tests: bump Avocado version to 88.1Willian Rampazzo1-1/+1
Besides some internal changes, new features, and bug fixes, on the QEMU side, this version fixes the following message seen when running the acceptance tests: "Error running method "pre_tests" of plugin "fetchasset": 'bytes' object has no attribute 'encode'". The release notes are available at https://avocado-framework.readthedocs.io/en/latest/releases/88_0.html. Signed-off-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210520204747.210764-2-willianr@redhat.com> Acked-by: Cleber Rosa <crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01tests/acceptance/virtiofs_submounts.py: fix setup of SSH pubkeyCleber Rosa1-1/+1
The public key argument should be a path to a file, and not the public key data. Reported-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210412044644.55083-12-crosa@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Acceptance Tests: introduce CPU hotplug testCleber Rosa1-0/+37
Even though there are qtest based tests for hotplugging CPUs (from which this test took some inspiration from), this one adds checks from a Linux guest point of view. It should also serve as an example for tests that follow a similar pattern and need to interact with QEMU (via qmp) and with the Linux guest via SSH. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Message-Id: <20210412044644.55083-11-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Acceptance Tests: add basic documentation on LinuxTest base classCleber Rosa1-0/+26
Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20210412044644.55083-10-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01tests/acceptance/virtiofs_submounts.py: remove launch_vm()Cleber Rosa1-8/+5
The LinuxTest class' launch_and_wait() method now behaves the same way as this test's custom launch_vm(), so let's just use the upper layer (common) method. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210412044644.55083-9-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Acceptance Tests: set up SSH connection by default after boot for LinuxTestCleber Rosa3-11/+13
The LinuxTest specifically targets users that need to interact with Linux guests. So, it makes sense to give a connection by default, and avoid requiring it as boiler-plate code. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210412044644.55083-8-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Acceptance Tests: make username/password configurableCleber Rosa1-2/+4
This makes the username/password used for authentication configurable, because some guest operating systems may have restrictions on accounts to be used for logins, and it just makes it better documented. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210412044644.55083-7-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Acceptance Tests: add port redirection for ssh by defaultCleber Rosa2-5/+4
For users of the LinuxTest class, let's set up the VM with the port redirection for SSH, instead of requiring each test to set the same arguments. It also sets the network device, by default, to virtio-net. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210412044644.55083-6-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Acceptance Tests: move useful ssh methods to base classCleber Rosa3-75/+50
Both the virtiofs submounts and the linux ssh mips malta tests contains useful methods related to ssh that deserve to be made available to other tests. Let's move them to an auxiliary, mix-in class that will be used on the base LinuxTest class. The method that helps with setting up an ssh connection will now support both key and password based authentication, defaulting to key based. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210412044644.55083-5-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Python: add utility function for retrieving port redirectionCleber Rosa5-32/+78
Slightly different versions for the same utility code are currently present on different locations. This unifies them all, giving preference to the version from virtiofs_submounts.py, because of the last tweaks added to it. While at it, this adds a "qemu.utils" module to host the utility function and a test. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210412044644.55083-4-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> [Squashed in below fix. --js] Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210601154546.130870-2-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01tests/acceptance/virtiofs_submounts.py: evaluate string not lengthCleber Rosa1-1/+1
If the vmlinuz variable is set to anything that evaluates to True, then the respective arguments should be set. If the variable contains an empty string, than it will evaluate to False, and the extra arguments will not be set. This keeps the same logic, but improves readability a bit. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210412044644.55083-3-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01tests/acceptance/virtiofs_submounts.py: add missing accel tagCleber Rosa1-0/+1
The tag is useful to select tests that depend/use a particular feature. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Message-Id: <20210412044644.55083-2-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Python: expose QEMUMachine's temporary directoryCleber Rosa3-12/+20
Each instance of qemu.machine.QEMUMachine currently has a "test directory", which may not have any relation to a "test", and it's really a temporary directory. Users instantiating the QEMUMachine class will be able to set the location of the directory that will *contain* the QEMUMachine unique temporary directory, so that parameter name has been changed from test_dir to base_temp_dir. A property has been added to allow users to access it without using private attributes, and with that, the directory is created on first use of the property. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210211220146.2525771-3-crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-05-30Merge remote-tracking branch 'remotes/kraxel/tags/usb-20210528-pull-request' ↵Peter Maydell5-11/+20
into staging usb: bugfixes for hid and xhci. # gpg: Signature made Fri 28 May 2021 15:21:51 BST # gpg: using RSA key A0328CFFB93A17A79901FE7D4CB6D8EED3E87138 # gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full] # gpg: aka "Gerd Hoffmann <gerd@kraxel.org>" [full] # gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full] # Primary key fingerprint: A032 8CFF B93A 17A7 9901 FE7D 4CB6 D8EE D3E8 7138 * remotes/kraxel/tags/usb-20210528-pull-request: hw/usb: hcd-xhci-pci: Fix spec violation of IP flag for MSI/MSI-X hw/usb: hcd-xhci-pci: Raise MSI/MSI-X interrupts only when told to hw/input/hid: Add support for keys of jp106 keyboard. Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-30Merge remote-tracking branch 'remotes/kraxel/tags/vga-20210527-pull-request' ↵Peter Maydell22-117/+799
into staging virtio-gpu: add blob resource support. vhost-user-gpu: security fixes. # gpg: Signature made Thu 27 May 2021 15:23:25 BST # gpg: using RSA key A0328CFFB93A17A79901FE7D4CB6D8EED3E87138 # gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full] # gpg: aka "Gerd Hoffmann <gerd@kraxel.org>" [full] # gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full] # Primary key fingerprint: A032 8CFF B93A 17A7 9901 FE7D 4CB6 D8EE D3E8 7138 * remotes/kraxel/tags/vga-20210527-pull-request: (23 commits) virtio-gpu: Update cursor data using blob virtio-gpu: Add virtio_gpu_set_scanout_blob virtio-gpu: Factor out update scanout virtio-gpu: Add helpers to create and destroy dmabuf objects ui/pixman: Add qemu_pixman_to_drm_format() virtio-gpu: Add virtio_gpu_resource_create_blob virtio-gpu: Add initial definitions for blob resources virtio-gpu: Refactor virtio_gpu_create_mapping_iov virtio-gpu: Refactor virtio_gpu_set_scanout virtio-gpu: Add virtio_gpu_find_check_resource stubs: Add stubs for udmabuf helpers virtio-gpu: Add udmabuf helpers headers: Add udmabuf.h ui: Get the fd associated with udmabuf driver hw/display/qxl: Set pci rom address aligned with page size vhost-user-gpu: abstract vg_cleanup_mapping_iov vhost-user-gpu: fix OOB write in 'virgl_cmd_get_capset' (CVE-2021-3546) vhost-user-gpu: fix memory leak in 'virgl_resource_attach_backing' (CVE-2021-3544) vhost-user-gpu: fix memory leak in 'virgl_cmd_resource_unref' (CVE-2021-3544) vhost-user-gpu: fix memory leak while calling 'vg_resource_unref' (CVE-2021-3544) ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-28Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into ↵Peter Maydell4-13/+6
staging # gpg: Signature made Thu 27 May 2021 04:06:17 BST # gpg: using RSA key EF04965B398D6211 # gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>" [marginal] # gpg: WARNING: This key is not certified with sufficiently trusted signatures! # gpg: It is not certain that the signature belongs to the owner. # Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211 * remotes/jasowang/tags/net-pull-request: tap-bsd: Remove special casing for older OpenBSD releases virtio-net: failover: add missing remove_migration_state_change_notifier() hw/net/imx_fec: return 0xffff when accessing non-existing PHY Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-28Merge remote-tracking branch 'remotes/rth-gitlab/tags/pull-tcg-20210526' ↵Peter Maydell53-404/+600
into staging Adjust types for some memory access functions. Reduce inclusion of tcg headers. Fix watchpoints vs replay. Fix tcg/aarch64 roli expansion. Introduce SysemuCPUOps structure. # gpg: Signature made Thu 27 May 2021 00:43:54 BST # gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F # gpg: issuer "richard.henderson@linaro.org" # gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full] # Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F * remotes/rth-gitlab/tags/pull-tcg-20210526: (31 commits) hw/core: Constify TCGCPUOps target/mips: Fold jazz behaviour into mips_cpu_do_transaction_failed cpu: Move CPUClass::get_paging_enabled to SysemuCPUOps cpu: Move CPUClass::get_memory_mapping to SysemuCPUOps cpu: Move CPUClass::get_phys_page_debug to SysemuCPUOps cpu: Move CPUClass::asidx_from_attrs to SysemuCPUOps cpu: Move CPUClass::write_elf* to SysemuCPUOps cpu: Move CPUClass::get_crash_info to SysemuCPUOps cpu: Move CPUClass::virtio_is_big_endian to SysemuCPUOps cpu: Move CPUClass::vmsd to SysemuCPUOps cpu: Introduce SysemuCPUOps structure cpu: Move AVR target vmsd field from CPUClass to DeviceClass cpu: Rename CPUClass vmsd -> legacy_vmsd cpu: Assert DeviceClass::vmsd is NULL on user emulation cpu: Directly use get_memory_mapping() fallback handlers in place cpu: Directly use get_paging_enabled() fallback handlers in place cpu: Directly use cpu_write_elf*() fallback handlers in place cpu: Introduce cpu_virtio_is_big_endian() cpu: Un-inline cpu_get_phys_page_debug and cpu_asidx_from_attrs cpu: Split as cpu-common / cpu-sysemu ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-28hw/usb: hcd-xhci-pci: Fix spec violation of IP flag for MSI/MSI-XRuimei Yan4-7/+15
Per xHCI spec v1.2 chapter 4.17.5 page 296: If MSI or MSI-X interrupts are enabled, Interrupt Pending (IP) shall be cleared automatically when the PCI dword write generated by the interrupt assertion is complete. Currently QEMU does not clear the IP flag in the MSI / MSI-X mode. This causes subsequent spurious interrupt to be delivered to guests. To solve this, we change the xhci intr_raise() hook routine to have a bool return value that is passed to its caller (the xhci core), with true indicating that IP should be self-cleared. Fixes: 62c6ae04cf43 ("xhci: Initial xHCI implementation") Fixes: 4c47f800631a ("xhci: add msix support") Signed-off-by: Ruimei Yan <ruimei.yan@windriver.com> [bmeng: move IP clear codes from xhci pci to xhci core] Signed-off-by: Bin Meng <bin.meng@windriver.com> Message-Id: <20210521024224.2277634-2-bmeng.cn@gmail.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2021-05-28hw/usb: hcd-xhci-pci: Raise MSI/MSI-X interrupts only when told toRuimei Yan1-2/+3
At present MSI / MSI-X interrupts are triggered regardless of the irq level. We should have checked the level to determine whether the interrupt needs to be delivered. The level check logic was present in early versions of the xhci model, but got dropped later by a rework of interrupt handling under commit 4c4abe7cc903 ("xhci: rework interrupt handling"). Fixes: 4c4abe7cc903 ("xhci: rework interrupt handling") Signed-off-by: Ruimei Yan <ruimei.yan@windriver.com> Signed-off-by: Bin Meng <bin.meng@windriver.com> Message-Id: <20210521024224.2277634-1-bmeng.cn@gmail.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2021-05-28hw/input/hid: Add support for keys of jp106 keyboard.Katsuhiro Ueno1-2/+2
Add support for the following keys: KATAKANAHIRAGANA, HENKAN, MUHENKAN, RO, and YEN. Before this commit, these keys did not work as expected when a jp106 keyboard was connected to the guest as a usb-kbd device. Signed-off-by: Katsuhiro Ueno <uenobk@gmail.com> Message-Id: <CA+pCdY3iG+pKKQqEVknnWF-W0wK36S4U1jxPvxmGAPp6FFvz1Q@mail.gmail.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2021-05-27Merge remote-tracking branch 'remotes/vivier/tags/m68k-for-6.1-pull-request' ↵Peter Maydell2-8/+51
into staging m68k pull request 20210526 implement m68k "any instruction" trace mode # gpg: Signature made Wed 26 May 2021 20:56:58 BST # gpg: using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C # gpg: issuer "laurent@vivier.eu" # gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full] # gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full] # gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full] # Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C * remotes/vivier/tags/m68k-for-6.1-pull-request: target/m68k: implement m68k "any instruction" trace mode target/m68k: introduce gen_singlestep_exception() function target/m68k: call gen_raise_exception() directly if single-stepping in gen_jmp_tb() target/m68k: introduce is_singlestepping() function Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-27Merge remote-tracking branch 'remotes/philmd/tags/gitlab-ci-20210527' into ↵Peter Maydell12-1124/+1180
staging GitLab CI patches queue - Explode .gitlab-ci.yml in reusable templates - Add job to cross build/test TCI on i386 host - Remove CentOS 7 linux-user build job - Temporarily set Avocado-based jobs in manual mode - Increase time to hold Avocado reports to 1 week # gpg: Signature made Thu 27 May 2021 15:40:58 BST # gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE # gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full] # Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE * remotes/philmd/tags/gitlab-ci-20210527: gitlab: Convert check-dco/check-patch jobs to the 'rules' syntax gitlab: Use $CI_DEFAULT_BRANCH instead of hardcoded 'master' gitlab: Run Avocado tests manually (except mainstream CI) gitlab: Keep Avocado reports during 1 week gitlab: Extract cross-container jobs to container-cross.yml gitlab: Document how forks can use different set of jobs gitlab: Move current job set to qemu-project.yml gitlab: Extract all default build/test jobs to buildtest.yml gitlab: Drop linux user build job for CentOS 7 gitlab: Extract core container jobs to container-core.yml gitlab: Extract default build/test jobs templates gitlab: Extract build stages to stages.yml gitlab: Extract DCO/style check jobs to static_checks.yml gitlab: Extract crossbuild job templates to crossbuild-template.yml gitlab: Extract container job template to container-template.yml gitlab: Enable cross-i386 builds of TCI gitlab: Rename ACCEL_CONFIGURE_OPTS to EXTRA_CONFIGURE_OPTS gitlab: Replace YAML anchors by extends (acceptance_test_job) Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-27gitlab: Convert check-dco/check-patch jobs to the 'rules' syntaxPhilippe Mathieu-Daudé1-7/+9
Per GitLab documentation [*]: "rules replaces only/except and they can’t be used together in the same job." Since the 'rules' syntax is more powerful and we are already using it, convert the check-dco/check-patch jobs so no job use the 'only/ except' syntax. [*] https://docs.gitlab.com/ee/ci/yaml/#rules Inspired-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210525132418.4133235-3-f4bug@amsat.org>
2021-05-27gitlab: Use $CI_DEFAULT_BRANCH instead of hardcoded 'master'Philippe Mathieu-Daudé1-2/+2
We want to skip the checkpatch and DCO signoff jobs when pushing to the default branch. Currently this branch is called 'master', but we don't need to hardcode this in the CI configuration, because the $CI_DEFAULT_BRANCH env variable exposes it. References: - https://sfconservancy.org/news/2020/jun/23/gitbranchname/ - https://about.gitlab.com/blog/2021/03/10/new-git-default-branch-name/ Suggested-by: Savitoj Singh <savsingh@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210525153826.4174157-1-philmd@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com>
generated by cgit v1.1 at 2024-12-29 14:06:23 +0000