diff options
Diffstat (limited to 'docs/system')
| -rw-r--r-- | docs/system/arm/cpu-features.rst | 346 | ||||
| -rw-r--r-- | docs/system/target-arm.rst | 6 |
2 files changed, 352 insertions, 0 deletions
diff --git a/docs/system/arm/cpu-features.rst b/docs/system/arm/cpu-features.rst new file mode 100644 index 0000000..7495b7b --- /dev/null +++ b/docs/system/arm/cpu-features.rst @@ -0,0 +1,346 @@ +Arm CPU Features +================ + +CPU features are optional features that a CPU of supporting type may +choose to implement or not. In QEMU, optional CPU features have +corresponding boolean CPU proprieties that, when enabled, indicate +that the feature is implemented, and, conversely, when disabled, +indicate that it is not implemented. An example of an ARM CPU feature +is the Performance Monitoring Unit (PMU). CPU types such as the +Cortex-A15 and the Cortex-A57, which respectively implement ARM +architecture reference manuals ARMv7-A and ARMv8-A, may both optionally +implement PMUs. For example, if a user wants to use a Cortex-A15 without +a PMU, then the `-cpu` parameter should contain `pmu=off` on the QEMU +command line, i.e. `-cpu cortex-a15,pmu=off`. + +As not all CPU types support all optional CPU features, then whether or +not a CPU property exists depends on the CPU type. For example, CPUs +that implement the ARMv8-A architecture reference manual may optionally +support the AArch32 CPU feature, which may be enabled by disabling the +`aarch64` CPU property. A CPU type such as the Cortex-A15, which does +not implement ARMv8-A, will not have the `aarch64` CPU property. + +QEMU's support may be limited for some CPU features, only partially +supporting the feature or only supporting the feature under certain +configurations. For example, the `aarch64` CPU feature, which, when +disabled, enables the optional AArch32 CPU feature, is only supported +when using the KVM accelerator and when running on a host CPU type that +supports the feature. While `aarch64` currently only works with KVM, +it could work with TCG. CPU features that are specific to KVM are +prefixed with "kvm-" and are described in "KVM VCPU Features". + +CPU Feature Probing +=================== + +Determining which CPU features are available and functional for a given +CPU type is possible with the `query-cpu-model-expansion` QMP command. +Below are some examples where `scripts/qmp/qmp-shell` (see the top comment +block in the script for usage) is used to issue the QMP commands. + +1. Determine which CPU features are available for the `max` CPU type + (Note, we started QEMU with qemu-system-aarch64, so `max` is + implementing the ARMv8-A reference manual in this case):: + + (QEMU) query-cpu-model-expansion type=full model={"name":"max"} + { "return": { + "model": { "name": "max", "props": { + "sve1664": true, "pmu": true, "sve1792": true, "sve1920": true, + "sve128": true, "aarch64": true, "sve1024": true, "sve": true, + "sve640": true, "sve768": true, "sve1408": true, "sve256": true, + "sve1152": true, "sve512": true, "sve384": true, "sve1536": true, + "sve896": true, "sve1280": true, "sve2048": true + }}}} + +We see that the `max` CPU type has the `pmu`, `aarch64`, `sve`, and many +`sve<N>` CPU features. We also see that all the CPU features are +enabled, as they are all `true`. (The `sve<N>` CPU features are all +optional SVE vector lengths (see "SVE CPU Properties"). While with TCG +all SVE vector lengths can be supported, when KVM is in use it's more +likely that only a few lengths will be supported, if SVE is supported at +all.) + +(2) Let's try to disable the PMU:: + + (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"pmu":false}} + { "return": { + "model": { "name": "max", "props": { + "sve1664": true, "pmu": false, "sve1792": true, "sve1920": true, + "sve128": true, "aarch64": true, "sve1024": true, "sve": true, + "sve640": true, "sve768": true, "sve1408": true, "sve256": true, + "sve1152": true, "sve512": true, "sve384": true, "sve1536": true, + "sve896": true, "sve1280": true, "sve2048": true + }}}} + +We see it worked, as `pmu` is now `false`. + +(3) Let's try to disable `aarch64`, which enables the AArch32 CPU feature:: + + (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"aarch64":false}} + {"error": { + "class": "GenericError", "desc": + "'aarch64' feature cannot be disabled unless KVM is enabled and 32-bit EL1 is supported" + }} + +It looks like this feature is limited to a configuration we do not +currently have. + +(4) Let's disable `sve` and see what happens to all the optional SVE + vector lengths:: + + (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"sve":false}} + { "return": { + "model": { "name": "max", "props": { + "sve1664": false, "pmu": true, "sve1792": false, "sve1920": false, + "sve128": false, "aarch64": true, "sve1024": false, "sve": false, + "sve640": false, "sve768": false, "sve1408": false, "sve256": false, + "sve1152": false, "sve512": false, "sve384": false, "sve1536": false, + "sve896": false, "sve1280": false, "sve2048": false + }}}} + +As expected they are now all `false`. + +(5) Let's try probing CPU features for the Cortex-A15 CPU type:: + + (QEMU) query-cpu-model-expansion type=full model={"name":"cortex-a15"} + {"return": {"model": {"name": "cortex-a15", "props": {"pmu": true}}}} + +Only the `pmu` CPU feature is available. + +A note about CPU feature dependencies +------------------------------------- + +It's possible for features to have dependencies on other features. I.e. +it may be possible to change one feature at a time without error, but +when attempting to change all features at once an error could occur +depending on the order they are processed. It's also possible changing +all at once doesn't generate an error, because a feature's dependencies +are satisfied with other features, but the same feature cannot be changed +independently without error. For these reasons callers should always +attempt to make their desired changes all at once in order to ensure the +collection is valid. + +A note about CPU models and KVM +------------------------------- + +Named CPU models generally do not work with KVM. There are a few cases +that do work, e.g. using the named CPU model `cortex-a57` with KVM on a +seattle host, but mostly if KVM is enabled the `host` CPU type must be +used. This means the guest is provided all the same CPU features as the +host CPU type has. And, for this reason, the `host` CPU type should +enable all CPU features that the host has by default. Indeed it's even +a bit strange to allow disabling CPU features that the host has when using +the `host` CPU type, but in the absence of CPU models it's the best we can +do if we want to launch guests without all the host's CPU features enabled. + +Enabling KVM also affects the `query-cpu-model-expansion` QMP command. The +affect is not only limited to specific features, as pointed out in example +(3) of "CPU Feature Probing", but also to which CPU types may be expanded. +When KVM is enabled, only the `max`, `host`, and current CPU type may be +expanded. This restriction is necessary as it's not possible to know all +CPU types that may work with KVM, but it does impose a small risk of users +experiencing unexpected errors. For example on a seattle, as mentioned +above, the `cortex-a57` CPU type is also valid when KVM is enabled. +Therefore a user could use the `host` CPU type for the current type, but +then attempt to query `cortex-a57`, however that query will fail with our +restrictions. This shouldn't be an issue though as management layers and +users have been preferring the `host` CPU type for use with KVM for quite +some time. Additionally, if the KVM-enabled QEMU instance running on a +seattle host is using the `cortex-a57` CPU type, then querying `cortex-a57` +will work. + +Using CPU Features +================== + +After determining which CPU features are available and supported for a +given CPU type, then they may be selectively enabled or disabled on the +QEMU command line with that CPU type:: + + $ qemu-system-aarch64 -M virt -cpu max,pmu=off,sve=on,sve128=on,sve256=on + +The example above disables the PMU and enables the first two SVE vector +lengths for the `max` CPU type. Note, the `sve=on` isn't actually +necessary, because, as we observed above with our probe of the `max` CPU +type, `sve` is already on by default. Also, based on our probe of +defaults, it would seem we need to disable many SVE vector lengths, rather +than only enabling the two we want. This isn't the case, because, as +disabling many SVE vector lengths would be quite verbose, the `sve<N>` CPU +properties have special semantics (see "SVE CPU Property Parsing +Semantics"). + +KVM VCPU Features +================= + +KVM VCPU features are CPU features that are specific to KVM, such as +paravirt features or features that enable CPU virtualization extensions. +The features' CPU properties are only available when KVM is enabled and +are named with the prefix "kvm-". KVM VCPU features may be probed, +enabled, and disabled in the same way as other CPU features. Below is +the list of KVM VCPU features and their descriptions. + + kvm-no-adjvtime By default kvm-no-adjvtime is disabled. This + means that by default the virtual time + adjustment is enabled (vtime is not *not* + adjusted). + + When virtual time adjustment is enabled each + time the VM transitions back to running state + the VCPU's virtual counter is updated to ensure + stopped time is not counted. This avoids time + jumps surprising guest OSes and applications, + as long as they use the virtual counter for + timekeeping. However it has the side effect of + the virtual and physical counters diverging. + All timekeeping based on the virtual counter + will appear to lag behind any timekeeping that + does not subtract VM stopped time. The guest + may resynchronize its virtual counter with + other time sources as needed. + + Enable kvm-no-adjvtime to disable virtual time + adjustment, also restoring the legacy (pre-5.0) + behavior. + +SVE CPU Properties +================== + +There are two types of SVE CPU properties: `sve` and `sve<N>`. The first +is used to enable or disable the entire SVE feature, just as the `pmu` +CPU property completely enables or disables the PMU. The second type +is used to enable or disable specific vector lengths, where `N` is the +number of bits of the length. The `sve<N>` CPU properties have special +dependencies and constraints, see "SVE CPU Property Dependencies and +Constraints" below. Additionally, as we want all supported vector lengths +to be enabled by default, then, in order to avoid overly verbose command +lines (command lines full of `sve<N>=off`, for all `N` not wanted), we +provide the parsing semantics listed in "SVE CPU Property Parsing +Semantics". + +SVE CPU Property Dependencies and Constraints +--------------------------------------------- + + 1) At least one vector length must be enabled when `sve` is enabled. + + 2) If a vector length `N` is enabled, then, when KVM is enabled, all + smaller, host supported vector lengths must also be enabled. If + KVM is not enabled, then only all the smaller, power-of-two vector + lengths must be enabled. E.g. with KVM if the host supports all + vector lengths up to 512-bits (128, 256, 384, 512), then if `sve512` + is enabled, the 128-bit vector length, 256-bit vector length, and + 384-bit vector length must also be enabled. Without KVM, the 384-bit + vector length would not be required. + + 3) If KVM is enabled then only vector lengths that the host CPU type + support may be enabled. If SVE is not supported by the host, then + no `sve*` properties may be enabled. + +SVE CPU Property Parsing Semantics +---------------------------------- + + 1) If SVE is disabled (`sve=off`), then which SVE vector lengths + are enabled or disabled is irrelevant to the guest, as the entire + SVE feature is disabled and that disables all vector lengths for + the guest. However QEMU will still track any `sve<N>` CPU + properties provided by the user. If later an `sve=on` is provided, + then the guest will get only the enabled lengths. If no `sve=on` + is provided and there are explicitly enabled vector lengths, then + an error is generated. + + 2) If SVE is enabled (`sve=on`), but no `sve<N>` CPU properties are + provided, then all supported vector lengths are enabled, which when + KVM is not in use means including the non-power-of-two lengths, and, + when KVM is in use, it means all vector lengths supported by the host + processor. + + 3) If SVE is enabled, then an error is generated when attempting to + disable the last enabled vector length (see constraint (1) of "SVE + CPU Property Dependencies and Constraints"). + + 4) If one or more vector lengths have been explicitly enabled and at + at least one of the dependency lengths of the maximum enabled length + has been explicitly disabled, then an error is generated (see + constraint (2) of "SVE CPU Property Dependencies and Constraints"). + + 5) When KVM is enabled, if the host does not support SVE, then an error + is generated when attempting to enable any `sve*` properties (see + constraint (3) of "SVE CPU Property Dependencies and Constraints"). + + 6) When KVM is enabled, if the host does support SVE, then an error is + generated when attempting to enable any vector lengths not supported + by the host (see constraint (3) of "SVE CPU Property Dependencies and + Constraints"). + + 7) If one or more `sve<N>` CPU properties are set `off`, but no `sve<N>`, + CPU properties are set `on`, then the specified vector lengths are + disabled but the default for any unspecified lengths remains enabled. + When KVM is not enabled, disabling a power-of-two vector length also + disables all vector lengths larger than the power-of-two length. + When KVM is enabled, then disabling any supported vector length also + disables all larger vector lengths (see constraint (2) of "SVE CPU + Property Dependencies and Constraints"). + + 8) If one or more `sve<N>` CPU properties are set to `on`, then they + are enabled and all unspecified lengths default to disabled, except + for the required lengths per constraint (2) of "SVE CPU Property + Dependencies and Constraints", which will even be auto-enabled if + they were not explicitly enabled. + + 9) If SVE was disabled (`sve=off`), allowing all vector lengths to be + explicitly disabled (i.e. avoiding the error specified in (3) of + "SVE CPU Property Parsing Semantics"), then if later an `sve=on` is + provided an error will be generated. To avoid this error, one must + enable at least one vector length prior to enabling SVE. + +SVE CPU Property Examples +------------------------- + + 1) Disable SVE:: + + $ qemu-system-aarch64 -M virt -cpu max,sve=off + + 2) Implicitly enable all vector lengths for the `max` CPU type:: + + $ qemu-system-aarch64 -M virt -cpu max + + 3) When KVM is enabled, implicitly enable all host CPU supported vector + lengths with the `host` CPU type:: + + $ qemu-system-aarch64 -M virt,accel=kvm -cpu host + + 4) Only enable the 128-bit vector length:: + + $ qemu-system-aarch64 -M virt -cpu max,sve128=on + + 5) Disable the 512-bit vector length and all larger vector lengths, + since 512 is a power-of-two. This results in all the smaller, + uninitialized lengths (128, 256, and 384) defaulting to enabled:: + + $ qemu-system-aarch64 -M virt -cpu max,sve512=off + + 6) Enable the 128-bit, 256-bit, and 512-bit vector lengths:: + + $ qemu-system-aarch64 -M virt -cpu max,sve128=on,sve256=on,sve512=on + + 7) The same as (6), but since the 128-bit and 256-bit vector + lengths are required for the 512-bit vector length to be enabled, + then allow them to be auto-enabled:: + + $ qemu-system-aarch64 -M virt -cpu max,sve512=on + + 8) Do the same as (7), but by first disabling SVE and then re-enabling it:: + + $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve512=on,sve=on + + 9) Force errors regarding the last vector length:: + + $ qemu-system-aarch64 -M virt -cpu max,sve128=off + $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve128=off,sve=on + +SVE CPU Property Recommendations +-------------------------------- + +The examples in "SVE CPU Property Examples" exhibit many ways to select +vector lengths which developers may find useful in order to avoid overly +verbose command lines. However, the recommended way to select vector +lengths is to explicitly enable each desired length. Therefore only +example's (1), (4), and (6) exhibit recommended uses of the properties. + diff --git a/docs/system/target-arm.rst b/docs/system/target-arm.rst index 86ea6f2..1425bd5 100644 --- a/docs/system/target-arm.rst +++ b/docs/system/target-arm.rst @@ -78,3 +78,9 @@ undocumented; you can get a complete list by running arm/stellaris arm/musicpal arm/sx1 + +Arm CPU features +================ + +.. toctree:: + arm/cpu-features |