- Jan 16, 2020
-
-
Will Deacon authored
Although we WARN in arm_smmu_add_device() if the device being added has been added already without a subsequent call to arm_smmu_remove_device(), we still continue half-heartedly, initialising the stream-table for any new StreamIDs that may have magically appeared and re-establishing device links that should still be there from last time. Given that calling ->add_device() twice without removing the device in the meantime is indicative of an error in the caller, just return -EBUSY after warning. Cc: Robin Murphy <robin.murphy@arm.com> Cc: Jean Philippe-Brucker <jean-philippe@linaro.org> Signed-off-by:Will Deacon <will@kernel.org>
-
Jean-Philippe Brucker authored
Let add_device() clean up after itself. The iommu_bus_init() function does call remove_device() on error, but other sites (e.g. of_iommu) do not. Don't free level-2 stream tables because we'd have to track if we allocated each of them or if they are used by other endpoints. It's not worth the hassle since they are managed resources. Reviewed-by:
Eric Auger <eric.auger@redhat.com> Reviewed-by:
Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by:
Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by:
-
Will Deacon authored
If, for some bizarre reason, the compiler decided to split up the write of STE DWORD 0, we could end up making a partial structure valid. Although this probably won't happen, follow the example of the context-descriptor code and use WRITE_ONCE() to ensure atomicity of the write. Reported-by:
-
Jean-Philippe Brucker authored
The SMMU can support up to 20 bits of SSID. Add a second level of page tables to accommodate this. Devices that support more than 1024 SSIDs now have a table of 1024 L1 entries (8kB), pointing to tables of 1024 context descriptors (64kB), allocated on demand. Signed-off-by:
-
Jean-Philippe Brucker authored
Second-level context descriptor tables will be allocated lazily in arm_smmu_write_ctx_desc(). Help with handling allocation failure by moving the CD write into arm_smmu_domain_finalise_s1(). Reviewed-by:
-
Jean-Philippe Brucker authored
Now that we support substream IDs, initialize s1cdmax with the number of SSID bits supported by a master and the SMMU. Context descriptor tables are allocated once for the first master attached to a domain. Therefore attaching multiple devices with different SSID sizes is tricky, and we currently don't support it. As a future improvement it would be nice to at least support attaching a SSID-capable device to a domain that isn't using SSID, by reallocating the SSID table. This would allow supporting a SSID-capable device that is in the same IOMMU group as a bridge, for example. Varying SSID size is less of a concern, since the PCIe specification "highly recommends" that devices supporting PASID implement all 20 bits of it. Tested-by:
Zhangfei Gao <zhangfei.gao@linaro.org> Reviewed-by:
-
Jean-Philippe Brucker authored
At the moment, the SMMUv3 driver implements only one stage-1 or stage-2 page directory per device. However SMMUv3 allows more than one address space for some devices, by providing multiple stage-1 page directories. In addition to the Stream ID (SID), that identifies a device, we can now have Substream IDs (SSID) identifying an address space. In PCIe, SID is called Requester ID (RID) and SSID is called Process Address-Space ID (PASID). A complete stage-1 walk goes through the context descriptor table: Stream tables Ctx. Desc. tables Page tables +--------+ ,------->+-------+ ,------->+-------+ : : | : : | : : +--------+ | +-------+ | +-------+ SID->| STE |---' SSID->| CD |---' IOVA->| PTE |--> IPA +--------+ +-------+ +-------+ : : : : : : +--------+ +-------+ +-------+ Rewrite arm_smmu_write_ctx_desc() to modify context descriptor table entries. To keep things simple we only implement one level of context descriptor tables here, but as with stream and page tables, an SSID can be split to index multiple levels of tables. Signed-off-by: -
Jean-Philippe Brucker authored
Support for SSID will require allocating context descriptor tables. Move the context descriptor allocation to separate functions. Signed-off-by:
-
Jean-Philippe Brucker authored
When adding SSID support to the SMMUv3 driver, we'll need to manipulate leaf pasid tables and context descriptors. Extract the context descriptor structure and align with the way stream tables are handled. Signed-off-by:
-
Jean-Philippe Brucker authored
Named component nodes in the IORT tables describe the number of Substream ID bits (aka. PASID) supported by the device. Propagate this value to the fwspec structure in order to enable PASID for platform devices. Acked-by:
Hanjun Guo <guohanjun@huawei.com> Reviewed-by:
-
Jean-Philippe Brucker authored
For platform devices that support SubstreamID (SSID), firmware provides the number of supported SSID bits. Restrict it to what the SMMU supports and cache it into master->ssid_bits, which will also be used for PCI PASID. Reviewed-by:
-
- Jan 15, 2020
-
-
Jean-Philippe Brucker authored
On Arm systems, some platform devices behind an SMMU may support the PASID feature, which offers multiple address space. Let the firmware tell us when a device supports PASID. Reviewed-by:
Rob Herring <robh@kernel.org> Signed-off-by:
-
Jean-Philippe Brucker authored
Since commit 518a2f19 ("dma-mapping: zero memory returned from dma_alloc_*"), dma_alloc_* always initializes memory to zero, so there is no need to use dma_zalloc_* or pass the __GFP_ZERO flag anymore. The flag was introduced by commit 04fa26c7 ("iommu/arm-smmu: Convert DMA buffer allocations to the managed API"), since the managed API didn't provide a dmam_zalloc_coherent() function. Reviewed-by:
-
- Jan 11, 2020
-
-
Robin Murphy authored
Make the SMR mask test more robust against SMR0 being live at probe time, which might happen once we start supporting firmware reservations for framebuffers and suchlike. Signed-off-by:
Robin Murphy <robin.murphy@arm.com> Signed-off-by:
-
- Jan 10, 2020
-
-
Robin Murphy authored
Now that we can correctly extract top-level indices without relying on the remaining upper bits being zero, the only remaining impediments to using a given table for TTBR1 are the address validation on map/unmap and the awkward TCR translation granule format. Add a quirk so that we can do the right thing at those points. Tested-by:
Jordan Crouse <jcrouse@codeaurora.org> Signed-off-by:
-
Will Deacon authored
Commit 05a648cd2dd7 ("iommu/io-pgtable-arm: Rationalise TCR handling") reworked the way in which the TCR register value is returned from the io-pgtable code when targetting the Arm long-descriptor format, in preparation for allowing page-tables to target TTBR1. As it turns out, the new interface is a lot nicer to use, so do the same conversion for the VTCR register even though there is only a single base register for stage-2 translation. Cc: Robin Murphy <robin.murphy@arm.com> Signed-off-by: -
Will Deacon authored
Now that we have arm-smmu.h defining various SMMU constants, ensure that they are namespaced with the ARM_SMMU_ prefix in order to avoid conflicts with the CPU, such as the one we're currently bodging around with the TCR. Cc: Robin Murphy <robin.murphy@arm.com> Signed-off-by: -
Robin Murphy authored
Although it's conceptually nice for the io_pgtable_cfg to provide a standard VMSA TCR value, the reality is that no VMSA-compliant IOMMU looks exactly like an Arm CPU, and they all have various other TCR controls which io-pgtable can't be expected to understand. Thus since there is an expectation that drivers will have to add to the given TCR value anyway, let's strip it down to just the essentials that are directly relevant to io-pgtable's inner workings - namely the various sizes and the walk attributes. Tested-by:
-
Will Deacon authored
ARM_64_LPAE_S2_TCR_RES1 is intended to map to bit 31 of the VTCR register, which is required to be set to 1 by the architecture. Unfortunately, we accidentally treat this as a signed quantity which means we also set the upper 32 bits of the VTCR to one, and they are required to be zero. Treat ARM_64_LPAE_S2_TCR_RES1 as unsigned to avoid the unwanted sign-extension up to 64 bits. Cc: Robin Murphy <robin.murphy@arm.com> Signed-off-by: -
Robin Murphy authored
By VMSA rules, using Normal Non-Cacheable type with a shareability attribute of anything other than Outer Shareable is liable to lead into unpredictable territory: | Overlaying the shareability attribute (B3-1377, ARM DDI 0406C.c) | | A memory region with a resultant memory type attribute of Normal, and | a resultant cacheability attribute of Inner Non-cacheable, Outer | Non-cacheable, must have a resultant shareability attribute of Outer | Shareable, otherwise shareability is UNPREDICTABLE Although the SMMU architectures seem to give some slightly stronger guarantees of Non-Cacheable output types becoming implicitly Outer Shareable in most cases, we may as well be explicit and not take any chances. It's also weird that LPAE attribute handling is currently split between prot_to_pte() and init_pte() given that it can all be statically determined up-front. Thus, collect *all* the LPAE attributes into prot_to_pte() in order to logically pick the shareability based on the incoming IOMMU API prot value, and tweak the short-descriptor code to stop setting TTBR0.NOS for Non-Cacheable walks. Signed-off-by:
-
Will Deacon authored
Commit 9e6ea59f ("iommu/io-pgtable: Support non-coherent page tables") added support for non-coherent page-table walks to the Arm IOMMU page-table backends. Unfortunately, it left the stage-2 allocator unchanged, so let's hook that up in the same way. Cc: Bjorn Andersson <bjorn.andersson@linaro.org> Cc: Robin Murphy <robin.murphy@arm.com> Signed-off-by:
-
Robin Murphy authored
TTBR1 values have so far been redundant since no users implement any support for split address spaces. Crucially, though, one of the main reasons for wanting to do so is to be able to manage each half entirely independently, e.g. context-switching one set of mappings without disturbing the other. Thus it seems unlikely that tying two tables together in a single io_pgtable_cfg would ever be particularly desirable or useful. Streamline the configs to just a single conceptual TTBR value representing the allocated table. This paves the way for future users to support split address spaces by simply allocating a table and dealing with the detailed TTBRn logistics themselves. Tested-by:
-
Masahiro Yamada authored
For ARCH=arm builds, OF is not necessarily enabled, that is, you can build this driver without CONFIG_OF. When CONFIG_OF is unset, of_match_ptr() is NULL, and arm_smmu_of_match is left orphan. Building it with W=1 emits a warning: drivers/iommu/arm-smmu.c:1904:34: warning: ‘arm_smmu_of_match’ defined but not used [-Wunused-const-variable=] static const struct of_device_id arm_smmu_of_match[] = { ^~~~~~~~~~~~~~~~~ There are two ways to fix this: - annotate arm_smmu_of_match with __maybe_unused (or surround the code with #ifdef CONFIG_OF ... #endif) - stop using of_match_ptr() This commit took the latter solution. It slightly increases the object size, but it is probably not a big deal because arm_smmu_device_dt_probe() is also compiled irrespective of CONFIG_OF. Signed-off-by:Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by:
-
Masahiro Yamada authored
The CONFIG option controlling this driver, ARM_SMMU_V3, depends on ARM64, which select's OF. So, CONFIG_OF is always defined when building this driver. of_match_ptr(arm_smmu_of_match) is the same as arm_smmu_of_match. Signed-off-by:
-
Masahiro Yamada authored
This is an off-by-one mistake. resource_size() returns res->end - res->start + 1. Reviewed-by:
-
Shameer Kolothum authored
CMDQ_OP_TLBI_NH_VA requires VMID and this was missing since commit 1c27df1c ("iommu/arm-smmu: Use correct address mask for CMD_TLBI_S2_IPA"). Add it back. Fixes: 1c27df1c ("iommu/arm-smmu: Use correct address mask for CMD_TLBI_S2_IPA") Signed-off-by:
Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Signed-off-by:
-
Will Deacon authored
Requiring each IOMMU driver to initialise the 'owner' field of their 'struct iommu_ops' is error-prone and easily forgotten. Follow the example set by PCI and USB by assigning THIS_MODULE automatically when registering the ops structure with IOMMU core. Reviewed-by:
Greg Kroah-Hartman <gregkh@linuxfoundation.org> Suggested-by:
-
- Dec 23, 2019
-
-
Will Deacon authored
I no longer work for Arm, so update the stale reference to my old email address. Signed-off-by:
Joerg Roedel <jroedel@suse.de>
-
Will Deacon authored
By conditionally dropping support for the legacy binding and exporting the newly introduced 'arm_smmu_impl_init()' function we can allow the ARM SMMU driver to be built as a module. Signed-off-by:
-
Will Deacon authored
When removing the SMMU driver, we need to clear any state that we registered during probe. This includes our bus ops, sysfs entries and the IOMMU device registered for early firmware probing of masters. Signed-off-by:
-
Will Deacon authored
By removing the redundant call to 'pci_request_acs()' we can allow the ARM SMMUv3 driver to be built as a module. Signed-off-by:
-
Ard Biesheuvel authored
Add support for SMMU drivers built as modules to the ACPI/IORT device probing path, by deferring the probe of the master if the SMMU driver is known to exist but has not been loaded yet. Given that the IORT code registers a platform device for each SMMU that it discovers, we can easily trigger the udev based autoloading of the SMMU drivers by making the platform device identifier part of the module alias. Reviewed-by:
Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Tested-by: John Garry <john.garry@huawei.com> # only manual smmu ko loading Signed-off-by:
Ard Biesheuvel <ardb@kernel.org> Signed-off-by:
-
Will Deacon authored
When removing the SMMUv3 driver, we need to clear any state that we registered during probe. This includes our bus ops, sysfs entries and the IOMMU device registered for early firmware probing of masters. Signed-off-by:
-
Will Deacon authored
Forcefully unbinding the Arm SMMU drivers is a pretty dangerous operation, since it will likely lead to catastrophic failure for any DMA devices mastering through the SMMU being unbound. When the driver then attempts to "handle" the fatal faults, it's very easy to trip over dead data structures, leading to use-after-free. On John's machine, he reports that the machine was "unusable" due to loss of the storage controller following a forced unbind of the SMMUv3 driver: | # cd ./bus/platform/drivers/arm-smmu-v3 | # echo arm-smmu-v3.0.auto > unbind | hisi_sas_v2_hw HISI0162:01: CQE_AXI_W_ERR (0x800) found! | platform arm-smmu-v3.0.auto: CMD_SYNC timeout at 0x00000146 | [hwprod 0x00000146, hwcons 0x00000000] Prevent this forced unbinding of the drivers by setting "suppress_bind_attrs" to true. Link: https://lore.kernel.org/lkml/06dfd385-1af0-3106-4cc5-6a5b8e864759@huawei.com Reported-by:
John Garry <john.garry@huawei.com> Signed-off-by:
-
Will Deacon authored
This reverts commit addb672f . Let's get the SMMU driver building as a module, which means putting back some dead code that we used to carry. Signed-off-by:
-
Will Deacon authored
This reverts commit c07b6426 . Let's get the SMMUv3 driver building as a module, which means putting back some dead code that we used to carry. Signed-off-by:
-
Will Deacon authored
'bus_set_iommu()' allows IOMMU drivers to register their ops for a given bus type. Unfortunately, it then doesn't allow them to be removed, which is necessary for modular drivers to shutdown cleanly so that they can be reloaded later on. Allow 'bus_set_iommu()' to take a NULL 'ops' argument, which clear the ops pointer for the selected bus_type. Signed-off-by:
-
Will Deacon authored
Ensure that we hold a reference to the IOMMU driver module while calling the '->of_xlate()' callback during early device probing. Signed-off-by:
-
Will Deacon authored
To avoid accidental removal of an active IOMMU driver module, take a reference to the driver module in 'iommu_probe_device()' immediately prior to invoking the '->add_device()' callback and hold it until the after the device has been removed by '->remove_device()'. Suggested-by:
Joerg Roedel <joro@8bytes.org> Signed-off-by:
-
Will Deacon authored
Building drivers for ATS-aware IOMMUs as modules requires access to pci_ats_disabled(). Export it as a GPL symbol to get things working. Acked-by:
Bjorn Helgaas <bhelgaas@google.com> Signed-off-by:
-
