| 082ad5ed | 08-May-2026 |
Jason Gunthorpe <jgg@nvidia.com> |
iommu/riscv: Add NAPOT range invalidation support
Use the RISC-V IOMMU Address Range Invalidation extension
(capabilities.S, spec section 9.3) to invalidate an IOVA range with
a single IOTINVAL.VMA
iommu/riscv: Add NAPOT range invalidation support
Use the RISC-V IOMMU Address Range Invalidation extension
(capabilities.S, spec section 9.3) to invalidate an IOVA range with
a single IOTINVAL.VMA command using NAPOT-encoded addressing.
One iommu_iotlb_gather maps to one NAPOT invalidation command. The
smallest power-of-two aligned range covering the gather is used since
over-invalidation is always safe.
S and NL seem to be orthogonal in the spec, so if NL is not
supported then global invalidation is probably always going to happen
as wiping a large range without a table change is not common.
Reviewed-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| 835d06ee | 08-May-2026 |
Jason Gunthorpe <jgg@nvidia.com> |
iommu/riscv: Include the dword number in RISCV_IOMMU_CMD macros
The command queue entry format is 128 bits. Follow the pattern of the
other drivers and encode the 64 bit dword number in the macro
it
iommu/riscv: Include the dword number in RISCV_IOMMU_CMD macros
The command queue entry format is 128 bits. Follow the pattern of the
other drivers and encode the 64 bit dword number in the macro
itself. RISC-V further has similarly named macros that are not field
layout macros, but field content macros which won't get a new number.
Overall this is clearer to understand the code and check for errors like
using the wrong macro in the wrong spot.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Tested-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| e4084c6b | 08-May-2026 |
Jason Gunthorpe <jgg@nvidia.com> |
iommu/riscv: Add RISCV_IOMMU_CAPABILITIES_NL
Non-leaf invalidation allows the single invalidate command to also
clear the walk cache. If NL is available, set the NL bit if the
gather indicates table
iommu/riscv: Add RISCV_IOMMU_CAPABILITIES_NL
Non-leaf invalidation allows the single invalidate command to also
clear the walk cache. If NL is available, set the NL bit if the
gather indicates tables have been changed. The stride is already
calculated properly.
Reviewed-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| bb62adcf | 08-May-2026 |
Jason Gunthorpe <jgg@nvidia.com> |
iommu/riscv: Compute best stride for single invalidation
Replace the per-page IOTLB invalidation loop with stride-based
invalidation that uses the level bitmaps from iommu_iotlb_gather.
Pre-calcula
iommu/riscv: Compute best stride for single invalidation
Replace the per-page IOTLB invalidation loop with stride-based
invalidation that uses the level bitmaps from iommu_iotlb_gather.
Pre-calculate the invalidation information before running over the
bonds loop as it is the same for every entry.
The lowest set bit in the PT_FEAT_DETAILED_GATHER bitmaps indicates
the stride. This design ignores the SVNAPOT contiguous pages on the
assumption that they still have to be individually invalidated like
ARM requires, though it is not clear from the spec.
Replace the 2M cutoff for global invalidation with a 512 command
limit. This is the same for a 4k stride and now scales with the
stride size.
Reviewed-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| 40a13b49 | 27-Mar-2026 |
Jason Gunthorpe <jgg@nvidia.com> |
iommu/riscv: Remove overflows on the invalidation path
Since RISC-V supports a sign extended page table it should support
a gather->end of ULONG_MAX, but if this happens it will infinite loop
becaus
iommu/riscv: Remove overflows on the invalidation path
Since RISC-V supports a sign extended page table it should support
a gather->end of ULONG_MAX, but if this happens it will infinite loop
because of the overflow.
Also avoid overflow computing the length by moving the +1 to the other
side of the <
Fixes: 488ffbf18171 ("iommu/riscv: Paging domain support")
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| 553a127c | 19-Mar-2026 |
Ethan Tidmore <ethantidmore06@gmail.com> |
iommu/riscv: Fix signedness bug
The function platform_irq_count() returns negative error codes and
iommu->irqs_count is an unsigned integer, so the check
(iommu->irqs_count <= 0) is always impossibl
iommu/riscv: Fix signedness bug
The function platform_irq_count() returns negative error codes and
iommu->irqs_count is an unsigned integer, so the check
(iommu->irqs_count <= 0) is always impossible.
Make the return value of platform_irq_count() be assigned to ret, check
for error, and then assign iommu->irqs_count to ret.
Detected by Smatch:
drivers/iommu/riscv/iommu-platform.c:119 riscv_iommu_platform_probe() warn:
'iommu->irqs_count' unsigned <= 0
Signed-off-by: Ethan Tidmore <ethantidmore06@gmail.com>
Fixes: 5c0ebbd3c6c6 ("iommu/riscv: Add RISC-V IOMMU platform device driver")
Reviewed-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| b2e56845 | 27-Feb-2026 |
Fangyu Yu <fangyu.yu@linux.alibaba.com> |
iommu/riscv: Stop polling when CQCSR reports an error
The cmdq wait loop busy-polls the consumer index until it advances
or the software timeout expires. If the IOMMU has already signaled
a command
iommu/riscv: Stop polling when CQCSR reports an error
The cmdq wait loop busy-polls the consumer index until it advances
or the software timeout expires. If the IOMMU has already signaled
a command queue failure in CQCSR, continuing to poll for progress is
pointless.
Make riscv_iommu_queue_wait() also terminate the poll when any of these
CQCSR error bits are observed.
This helps the caller return earlier in failure cases and avoids
spinning until the full timeout interval when the hardware has already
reported an error. On single-core systems in particular, the current
busy-wait can delay servicing the command-timeout interrupt until the
software timeout expires (90s by default).
Fixes: 856c0cfe5c5f ("iommu/riscv: Command and fault queue support")
Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| 7cd0c655 | 27-Feb-2026 |
Jason Gunthorpe <jgg@nvidia.com> |
iommu/riscv: Allow RISC_VIOMMU to COMPILE_TEST
This driver used to use a lot of page table constants from the architecture
code which prevented COMPILE_TEST on other architectures. Now that iommupt
iommu/riscv: Allow RISC_VIOMMU to COMPILE_TEST
This driver used to use a lot of page table constants from the architecture
code which prevented COMPILE_TEST on other architectures. Now that iommupt
provides all of the constants internally there are only two small bumps
preventing COMPILE_TEST.
- Use the generic functions for the riscv specific phys_to_pfn() and
pfn_to_phys()
- Use CONFIG_MMIOWB to block off the mmiowb() barrier
- Require 64 bit because of writeq use failing compilation on 32 bit
Tested-by: Vincent Chen <vincent.chen@sifive.com>
Acked-by: Paul Walmsley <pjw@kernel.org> # arch/riscv
Reviewed-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Tested-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| c70d20b2 | 27-Feb-2026 |
Jason Gunthorpe <jgg@nvidia.com> |
iommu/riscv: Add missing GENERIC_MSI_IRQ
The commit below added MSI related calls to the driver that depends on
GENERIC_MSI_IRQ. It is possible to build RISC-V without this selected.
This is also n
iommu/riscv: Add missing GENERIC_MSI_IRQ
The commit below added MSI related calls to the driver that depends on
GENERIC_MSI_IRQ. It is possible to build RISC-V without this selected.
This is also necessary to make the driver COMPILE_TEST.
Fixes: d5f88acdd6ff ("iommu/riscv: Add support for platform msi")
Tested-by: Vincent Chen <vincent.chen@sifive.com>
Tested-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| 69541898 | 27-Feb-2026 |
Jason Gunthorpe <jgg@nvidia.com> |
iommu/riscv: Enable SVNAPOT support for contiguous ptes
This turns on a 64k page size. The "RISC-V IOMMU Architecture
Specification" states:
6.4 IOMMU capabilities
[..]
IOMMU implementations
iommu/riscv: Enable SVNAPOT support for contiguous ptes
This turns on a 64k page size. The "RISC-V IOMMU Architecture
Specification" states:
6.4 IOMMU capabilities
[..]
IOMMU implementations must support the Svnapot standard extension for
NAPOT Translation Contiguity.
So just switch it on unconditionally.
Cc: Xu Lu <luxu.kernel@bytedance.com>
Tested-by: Vincent Chen <vincent.chen@sifive.com>
Acked-by: Paul Walmsley <pjw@kernel.org> # arch/riscv
Reviewed-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Tested-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| e5ef3219 | 27-Feb-2026 |
Jason Gunthorpe <jgg@nvidia.com> |
iommu/riscv: Use the generic iommu page table
This is a fairly straightforward conversion of the RISC-V iommu driver to
use the generic iommu page table code.
Invalidation stays as it is now with t
iommu/riscv: Use the generic iommu page table
This is a fairly straightforward conversion of the RISC-V iommu driver to
use the generic iommu page table code.
Invalidation stays as it is now with the driver pretending to implement
simple range based invalidation even though the HW is more like ARM SMMUv3
than AMD where the HW implements a single-PTE based invalidation. Future
work to extend the generic invalidate mechanism to support more ARM-like
semantics would benefit this driver as well.
Delete the existing page table code.
Tested-by: Vincent Chen <vincent.chen@sifive.com>
Acked-by: Paul Walmsley <pjw@kernel.org> # arch/riscv
Reviewed-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Tested-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|
| 7217cee3 | 30-Jan-2026 |
Yaxing Guo <guoyaxing@bosc.ac.cn> |
iommu/riscv: Skip IRQ count check when using MSI interrupts
In RISC-V IOMMU platform devices that use MSI interrupts (indicated by the
presence of 'msi-parent' in the device tree), there are no wire
iommu/riscv: Skip IRQ count check when using MSI interrupts
In RISC-V IOMMU platform devices that use MSI interrupts (indicated by the
presence of 'msi-parent' in the device tree), there are no wired interrupt
lines, so calling platform_get_irq_count() returns 0 or -ENXIO, causing the
driver to fail during probe.
However, MSI interrupts are allocated dynamically via the MSI subsystem and
do not appear in the device tree 'interrupts' property. Therefore, the
driver should not require a non-zero IRQ count when 'msi-parent' is present.
This patch fixes the bug where probe fails when using MSI interrupts
(which do not have an 'interrupts' property in the device tree)..
Fixes: <d5f88acdd6ff> ("iommu/riscv: Add support for platform msi")
Signed-off-by: Yaxing Guo <guoyaxing@bosc.ac.cn>
Reviewed-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
show more ...
|