iommu: move context link and ref count into device-independent parts

This also allows to move some bits of ddb print routines into
iommu_utils.c common for x86 iommu drivers.

Sponsored by: Advanced

iommu: move context link and ref count into device-independent parts

This also allows to move some bits of ddb print routines into
iommu_utils.c common for x86 iommu drivers.

Sponsored by: Advanced Micro Devices (AMD)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

show more ...

iommu.h: improve header self-sufficiency

The header embeds struct task into defined structures. Also it needs
the PCI_BUSMAX constant.

Sponsored by: Advanced Micro Devices (AMD)
Sponsored by: The

iommu.h: improve header self-sufficiency

The header embeds struct task into defined structures. Also it needs
the PCI_BUSMAX constant.

Sponsored by: Advanced Micro Devices (AMD)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

show more ...

iommu: extend iommu_map_entry to store the list of associated freed page table pages

The pages are inserted into the added slist if the entry parameter is
passed to iommu_pgfree(). For now it is no

iommu: extend iommu_map_entry to store the list of associated freed page table pages

The pages are inserted into the added slist if the entry parameter is
passed to iommu_pgfree(). For now it is nop.

Sponsored by: Advanced Micro Devices (AMD)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

show more ...

iommu: change iommu_domain_map_ops to take iommu_map_entry

instead of base/size.

Sponsored by: Advanced Micro Devices (AMD)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

iommu: add per-unit sysctls reporting the state of DMA and interrupt remapping

Sponsored by: Advanced Micro Devices (AMD)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

busdma_iommu: indirect dmar-specific method calls in iommu_get_dev_ctx()

Sponsored by: Advanced Micro Devices (AMD)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

iommu_gas: start space search from 1st free space

Maintain a pointer to an element in the domain map that is left of any
sufficiently large free gap in the tree and start the search for free
space t

iommu_gas: start space search from 1st free space

Maintain a pointer to an element in the domain map that is left of any
sufficiently large free gap in the tree and start the search for free
space there, rather than at the root of the tree. On find_space, move
that pointer to the leftmost leaf in the subtree of nodes with
free_down greater than or equal to the minimum allocation size before
starting the search for space from that pointer. On removal of a node
with address less than that pointer, update that pointer to point to
the predecessor or successor of the removed node.

In experiments with netperf streaming, this reduces by about 40% the
number of map entries examined in first-fit allocation.

Reviewed by: alc, kib
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D36624

show more ...

iommu_gas: add iommu_gas_remove()

The function removes the range of addresses from GAS. Right now it is
unused.

The initial version of the patch was extracted from the stalled work for
integration

iommu_gas: add iommu_gas_remove()

The function removes the range of addresses from GAS. Right now it is
unused.

The initial version of the patch was extracted from the stalled work for
integration of DMAR and bhyve, see D25672. Having the function in the
main tree would allow it to co-evolve with other active changes to the
IOMMU driver.

Requested by: alc
Reviewed by: alc, dougm
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D36001

show more ...

iommu_gas: Eliminate redundant parameters and push down lock acquisition

Since IOMMU map entries store a reference to the domain in which they
reside, there is no need to pass the domain to iommu_ga

iommu_gas: Eliminate redundant parameters and push down lock acquisition

Since IOMMU map entries store a reference to the domain in which they
reside, there is no need to pass the domain to iommu_gas_free_entry(),
iommu_gas_free_space(), and iommu_gas_free_region().

Push down the acquisition and release of the IOMMU domain lock into
iommu_gas_free_space() and iommu_gas_free_region().

Both of these changes allow for simplifications in the callers of the
functions without really complicating the functions themselves.
Moreover, the latter change eliminates the direct use of the IOMMU
domain lock from the x86-specific DMAR code.

Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35995

show more ...

x86/iommu: Reduce DMAR lock contention

Replace the DMAR unit's tlb_flush TAILQ by a custom list implementation
that enables dmar_qi_task() to dequeue entries without holding the DMAR
lock.

Reviewed

x86/iommu: Reduce DMAR lock contention

Replace the DMAR unit's tlb_flush TAILQ by a custom list implementation
that enables dmar_qi_task() to dequeue entries without holding the DMAR
lock.

Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35951

show more ...

iommu_gas: Eliminate a possible case of use-after-free

Eliminate a possible case of use-after-free in an error handling path
after a mapping failure. Specifically, eliminate IOMMU_MAP_ENTRY_QI_NF
a

iommu_gas: Eliminate a possible case of use-after-free

Eliminate a possible case of use-after-free in an error handling path
after a mapping failure. Specifically, eliminate IOMMU_MAP_ENTRY_QI_NF
and instead perform the IOTLB invalidation synchronously. Otherwise,
when iommu_domain_unload_entry() is called and told not to free the
IOMMU map entry, the caller could free the entry before dmar_qi_task()
is finished with it.

Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35878

show more ...

iommu: Shrink the iommu map entry structure

Eliminate the unroll_entry field from struct iommu_map_entry, shrinking
the struct by 16 bytes on 64-bit architectures.

Reviewed by: kib
MFC after: 2 wee

iommu: Shrink the iommu map entry structure

Eliminate the unroll_entry field from struct iommu_map_entry, shrinking
the struct by 16 bytes on 64-bit architectures.

Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D35769

show more ...

iommu_gas: Eliminate unnecessary wrappers

Eliminate trivial wrappers for several iommu_gas functions that serve
no functional purpose.

Reviewed by: br, dougm, kib
MFC after: 3 weeks
Differential Re

iommu_gas: Eliminate unnecessary wrappers

Eliminate trivial wrappers for several iommu_gas functions that serve
no functional purpose.

Reviewed by: br, dougm, kib
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D35487

show more ...

vm_extern: use standard address checkers everywhere

Define simple functions for alignment and boundary checks and use them
everywhere instead of having slightly different implementations
scattered a

vm_extern: use standard address checkers everywhere

Define simple functions for alignment and boundary checks and use them
everywhere instead of having slightly different implementations
scattered about. Define them in vm_extern.h and use them where
possible where vm_extern.h is included.

Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D33685

show more ...

Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights

These ones were unambiguous cases where the Foundation was the only
listed copyright holder (in the associated license block).

S

Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights

These ones were unambiguous cases where the Foundation was the only
listed copyright holder (in the associated license block).

Sponsored by: The FreeBSD Foundation

show more ...

dmar: reserve memory windows of PCIe root port

PCI memory address space is shared between memory-mapped devices (MMIO)
and host memory (which may be remapped by an IOMMU). Device accesses to
an addr

dmar: reserve memory windows of PCIe root port

PCI memory address space is shared between memory-mapped devices (MMIO)
and host memory (which may be remapped by an IOMMU). Device accesses to
an address within a memory aperture in a PCIe root port will be treated
as peer-to-peer and not forwarded to an IOMMU. To avoid this, reserve
the address space of the root port's memory apertures in the address
space used by the IOMMU for remapping.

Reviewed by: kib, tychon
Discussed with: Anton Rang <rang@acm.org>
Tested by: tychon
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D27503

show more ...

Add device_t member to struct iommu.

This is needed on arm64 for the interface between iommu framework
and iommu controller drivers.

Reviewed by: kib
Sponsored by: Innovate DSbD
Differential Revisi

Add device_t member to struct iommu.

This is needed on arm64 for the interface between iommu framework
and iommu controller drivers.

Reviewed by: kib
Sponsored by: Innovate DSbD
Differential Revision: https://reviews.freebsd.org/D27229

show more ...

Assign the reserved apic region (GAS entry) to the iommu domain msi_entry.

Requested by: kib
Reviewed by: kib
Sponsored by: Innovate DSbD
Differential Revision: https://reviews.freebsd.org/D26859

Manage MSI iommu pages.

This allows the interrupt controller driver only need a small change to
create a map for the page the device will write to raise an interrupt.

Submitted by: andrew
Reviewed

Manage MSI iommu pages.

This allows the interrupt controller driver only need a small change to
create a map for the page the device will write to raise an interrupt.

Submitted by: andrew
Reviewed by: kib
Sponsored by: Innovate DSbD
Differential Revision: https://reviews.freebsd.org/D26705

show more ...

Split-out Guest Address Space (GAS) macroses to a separate header.

Sponsored by: Innovate DSbD

Split-out iommu type definitions to a separate header.

Reviewed by: kib
Sponsored by: Innovate DSbD
Differential Revision: https://reviews.freebsd.org/D26780

Add iommu_get_ctx_domain() that allows to get iommu domain for a given
iommu context.

Submitted by: andrew
Sponsored by: Innovate DSbD

Rename a header protection macro.

Sponsored by: DARPA, AFRL