MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

iflib: netmap: improve rxsync to support IFLIB_HAS_RXCQ

For drivers with IFLIB_HAS_RXCQ set, there is a separate completion
queue. In this case, the netmap rxsync routine needs to update
rxq->ifr_cq

iflib: netmap: improve rxsync to support IFLIB_HAS_RXCQ

For drivers with IFLIB_HAS_RXCQ set, there is a separate completion
queue. In this case, the netmap rxsync routine needs to update
rxq->ifr_cq_cidx in the same way it is updated by iflib_rxeof().
This improves the situation for vmx(4) and bnxt(4) drivers, which
use iflib and have the IFLIB_HAS_RXCQ bit set.

PR: 248494
MFC after: 3 weeks

show more ...

iflib: refactor netmap_fl_refill and fix off-by-one issue

First, fix the initialization of the fl->ifl_rxd_idxs array,
which was affected by an off-by-one bug.
Once there, refactor the function to u

iflib: refactor netmap_fl_refill and fix off-by-one issue

First, fix the initialization of the fl->ifl_rxd_idxs array,
which was affected by an off-by-one bug.
Once there, refactor the function to use better names for
local variables, optimize the variable assignments, and
merge the bus_dmamap_sync() inner loop with the outer one.

PR: 248494
MFC after: 3 weeks

show more ...

Merge ^/head r363989 through r364034.

iflib: netmap: drop redundant check

The validity of head is already checked by nm_rxsync_prologue().

MFC after: 2 weeks

iflib: netmap: don't increment ifl_cidx on the wrong free list

Netmap only uses free list 0 to keep it consistent with its
one-to-one mapping between each netmap ring and a device RX
(or TX) queue.

iflib: netmap: don't increment ifl_cidx on the wrong free list

Netmap only uses free list 0 to keep it consistent with its
one-to-one mapping between each netmap ring and a device RX
(or TX) queue.
However, the current iflib_netmap_rxsync() routine was
mistakenly updating the ifl_cidx field of both free lists.

PR: 248494
MFC after: 2 weeks

show more ...

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

Merge ^/head r363583 through r363738.

iflib: fix LOR with bpf detach

Reported by: grehan@
Approved by: grehan@
MFC after: 1 week
Sponsored by: Netgate
Differential Revision: https://reviews.freebsd.org/D25530

iflib: initialize netmap with the correct number of descriptors

In case the network device has a RX or TX control queue, the correct
number of TX/RX descriptors is contained in the second entry of t

iflib: initialize netmap with the correct number of descriptors

In case the network device has a RX or TX control queue, the correct
number of TX/RX descriptors is contained in the second entry of the
isc_ntxd (or isc_nrxd) array, rather than in the first entry.
This case is correctly handled by iflib_device_register() and
iflib_pseudo_register(), but not by iflib_netmap_attach().
If the first entry is larger than the second, this can result in a
panic. This change fixes the bug by introducing two helper functions
that also lead to some code simplification.

PR: 247647
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D25541

show more ...

iflib: Fix some nits in the rx refill code.

- Get rid of the ifl_vm_addrs array. It is not used by any existing
consumer, so we are just dirtying a couple of cache lines for no
reason.
- Use um

iflib: Fix some nits in the rx refill code.

- Get rid of the ifl_vm_addrs array. It is not used by any existing
consumer, so we are just dirtying a couple of cache lines for no
reason.
- Use uma_zalloc(fl->ifl_zone) instead of m_cljget(). Otherwise
m_cljget() is doing unnecessary work to look up the correct zone, when
iflib already knows what that zone is.
- ifl_gen is only used when INVARIANTS is on, so make that more clear.
- Fix some style nits and inconsistencies.

Reviewed by: gallatin
Tested by: pho
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25490

show more ...

iflib: Fix handling of mbuf cluster allocation failures.

When refilling an rx freelist, make sure we only update the hardware
producer index if at least one cluster was allocated. Otherwise the
NIC

iflib: Fix handling of mbuf cluster allocation failures.

When refilling an rx freelist, make sure we only update the hardware
producer index if at least one cluster was allocated. Otherwise the
NIC is programmed to write a previously used cluster, typically
resulting in a use-after-free when packet data is written by the
hardware.

Also make sure that we don't update the fragment index cursor if the
last allocation attempt didn't succeed. For at least Intel drivers,
iflib assumes that the consumer index and fragment index cursor stay in
lockstep, but this assumption was violated in the face of cluster
allocation failures.

Reported and tested by: pho
Reviewed by: gallatin, hselasky
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25489

show more ...

iflib: fix compilation issue introduced in r362621

The ifp local variable is useful even without netmap
and altq, as it is used to check for IFF_DRV_RUNNING.

MFC after: 2 weeks

iflib: netmap: add support for partial ring openings

Reviewed by: gallatin
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25254

iflib: netmap: add per-tx-queue netmap support

Reviewed by: gallatin
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25253

iflib: netmap: fix rsync index overrun

In the current iflib_netmap_rxsync, there is nothing that prevents
kring->nr_hwtail to overrun kring->nr_hwcur during the descriptor
import phase. This may cau

iflib: netmap: fix rsync index overrun

In the current iflib_netmap_rxsync, there is nothing that prevents
kring->nr_hwtail to overrun kring->nr_hwcur during the descriptor
import phase. This may cause errors in netmap applications, such as:

em1 RX0: fail 'head < kring->nr_hwcur || head > kring->nr_hwtail'
h 795 c 795 t 282 rh 795 rc 795 rt 282 hc 282 ht 282

Reviewed by: gallatin
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25252

show more ...

iflib: fix cloneattach fail and generalize pseudo device handling

- a cloneattach failure will not currently be handled correctly,
jump to the right target

- pseudo devices are all treat as if th

iflib: fix cloneattach fail and generalize pseudo device handling

- a cloneattach failure will not currently be handled correctly,
jump to the right target

- pseudo devices are all treat as if they're ethernet devices -
this often doesn't make sense

MFC after: 1 week
Sponsored by: Netgate, Inc.
Differential Revision: https://reviews.freebsd.org/D25083

show more ...

iflib: netmap: enter/exit netmap mode after device stops

Avoid possible race conditions by calling nm_set_native_flags()
and nm_clear_native_flags() only after the device has been
stopped.

MFC afte

iflib: netmap: enter/exit netmap mode after device stops

Avoid possible race conditions by calling nm_set_native_flags()
and nm_clear_native_flags() only after the device has been
stopped.

MFC after: 1 week

show more ...

iflib: netmap: honor netmap_irx_irq return values

In the receive interrupt routine, always call netmap_rx_irq().
The latter function will return != NM_IRQ_PASS if netmap is not
active on that specif

iflib: netmap: honor netmap_irx_irq return values

In the receive interrupt routine, always call netmap_rx_irq().
The latter function will return != NM_IRQ_PASS if netmap is not
active on that specific receive queue, so that the driver can go
on with iflib_rxeof(). Note that netmap supports partial opening,
where only a subset of the RX or TX rings can be open in netmap mode.
Checking the IFCAP_NETMAP flag is not enough to make sure that the
queue is indeed in netmap mode.
Moreover, in case netmap_rx_irq() returns NM_IRQ_RESCHED, it means
that netmap expects the driver to call netmap_rx_irq() again as soon
as possible. Currently, this may happen when the device is attached
to a VALE switch.

Reviewed by: gallatin
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25167

show more ...

Fix panics when using iflib pseudo device support

Reviewed by: gallatin@, hselasky@
MFC after: 1 week
Sponsored by: Netgate, Inc.
Differential Revision: https://reviews.freebsd.org/D23710

Increase the iflib txq callout mutex name length to 32 bytes.

With a length of 16, the name ("<if name>:TX(<qid>):callout") typically
gets truncated.

PR: 245712
Reported by: ghuckriede@blackberry.

Increase the iflib txq callout mutex name length to 32 bytes.

With a length of 16, the name ("<if name>:TX(<qid>):callout") typically
gets truncated.

PR: 245712
Reported by: ghuckriede@blackberry.com
MFC after: 1 week

show more ...

iflib: Stop interface before (un)registering VLAN

This patch is intended to solve a specific problem that iavf(4)
encounters, but what it does can be extended to solve other issues.

To summarize th

iflib: Stop interface before (un)registering VLAN

This patch is intended to solve a specific problem that iavf(4)
encounters, but what it does can be extended to solve other issues.

To summarize the iavf(4) issue, if the PF driver configures VLAN
anti-spoof, then the VF driver needs to make sure no untagged traffic is
sent if a VLAN is configured, and vice-versa. This can be an issue when
a VLAN is being registered or unregistered, e.g. when a packet may be on
the ring with a VLAN in it, but the VLANs are being unregistered. This
can cause that tagged packet to go out and cause an MDD event.

To fix this, include a new interface-dependent function that drivers can
implement named IFDI_NEEDS_RESTART(). Right now, this function is called
in iflib_vlan_unregister/register() to determine whether the interface
needs to be stopped and started when a VLAN is registered or
unregistered. The default return value of IFDI_NEEDS_RESTART() is true,
so this fixes the MDD problem that iavf(4) encounters, since the
interface rings are flushed during a stop/init.

A future change to iavf(4) will implement that function just in case the
default value changes, and to make it explicit that this interface reset
is required when a VLAN is added or removed.

Reviewed by: gallatin@
MFC after: 1 week
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D22086

show more ...

Simplify taskqgroup inititialization.

taskqgroup initialization was broken into two steps:

1. allocate the taskqgroup structure, at SI_SUB_TASKQ;
2. initialize taskqueues, start taskqueue threads,

Simplify taskqgroup inititialization.

taskqgroup initialization was broken into two steps:

1. allocate the taskqgroup structure, at SI_SUB_TASKQ;
2. initialize taskqueues, start taskqueue threads, enqueue "binder"
tasks to bind threads to specific CPUs, at SI_SUB_SMP.

Step 2 tries to handle the case where tasks have already been attached
to a queue, by migrating them to their intended queue. In particular,
tasks can't be enqueued before step 2 has completed. This breaks NFS
mountroot on systems using an iflib-based driver when EARLY_AP_STARTUP
is not defined, since mountroot happens before SI_SUB_SMP in this case.

Simplify initialization: do all initialization except for CPU binding at
SI_SUB_TASKQ. This means that until CPU binding is completed, group
tasks may be executed on a CPU other than that to which they were bound,
but this should not be a problem for existing users of the taskqgroup
KPIs.

Reported by: sbruno
Tested by: bdragon, sbruno
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D24188

show more ...

iflib: simplify MPASS assertion

Submitted by: andrew

iflib: split compound assertion

ThunderX cluster systems are panicking on boot with a failed assertion
MPASS(gtask != NULL && gtask->gt_taskqueue != NULL). Split the
assertion so that it's clear wh

iflib: split compound assertion

ThunderX cluster systems are panicking on boot with a failed assertion
MPASS(gtask != NULL && gtask->gt_taskqueue != NULL). Split the
assertion so that it's clear which part is failing.

show more ...