iflib: Some style(9) tweaks

Prefer tab over eight whitespaces for indentation, and four spaces as
second level idents.

MFC after: 3 days

iflib: Simplify CACHE_PTR_NEXT

Reviewed by: Krzysztof Galazka <krzysztof.galazka@intel.com>
Obtained from: CheriBSD
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D48446

iflib(4): Replace admin taskqueue group with per-interface taskqueues

Using one taskqueue group with single thread to execute all admin
tasks may lead to unexpected timeouts when long running task (

iflib(4): Replace admin taskqueue group with per-interface taskqueues

Using one taskqueue group with single thread to execute all admin
tasks may lead to unexpected timeouts when long running task (e.g.
handling a reset after FW update) for one interface prevents
tasks from other interfaces being executed. Taskqueue group API
doesn't let to dynamically add threads, and pre-allocating thread
for each CPU as it's done for traffic queues would be a waste
of resources on systems with small number of interfaces. Replace
global taskqueue group for admin tasks with taskqueue allocated
for each interface to allow independent execution.

Signed-off-by: Krzysztof Galazka <krzysztof.galazka@intel.com>
Reviewed by: imp, jhb
Pull Request: https://github.com/freebsd/freebsd-src/pull/1336

show more ...

iflib: Set the NUMA domain in receive packet headers

Reading PR 273559 made me realize that commit 767723ddebe9 is
incomplete. iflib should set the NUMA domain of received packets before
passing th

iflib: Set the NUMA domain in receive packet headers

Reading PR 273559 made me realize that commit 767723ddebe9 is
incomplete. iflib should set the NUMA domain of received packets before
passing them to protocol layers.

PR: 273559
Reviewed by: zlei, kbowling, erj
Fixes: 767723ddebe9 ("iflib: Use if_alloc_dev() to allocate the ifnet")
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D47841

show more ...

tcp: extend the use of the th_flags accessor function

Formally, there are 12 bits for TCP header flags.
Use the accessor functions in more (kernel) places.

No functional change.

Reviewed By: cc, #

tcp: extend the use of the th_flags accessor function

Formally, there are 12 bits for TCP header flags.
Use the accessor functions in more (kernel) places.

No functional change.

Reviewed By: cc, #transport, cy, glebius, #iflib, kbowling
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D47063

show more ...

iflib: Make iflib_stop() static

It is declared as static. Make the definition consistent with the
declaration.

This follows 7ff9ae90f0bd and partially reverts 09f6ff4f1a47.

Reviewed by: erj
MFC af

iflib: Make iflib_stop() static

It is declared as static. Make the definition consistent with the
declaration.

This follows 7ff9ae90f0bd and partially reverts 09f6ff4f1a47.

Reviewed by: erj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D46185

show more ...

iflib: Simplify iflib_legacy_setup

Follow the pattern from iflib_irq_alloc_generic function and use
iflib_fast_intr as a handler for RX only interrupts.

Also remove some intermediate variables and

iflib: Simplify iflib_legacy_setup

Follow the pattern from iflib_irq_alloc_generic function and use
iflib_fast_intr as a handler for RX only interrupts.

Also remove some intermediate variables and use consistent
way for referencing queue's structures.

Signed-off-by: Krzysztof Galazka <krzysztof.galazka@intel.com>

Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D46061

show more ...

iflib: Many style fixes

MFC after: 3 days

iflib: Prefer C99's __func__ over GCC's __FUNCTION__

MFC after: 3 days

iflib: Use if_alloc_dev() to allocate the ifnet

This ensures that the ifnet's NUMA affinity is accurate.

Reviewed by: kbowling
MFC after: 2 weeks
Sponsored by: Klara, Inc.
Differential Revision: ht

iflib: Use if_alloc_dev() to allocate the ifnet

This ensures that the ifnet's NUMA affinity is accurate.

Reviewed by: kbowling
MFC after: 2 weeks
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D46667

show more ...

net: Remove unneeded NULL check for the allocated ifnet

Change 4787572d0580 made if_alloc_domain() never fail, then also do the
wrappers if_alloc(), if_alloc_dev(), and if_gethandle().

No functiona

net: Remove unneeded NULL check for the allocated ifnet

Change 4787572d0580 made if_alloc_domain() never fail, then also do the
wrappers if_alloc(), if_alloc_dev(), and if_gethandle().

No functional change intended.

Reviewed by: kp, imp, glebius, stevek
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D45740

show more ...

iflib: Add subinterface interrupt allocation function

The ice(4) driver will add the ability to create extra interfaces
that hang off of the base interface; to do that the driver requires
a method f

iflib: Add subinterface interrupt allocation function

The ice(4) driver will add the ability to create extra interfaces
that hang off of the base interface; to do that the driver requires
a method for the subinterface to request hardware interrupt resources
from the base interface.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

MFC after: 3 days
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D39930

show more ...

iflib: Add sysctl to request extra MSIX vectors on driver load

Intended to be used with upcoming feature to add sub-interfaces, since
those new interfaces will be dynamically created and will need t

iflib: Add sysctl to request extra MSIX vectors on driver load

Intended to be used with upcoming feature to add sub-interfaces, since
those new interfaces will be dynamically created and will need to have
spare MSI-X interrupts already allocated for them on driver load.

This sysctl is marked as a tunable since it will need to be set before
the driver is loaded since MSI-X interrupt allocation and setup is
done during the attach process.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

MFC after: 3 days
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D41326

show more ...

iflib: Correct indentation according to style(9)

The indentation style for the SYSCTL_* macros used was not matching KNF.

Reported by: jhb
Differential Revision: https://reviews.freebsd.org/D44811

iflib: Fix compiler warnings

Some of the QUAD sysctls are actually for unsigned quad values.
Switch to using UQUAD instead, as that is meant for unsigned.

Reviewed by: erj, jhb
Obtained from: Junip

iflib: Fix compiler warnings

Some of the QUAD sysctls are actually for unsigned quad values.
Switch to using UQUAD instead, as that is meant for unsigned.

Reviewed by: erj, jhb
Obtained from: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D44620

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

iflib: Remove redundant variable

In iflib_init_locked(), sctx and scctx both point to the same value,
which is the ifc_softc_ctx field in the iflib softc. Remove the
declaration and assignment to sc

iflib: Remove redundant variable

In iflib_init_locked(), sctx and scctx both point to the same value,
which is the ifc_softc_ctx field in the iflib softc. Remove the
declaration and assignment to sctx since scctx can be used instead, and
the name of scctx follows the naming convention used for local variables
that point to ifc_softc_ctx.

In theory there should be no functional impact with this change.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Reviewed by: kbowling@
MFC after: 3 days
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D41325

show more ...

iflib: Fix white space and reduce some line lengths

This helps align some of the code with the rest of the style used in
iflib, but as marius@ points out, this is not style(9).

Signed-off-by: Eric

iflib: Fix white space and reduce some line lengths

This helps align some of the code with the rest of the style used in
iflib, but as marius@ points out, this is not style(9).

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Reviewed by: kbowling@
MFC after: 3 days
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D41324

show more ...

iflib(9): Remove support for cloning pseudo interfaces

This code was used by the first incarnation of wg(4) and is dead ever
since f187d6dfbf633665ba6740fe22742aec60ce02a2 has removed the latter
aga

iflib(9): Remove support for cloning pseudo interfaces

This code was used by the first incarnation of wg(4) and is dead ever
since f187d6dfbf633665ba6740fe22742aec60ce02a2 has removed the latter
again. Moreover, this code matched iflib(4) like a square peg fits in
a round hole, was incomplete and despite some hacks still tailored to
VPC and wg(4) but not generic. In effect, this reverts the following:
09f6ff4f1a47c3009dc16fdc609a44f2341bc7ac (w/ its "ancillary changes")
9aeca21324f481f57f2ecb7009f461f4f51b62b3
1f93e931d9f0c688f43f98ef777e04636a325526
0f9544d03e89d180f94a7a84b110ec7d2b6c625a
0dd691b41276ce13d25ffb1443af27f85038aa3f

Reviewed by: erj, kbowling
Differential Revision: <https://reviews.freebsd.org/D41196>

show more ...

iflib: Fix panic during driver reload stress test

During a driver reload stress test, after 50-300 reloads a panic occurs.
After adding sleeps in between loading and unloading the driver, the
issue

iflib: Fix panic during driver reload stress test

During a driver reload stress test, after 50-300 reloads a panic occurs.
After adding sleeps in between loading and unloading the driver, the
issue does not occur. It's possible that loading/unloading too fast may
cause the gt_taskqueue pointer to be freed earlier than expected;
checking for a null pointer first fixes it.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Reviewed by: erj@
Tested by: jeffrey.e.pieper@intel.com
MFC after: 3 days
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D39457

show more ...

iflib: Unlock ctx lock around call to ether_ifattach()

Panic occurs during loading driver using kldload. It exists since netlink is
enabled. There is problem with double locking ctx. This fix allow

iflib: Unlock ctx lock around call to ether_ifattach()

Panic occurs during loading driver using kldload. It exists since netlink is
enabled. There is problem with double locking ctx. This fix allows to call
ether_ifattach() without locked ctx.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

PR: 271768
Reviewed by: erj@, jhb@
MFC after: 1 day
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D40557

show more ...

net: replace IFF_KNOWSEPOCH with IFF_NEEDSEPOCH

Expect that drivers call into the network stack with the net epoch
entered. This has already been the fact since early 2020. The net
interrupts, that

net: replace IFF_KNOWSEPOCH with IFF_NEEDSEPOCH

Expect that drivers call into the network stack with the net epoch
entered. This has already been the fact since early 2020. The net
interrupts, that are marked with INTR_TYPE_NET, were entering epoch
since 511d1afb6bf. For the taskqueues there is NET_TASK_INIT() and
all drivers that were known back in 2020 we marked with it in
6c3e93cb5a4. However in e87c4940156 we took conservative approach
and preferred to opt-in rather than opt-out for the epoch.

This change not only reverts e87c4940156 but adds a safety belt to
avoid panicing with INVARIANTS if there is a missed driver. With
INVARIANTS we will run in_epoch() check, print a warning and enter
the net epoch. A driver that prints can be quickly fixed with the
IFF_NEEDSEPOCH flag, but better be augmented to properly enter the
epoch itself.

Note on TCP LRO: it is a backdoor to enter the TCP stack bypassing
some layers of net stack, ignoring either old IFF_KNOWSEPOCH or the
new IFF_NEEDSEPOCH. But the tcp_lro_flush_all() asserts the presence
of network epoch. Indeed, all NIC drivers that support LRO already
provide the epoch, either with help of INTR_TYPE_NET or just running
NET_EPOCH_ENTER() in their code.

Reviewed by: zlei, gallatin, erj
Differential Revision: https://reviews.freebsd.org/D39510

show more ...

iflib: Further convert to use IfAPI accessors

Summary:
When iflib was first converted some IfAPI APIs were not yet present, so
were tagged with "XXX" comments. Finish the conversion by using these

iflib: Further convert to use IfAPI accessors

Summary:
When iflib was first converted some IfAPI APIs were not yet present, so
were tagged with "XXX" comments. Finish the conversion by using these
new APIs.

Reviewed by: gallatin, erj
Sponsored by: Juniper Networks, Inc
Differential Revision: https://reviews.freebsd.org/D38928

show more ...

iflib: fix regression with new pfil(9) KPI

Do not pass the pointer to our valid mbuf to pfil(9). Pass an
uninitialized one only. This was unsafe with the old KPI, too,
but for some reason didn't f

iflib: fix regression with new pfil(9) KPI

Do not pass the pointer to our valid mbuf to pfil(9). Pass an
uninitialized one only. This was unsafe with the old KPI, too,
but for some reason didn't fail.

Fixes: caf32b260ad46b17a4c1a8ce6383e37ac489f023

show more ...

pfil: add pfil_mem_{in,out}() and retire pfil_run_hooks()

The 0b70e3e78b0 changed the original design of a single entry point
into pfil(9) chains providing separate functions for the filtering
point

pfil: add pfil_mem_{in,out}() and retire pfil_run_hooks()

The 0b70e3e78b0 changed the original design of a single entry point
into pfil(9) chains providing separate functions for the filtering
points that always provide mbufs and know the direction of a flow.
The motivation was to reduce branching. The logical continuation
would be to do the same for the filtering points that always provide
a memory pointer and retire the single entry point.

o Hooks now provide two functions: one for mbufs and optional for
memory pointers.
o pfil_hook_args() has a new member and pfil_add_hook() has a
requirement to zero out uninitialized data. Bump PFIL_VERSION.
o As it was before, a hook function for a memory pointer may realloc
into an mbuf. Such mbuf would be returned via a pointer that must
be provided in argument.
o The only hook that supports memory pointers is ipfw:default-link.
It is rewritten to provide two functions.
o All remaining uses of pfil_run_hooks() are converted to
pfil_mem_in().
o Transparent union of pfil_packet_t and tricks to fix pointer
alignment are retired. Internal pfil_realloc() reduces down to
m_devget() and thus is retired, too.

Reviewed by: mjg, ocochard
Differential revision: https://reviews.freebsd.org/D37977

show more ...