sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

net: replace IFF_KNOWSEPOCH with IFF_NEEDSEPOCH

Expect that drivers call into the network stack with the net epoch
entered. This has already been the fact since early 2020. The net
interrupts, that

net: replace IFF_KNOWSEPOCH with IFF_NEEDSEPOCH

Expect that drivers call into the network stack with the net epoch
entered. This has already been the fact since early 2020. The net
interrupts, that are marked with INTR_TYPE_NET, were entering epoch
since 511d1afb6bf. For the taskqueues there is NET_TASK_INIT() and
all drivers that were known back in 2020 we marked with it in
6c3e93cb5a4. However in e87c4940156 we took conservative approach
and preferred to opt-in rather than opt-out for the epoch.

This change not only reverts e87c4940156 but adds a safety belt to
avoid panicing with INVARIANTS if there is a missed driver. With
INVARIANTS we will run in_epoch() check, print a warning and enter
the net epoch. A driver that prints can be quickly fixed with the
IFF_NEEDSEPOCH flag, but better be augmented to properly enter the
epoch itself.

Note on TCP LRO: it is a backdoor to enter the TCP stack bypassing
some layers of net stack, ignoring either old IFF_KNOWSEPOCH or the
new IFF_NEEDSEPOCH. But the tcp_lro_flush_all() asserts the presence
of network epoch. Indeed, all NIC drivers that support LRO already
provide the epoch, either with help of INTR_TYPE_NET or just running
NET_EPOCH_ENTER() in their code.

Reviewed by: zlei, gallatin, erj
Differential Revision: https://reviews.freebsd.org/D39510

show more ...

infiniband: Opt-in for net epoch

This is counterpart to e87c4940156c, which did the same for ethernet.

Suggested by: hselasky
Reviewed by: hselasky, kib
MFC after: 2 weeks
Differential Revision: ht

infiniband: Opt-in for net epoch

This is counterpart to e87c4940156c, which did the same for ethernet.

Suggested by: hselasky
Reviewed by: hselasky, kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D39405

show more ...

infiniband: Widen NET_EPOCH coverage

From static code analysis, some device drivers (cxgbe, mlx4, mthca, and qlnx)
do not enter net epoch before lagg_input_infiniband(). If IPoIB interface is a
memb

infiniband: Widen NET_EPOCH coverage

From static code analysis, some device drivers (cxgbe, mlx4, mthca, and qlnx)
do not enter net epoch before lagg_input_infiniband(). If IPoIB interface is a
member of lagg(4) interface, and after returning from lagg_input_infiniband()
the receiving interface of mbuf is set to lagg(4) interface, then when
concurrently destroying the lagg(4) interface, there is a small window that the
interface gets destroyed and becomes invalid before infiniband_input() re-enter
net epoch, thus leading use-after-free.

Widen NET_EPOCH coverage to prevent use-after-free.

Thanks hselasky@ for testing with mlx5 devices.

Reviewed by: hselasky
Tested by: hselasky
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39275

show more ...

infiniband: Bring back M_ASSERTVALID() check in infiband_bpf_mtap()

Reported by: rpokala
Fixes: adf62e83631

infiniband: Convert BPF handling for IfAPI

Summary:
All callers of infiniband_bpf_mtap() call it through the wrapper macro,
which checks the if_bpf member explicitly. Since this is getting
hidden,

infiniband: Convert BPF handling for IfAPI

Summary:
All callers of infiniband_bpf_mtap() call it through the wrapper macro,
which checks the if_bpf member explicitly. Since this is getting
hidden, move this check into the internal function and remove the
wrapper macro.

Reviewed by: hselasky
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D39024

show more ...

ifnet/API: Move struct ifnet definition to a <net/if_private.h>

Hide the ifnet structure definition, no user serviceable parts inside,
it's a netstack implementation detail. Include it temporarily

ifnet/API: Move struct ifnet definition to a <net/if_private.h>

Hide the ifnet structure definition, no user serviceable parts inside,
it's a netstack implementation detail. Include it temporarily in
<net/if_var.h> until all drivers are updated to use the accessors
exclusively.

Reviewed by: glebius
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38046

show more ...

infiniband_resolve_addr: ih is only used for INET or INET6.

routing: Allow using IPv6 next-hops for IPv4 routes (RFC 5549).

Implement kernel support for RFC 5549/8950.

* Relax control plane restrictions and allow specifying IPv6 gateways
for IPv4 routes. T

routing: Allow using IPv6 next-hops for IPv4 routes (RFC 5549).

Implement kernel support for RFC 5549/8950.

* Relax control plane restrictions and allow specifying IPv6 gateways
for IPv4 routes. This behavior is controlled by the
net.route.rib_route_ipv6_nexthop sysctl (on by default).

* Always pass final destination in ro->ro_dst in ip_forward().

* Use ro->ro_dst to exract packet family inside if_output() routines.
Consistently use RO_GET_FAMILY() macro to handle ro=NULL case.

* Pass extracted family to nd6_resolve() to get the LLE with proper encap.
It leverages recent lltable changes committed in c541bd368f86.

Presence of the functionality can be checked using ipv4_rfc5549_support feature(3).
Example usage:
route add -net 192.0.0.0/24 -inet6 fe80::5054:ff:fe14:e319%vtnet0

Differential Revision: https://reviews.freebsd.org/D30398
MFC after: 2 weeks

show more ...

lltable: Add support for "child" LLEs holding encap for IPv4oIPv6 entries.

Currently we use pre-calculated headers inside LLE entries as prepend data
for `if_output` functions. Using these headers

lltable: Add support for "child" LLEs holding encap for IPv4oIPv6 entries.

Currently we use pre-calculated headers inside LLE entries as prepend data
for `if_output` functions. Using these headers allows saving some
CPU cycles/memory accesses on the fast path.

However, this approach makes adding L2 header for IPv4 traffic with IPv6
nexthops more complex, as it is not possible to store multiple
pre-calculated headers inside lle. Additionally, the solution space is
limited by the fact that PCB caching saves LLEs in addition to the nexthop.

Thus, add support for creating special "child" LLEs for the purpose of holding
custom family encaps and store mbufs pending resolution. To simplify handling
of those LLEs, store them in a linked-list inside a "parent" (e.g. normal) LLE.
Such LLEs are not visible when iterating LLE table. Their lifecycle is bound
to the "parent" LLE - it is not possible to delete "child" when parent is alive.
Furthermore, "child" LLEs are static (RTF_STATIC), avoding complex state
machine used by the standard LLEs.

nd6_lookup() and nd6_resolve() now accepts an additional argument, family,
allowing to return such child LLEs. This change uses `LLE_SF()` macro which
packs family and flags in a single int field. This is done to simplify merging
back to stable/. Once this code lands, most of the cases will be converted to
use a dedicated `family` parameter.

Differential Revision: https://reviews.freebsd.org/D31379
MFC after: 2 weeks

show more ...

[lltable] Unify datapath feedback mechamism.

Use newly-create llentry_request_feedback(),
llentry_mark_used() and llentry_get_hittime() to
request datapatch usage check and fetch the results
in t

[lltable] Unify datapath feedback mechamism.

Use newly-create llentry_request_feedback(),
llentry_mark_used() and llentry_get_hittime() to
request datapatch usage check and fetch the results
in the same fashion both in IPv4 and IPv6.

While here, simplify llentry_provide_feedback() wrapper
by eliminating 1 condition check.

MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D31390

show more ...

Streamline the infiniband code according to the ethernet code.

Fix LINT-NOIP kernel build.

Submitted by: rlibby @
Differential Revision: https://reviews.freebsd.org/D27861
MFC after: 1 week
Sponsor

Streamline the infiniband code according to the ethernet code.

Fix LINT-NOIP kernel build.

Submitted by: rlibby @
Differential Revision: https://reviews.freebsd.org/D27861
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

Streamline the infiniband code according to the ethernet code.

Specifically implement the if_requestencap callback function for infiniband.
Most of the changes are simply a cut and paste of the equi

Streamline the infiniband code according to the ethernet code.

Specifically implement the if_requestencap callback function for infiniband.
Most of the changes are simply a cut and paste of the equivalent ethernet part.

Reviewed by: melifaro @
Differential Revision: https://reviews.freebsd.org/D27631
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

More style fixes (partial revert of r366994).

Suggested by: danfe@
Differential Revision: https://reviews.freebsd.org/D26254
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networ

More style fixes (partial revert of r366994).

Suggested by: danfe@
Differential Revision: https://reviews.freebsd.org/D26254
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

Fix order of header files:
sys/systm.h should come right after sys/param.h

Suggested by: kib@
Differential Revision: https://reviews.freebsd.org/D26254
MFC after: 1 week
Sponsored by: Mellanox Te

Fix order of header files:
sys/systm.h should come right after sys/param.h

Suggested by: kib@
Differential Revision: https://reviews.freebsd.org/D26254
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

Run code through "clang-format -style=file" with some additional fixes.
No functional change.

Suggested by: kib@ and emaste@
Differential Revision: https://reviews.freebsd.org/D26254
MFC after: 1

Run code through "clang-format -style=file" with some additional fixes.
No functional change.

Suggested by: kib@ and emaste@
Differential Revision: https://reviews.freebsd.org/D26254
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

Compile fix for MIPS, MIPS64, POWERPC and POWERPC64.
Add missing include files.

Differential Revision: https://reviews.freebsd.org/D26254
Reviewed by: melifaro@
MFC after: 1 week
Sponsored by: Me

Compile fix for MIPS, MIPS64, POWERPC and POWERPC64.
Add missing include files.

Differential Revision: https://reviews.freebsd.org/D26254
Reviewed by: melifaro@
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

Add support for IP over infiniband, IPoIB, to lagg(4). Currently only
the failover protocol is supported due to limitations in the IPoIB
architecture. Refer to the lagg(4) manual page for how to conf

Add support for IP over infiniband, IPoIB, to lagg(4). Currently only
the failover protocol is supported due to limitations in the IPoIB
architecture. Refer to the lagg(4) manual page for how to configure
and use this new feature. A new network interface type,
IFT_INFINIBANDLAG, has been added, similar to the existing
IFT_IEEE8023ADLAG .

ifconfig(8) has been updated to accept a new laggtype argument when
creating lagg(4) network interfaces. This new argument is used to
distinguish between ethernet and infiniband type of lagg(4) network
interface. The laggtype argument is optional and defaults to
ethernet. The lagg(4) command line syntax is backwards compatible.

Differential Revision: https://reviews.freebsd.org/D26254
Reviewed by: melifaro@
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

Factor out generic IP over infiniband, IPoIB, definitions and code
into net/if_infiniband.c and net/infiniband.h . No functional change
intended.

Differential Revision: https://reviews.freebsd.org/D

Factor out generic IP over infiniband, IPoIB, definitions and code
into net/if_infiniband.c and net/infiniband.h . No functional change
intended.

Differential Revision: https://reviews.freebsd.org/D26254
Reviewed by: melifaro@
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...