mlx5en: add IPSEC_OFFLOAD support

Right now, only IPv4 transport mode, with aes-gcm ESP, is supported.
Driver also cooperates with NAT-T, and obeys socket policies, which
makes IKEd like StrongSwan

mlx5en: add IPSEC_OFFLOAD support

Right now, only IPv4 transport mode, with aes-gcm ESP, is supported.
Driver also cooperates with NAT-T, and obeys socket policies, which
makes IKEd like StrongSwan working.

Sponsored by: NVIDIA networking

show more ...

mlx4, mlx5: Eliminate redundent NULL check for packet filter

mlx4 and mlx5 are Ethernet devices and ether_ifattach() does an
unconditional bpfattach(). From commit 16d878cc99ef [1] and on, we
should

mlx4, mlx5: Eliminate redundent NULL check for packet filter

mlx4 and mlx5 are Ethernet devices and ether_ifattach() does an
unconditional bpfattach(). From commit 16d878cc99ef [1] and on, we
should not check ifp->if_bpf to tell us whether or not we have any bpf
peers that might be interested in receiving packets. And since commit
2b9600b4497b [2], ifp->if_bpf can not be NULL even after the network
interface has been detached.

No functional change intended.

1. 16d878cc99ef Fix the following bpf(4) race condition which can result in a panic
2. 2b9600b4497b Add dead_bpf_if structure, that should be used as fake bpf_if during ifnet detach

Reviewed by: kp, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D45196

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

Mechanically convert mlx5en(4) to IfAPI

Reviewed by: zlei
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38595

mlx5en: Improve CQE error debugging.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5en: Improve RX- and TX- TLS refcounting.

Use the send tag refcounting mechanism to refcount the RX- and TX- TLS
send tags. Then it is no longer needed to wait for refcounts to reach
zero when de

mlx5en: Improve RX- and TX- TLS refcounting.

Use the send tag refcounting mechanism to refcount the RX- and TX- TLS
send tags. Then it is no longer needed to wait for refcounts to reach
zero when destroying RX- and TX- TLS send tags as a result of pending
data or WQE commands.

This also ensures that when TX-TLS and rate limiting is used at the same
time, the underlying SQ is not prematurely destroyed.

MFC after: 1 week
Sponsored by: NVIDIA Networking

show more ...

mlx5: Remove support for FreeBSD 10 and older.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5en: Patch to inhibit transmit doorbell writes during packet reception.

During packet reception the network stack frequently transmit data in
response to TCP window updates. To reduce the number

mlx5en: Patch to inhibit transmit doorbell writes during packet reception.

During packet reception the network stack frequently transmit data in
response to TCP window updates. To reduce the number of transmit doorbells
needed, inhibit all transmit doorbells designated for the same channel until
after the reception of packets for the given channel is completed.

While at it slightly refactor the mlx5e_tx_notify_hw() function:

1) The doorbell information is always stored into sq->doorbell.d64 .
No need to pass a separate pointer to this variable.

2) Move checks for skipping doorbell writes inside this function.

MFC after: 1 week
Sponsored by: NVIDIA Networking

show more ...

mlx5en: idiomatic use of preprocessor, in particular paths

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5en: normalize use of the opt_*.h files

MFC after: 1 week
Sponsored by: NVIDIA Networking

Add a switch structure for send tags.

Move the type and function pointers for operations on existing send
tags (modify, query, next, free) out of 'struct ifnet' and into a new
'struct if_snd_tag_sw'

Add a switch structure for send tags.

Move the type and function pointers for operations on existing send
tags (modify, query, next, free) out of 'struct ifnet' and into a new
'struct if_snd_tag_sw'. A pointer to this structure is added to the
generic part of send tags and is initialized by m_snd_tag_init()
(which now accepts a switch structure as a new argument in place of
the type).

Previously, device driver ifnet methods switched on the type to call
type-specific functions. Now, those type-specific functions are saved
in the switch structure and invoked directly. In addition, this more
gracefully permits multiple implementations of the same tag within a
driver. In particular, NIC TLS for future Chelsio adapters will use a
different implementation than the existing NIC TLS support for T6
adapters.

Reviewed by: gallatin, hselasky, kib (older version)
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D31572

show more ...

mlx5en: vxlan: do not report TCP inner packet as UDP for software parser

Reviewed by: hselasky
Sponsored by: Mellanox Technologies // NVIDIA Networking
MFC after: 1 week

mlx5en: Add more error checks in the transmit path.

- Upon error more completion events than requested may be generated,
particularly when using the completion event factor feature.
- Count number

mlx5en: Add more error checks in the transmit path.

- Upon error more completion events than requested may be generated,
particularly when using the completion event factor feature.
- Count number of event errors in the transmit path.

MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

mlx5en: Fix for IPv6 VxLAN checksum offload.

Add missing check for IPv6 mbuf offload flags and remove L3 WQE checksum flag for IPv6.

MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Techno

mlx5en: Fix for IPv6 VxLAN checksum offload.

Add missing check for IPv6 mbuf offload flags and remove L3 WQE checksum flag for IPv6.

MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

mlx5en: handle checksum and TSO offloading for VxLAN packets on TX

Reviewed by: hselasky
Sponsored by: Mellanox Technologies/NVidia Networking
MFC after: 1 week

mlx5en: eliminate magic constant

Use sizeof(struct udphdr) where appropriate

Reviewed by: hselasky
Sponsored by: Mellanox Technologies/NVidia Networking
MFC after: 1 week

mlx5en: style, remove redundant parentheses

Reviewed by: hselasky
Sponsored by: Mellanox Technologies/NVidia Networking
MFC after: 1 week

Update user access region, UAR, APIs in the core in mlx5core.

This change include several changes as listed below all related to UAR.
UAR is a special PCI memory area where the so-called doorbell re

Update user access region, UAR, APIs in the core in mlx5core.

This change include several changes as listed below all related to UAR.
UAR is a special PCI memory area where the so-called doorbell register and
blue flame register live. Blue flame is a feature for sending small packets
more efficiently via a PCI memory page, instead of using PCI DMA.

- All structures and functions named xxx_uuars were renamed into xxx_bfreg.
- Remove partially implemented Blueflame support from mlx5en(4) and mlx5ib.
- Implement blue flame register allocator.
- Use blue flame register allocator in mlx5ib.
- A common UAR page is now allocated by the core to support doorbell register
writes for all of mlx5en and mlx5ib, instead of allocating one UAR per
sendqueue.
- Add support for DEVX query UAR.
- Add support for 4K UAR for libmlx5.

Linux commits:
7c043e908a74ae0a935037cdd984d0cb89b2b970
2f5ff26478adaff5ed9b7ad4079d6a710b5f27e7
0b80c14f009758cefeed0edff4f9141957964211
30aa60b3bd12bd79b5324b7b595bd3446ab24b52
5fe9dec0d045437e48f112b8fa705197bd7bc3c0
0118717583cda6f4f36092853ad0345e8150b286
a6d51b68611e98f05042ada662aed5dbe3279c1e

MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

Report EQE data upon CQ completion in mlx5core.

Report EQE data upon CQ completion to let upper layers use this data.

Linux commit:
4e0e2ea1886afe8c001971ff767f6670312a9b04

MFC after: 1 week
Spons

Report EQE data upon CQ completion in mlx5core.

Report EQE data upon CQ completion to let upper layers use this data.

Linux commit:
4e0e2ea1886afe8c001971ff767f6670312a9b04

MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

show more ...

Remove IF_SND_TAG_TYPE_TLS_RATE_LIMIT conditionals.

Support for TLS rate limit tags is now in the tree, so this macro is
always defined.

Reviewed by: hselasky
Sponsored by: Netflix
Differential Rev

Remove IF_SND_TAG_TYPE_TLS_RATE_LIMIT conditionals.

Support for TLS rate limit tags is now in the tree, so this macro is
always defined.

Reviewed by: hselasky
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D27020

show more ...

Store the send tag type in the common send tag header.

Both cxgbe(4) and mlx5(4) wrapped the existing send tag header with
their own identical headers that stored the type that the
type-specific tag

Store the send tag type in the common send tag header.

Both cxgbe(4) and mlx5(4) wrapped the existing send tag header with
their own identical headers that stored the type that the
type-specific tag structures inherited from, so in practice it seems
drivers need this in the tag anyway. This permits removing these
extra header indirections (struct cxgbe_snd_tag and struct
mlx5e_snd_tag).

In addition, this permits driver-independent code to query the type of
a tag, e.g. to know what type of tag is being queried via
if_snd_query.

Reviewed by: gallatin, hselasky, np, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D26689

show more ...

Make sure packets generated by raw IP code is let through by mlx5en(4).

Allow the TCP header to reside in the mbuf following the IP header.
Else such packets will get dropped.

Backtrace:
mlx5e_sq_x

Make sure packets generated by raw IP code is let through by mlx5en(4).

Allow the TCP header to reside in the mbuf following the IP header.
Else such packets will get dropped.

Backtrace:
mlx5e_sq_xmit()
mlx5e_xmit()
ether_output_frame()
ether_output()
ip_output_send()
ip_output()
rip_output()
sosend_generic()
sosend()
kern_sendit()
sendit()
sys_sendto()
amd64_syscall()
fast_syscall_common()

MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...

Extend use of unlikely() in the fast path, in mlx5en(4).

Typically the TCP/IP headers fit within the first mbuf and should not
trigger any of the error cases. Use unlikely() for these cases.

No fun

Extend use of unlikely() in the fast path, in mlx5en(4).

Typically the TCP/IP headers fit within the first mbuf and should not
trigger any of the error cases. Use unlikely() for these cases.

No functional change.

MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...

Use const keyword when parsing the TCP/IP header in the fast path in mlx5en(4).

When parsing the TCP/IP header in the fast path, make it clear by using
the const keyword, no fields are to be modifie

Use const keyword when parsing the TCP/IP header in the fast path in mlx5en(4).

When parsing the TCP/IP header in the fast path, make it clear by using
the const keyword, no fields are to be modified inside the transmitted
packet.

No functional change.

MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...

Don't drop packets having too many TCP option headers in mlx5en(4).

When using SACK it can happen there are multiple option headers.
Don't drop these packets, but instead limit the amount of inlinin

Don't drop packets having too many TCP option headers in mlx5en(4).

When using SACK it can happen there are multiple option headers.
Don't drop these packets, but instead limit the amount of inlining
to the maximum supported.

MFC after: 1 week
Sponsored by: Mellanox Technologies

show more ...