Revision tags: release/14.0.0 |
|
#
95ee2897 |
| 16-Aug-2023 |
Warner Losh <imp@FreeBSD.org> |
sys: Remove $FreeBSD$: two-line .h pattern
Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
|
Revision tags: release/13.2.0, release/12.4.0 |
|
#
5dc00f00 |
| 19-Sep-2022 |
Justin Hibbits <jhibbits@FreeBSD.org> |
Mechanically convert mlx5en(4) to IfAPI
Reviewed by: zlei Sponsored by: Juniper Networks, Inc. Differential Revision: https://reviews.freebsd.org/D38595
|
Revision tags: release/13.1.0 |
|
#
c3b1cbc9 |
| 12-Apr-2022 |
John Baldwin <jhb@FreeBSD.org> |
mlx5 RATELIMIT: Remove an unused variable.
|
#
a8e715d2 |
| 01-Feb-2022 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
mlx5en: Add race protection for SQ remap
Add a refcount for posted WQEs to avoid a race between post WQE and FW command flows.
MFC after: 1 week Sponsored by: NVIDIA Networking
|
#
266c81aa |
| 01-Feb-2022 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
mlx5/mlx5en: Add SQ remap support
Add support to map an SQ to a specific schedule queue using a special WQE as performance enhancement.
SQ remap operation is handled by a privileged internal queue,
mlx5/mlx5en: Add SQ remap support
Add support to map an SQ to a specific schedule queue using a special WQE as performance enhancement.
SQ remap operation is handled by a privileged internal queue, IQ, and the mapping is enabled from one rate to another.
The transition from paced to non-paced should however always go through FW.
MFC after: 1 week Sponsored by: NVIDIA Networking
show more ...
|
Revision tags: release/12.3.0 |
|
#
89918a23 |
| 14-Jun-2021 |
Konstantin Belousov <konstantinb@nvidia.com> |
mlx5en: idiomatic use of preprocessor, in particular paths
MFC after: 1 week Sponsored by: NVIDIA Networking
|
#
b984b956 |
| 14-Jun-2021 |
Konstantin Belousov <konstantinb@nvidia.com> |
mlx5en: normalize use of the opt_*.h files
MFC after: 1 week Sponsored by: NVIDIA Networking
|
#
c782ea8b |
| 14-Sep-2021 |
John Baldwin <jhb@FreeBSD.org> |
Add a switch structure for send tags.
Move the type and function pointers for operations on existing send tags (modify, query, next, free) out of 'struct ifnet' and into a new 'struct if_snd_tag_sw'
Add a switch structure for send tags.
Move the type and function pointers for operations on existing send tags (modify, query, next, free) out of 'struct ifnet' and into a new 'struct if_snd_tag_sw'. A pointer to this structure is added to the generic part of send tags and is initialized by m_snd_tag_init() (which now accepts a switch structure as a new argument in place of the type).
Previously, device driver ifnet methods switched on the type to call type-specific functions. Now, those type-specific functions are saved in the switch structure and invoked directly. In addition, this more gracefully permits multiple implementations of the same tag within a driver. In particular, NIC TLS for future Chelsio adapters will use a different implementation than the existing NIC TLS support for T6 adapters.
Reviewed by: gallatin, hselasky, kib (older version) Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D31572
show more ...
|
#
b633e08c |
| 16-Jun-2021 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
ibcore: Kernel space update based on Linux 5.7-rc1.
Overview:
This is the first stage of a RDMA stack upgrade introducing kernel changes only based on Linux 5.7-rc1.
This patch is based on about f
ibcore: Kernel space update based on Linux 5.7-rc1.
Overview:
This is the first stage of a RDMA stack upgrade introducing kernel changes only based on Linux 5.7-rc1.
This patch is based on about four main areas of work: - Update of the IB uobjects system: - The memory holding so-called AH, CQ, PD, SRQ and UCONTEXT objects is now managed by ibcore. This also require some changes in the kernel verbs API. The updated verbs changes are typically about initialize and deinitialize objects, and remove allocation and free of memory.
- Update of the uverbs IOCTL framework: - The parsing and handling of user-space commands has been completely refactored to integrate with the updated IB uobjects system.
- Various changes and updates to the generic uverbs interfaces in device drivers including the new uAPI surface.
- The mlx5_ib_devx.c in mlx5ib and related mlx5 core changes.
Dependencies:
- The mlx4ib driver code has been updated with the minimum changes needed.
- The mlx5ib driver code has been updated with the minimum changes needed including DV support.
Compatibility:
- All user-space facing APIs are backwards compatible after this change.
- All kernel-space facing RDMA APIs are backwards compatible after this change, with exception of ib_create_ah() and ib_destroy_ah() which takes a new flag.
- The "ib_device_ops" structure exist, but only contains the driver ID and some structure sizes.
Differences from Linux:
- Infiniband drivers must use the INIT_IB_DEVICE_OPS() macro to set the sizes needed for allocating various IB objects, when adding IB device instances.
Security:
- PRIV_NET_RAW is needed to use raw ethernet transmit features. - PRIV_DRIVER is needed to use other privileged operations.
Based on upstream Linux, Torvalds (5.7-rc1): 8632e9b5645bbc2331d21d892b0d6961c1a08429
MFC after: 1 week Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D31149 Sponsored by: NVIDIA Networking
show more ...
|
#
7c3eff94 |
| 21-Jun-2021 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
mlx5: Numa domain improvements.
Properly allocate all mlx5en(4) structures from correct numa domain.
While at it cleanup unused numa domain integers deriving from the Linux version of mlx5en(4).
M
mlx5: Numa domain improvements.
Properly allocate all mlx5en(4) structures from correct numa domain.
While at it cleanup unused numa domain integers deriving from the Linux version of mlx5en(4).
MFC after: 1 week Reviewed by: kib Sponsored by: Mellanox Technologies // NVIDIA Networking
show more ...
|
#
9dfa2148 |
| 16-Jun-2021 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
mlx5en: Allocate per-channel doorbells.
To avoid congestion on the same PCI memory register space when traffic consists mostly of small packets.
MFC after: 1 week Reviewed by: kib Sponsored by: Mel
mlx5en: Allocate per-channel doorbells.
To avoid congestion on the same PCI memory register space when traffic consists mostly of small packets.
MFC after: 1 week Reviewed by: kib Sponsored by: Mellanox Technologies // NVIDIA Networking
show more ...
|
Revision tags: release/13.0.0 |
|
#
b8051298 |
| 26-Jan-2021 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
Fix missing value in uar_page field for ratelimit in mlx5en(4). This is a regression issue after the new UAR API was introduced by f8f5b459d21e .
MFC after: 1 week Sponsored by: Mellanox Technologie
Fix missing value in uar_page field for ratelimit in mlx5en(4). This is a regression issue after the new UAR API was introduced by f8f5b459d21e .
MFC after: 1 week Sponsored by: Mellanox Technologies // NVIDIA Networking
show more ...
|
#
43198046 |
| 08-Jan-2021 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
Don't offset the UAR map twice in mlx5en(4).
The new UAR API already offsets the UAR map pointer the mlx5en(4) is using. While at it remove some no longer needed variables for keeping track of the c
Don't offset the UAR map twice in mlx5en(4).
The new UAR API already offsets the UAR map pointer the mlx5en(4) is using. While at it remove some no longer needed variables for keeping track of the current BF offset.
This fixes a regression issue after the new UAR allocation APIs were introduced.
MFC after: 1 week Sponsored by: Mellanox Technologies // NVIDIA Networking
show more ...
|
#
f8f5b459 |
| 08-Jan-2021 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
Update user access region, UAR, APIs in the core in mlx5core.
This change include several changes as listed below all related to UAR. UAR is a special PCI memory area where the so-called doorbell re
Update user access region, UAR, APIs in the core in mlx5core.
This change include several changes as listed below all related to UAR. UAR is a special PCI memory area where the so-called doorbell register and blue flame register live. Blue flame is a feature for sending small packets more efficiently via a PCI memory page, instead of using PCI DMA.
- All structures and functions named xxx_uuars were renamed into xxx_bfreg. - Remove partially implemented Blueflame support from mlx5en(4) and mlx5ib. - Implement blue flame register allocator. - Use blue flame register allocator in mlx5ib. - A common UAR page is now allocated by the core to support doorbell register writes for all of mlx5en and mlx5ib, instead of allocating one UAR per sendqueue. - Add support for DEVX query UAR. - Add support for 4K UAR for libmlx5.
Linux commits: 7c043e908a74ae0a935037cdd984d0cb89b2b970 2f5ff26478adaff5ed9b7ad4079d6a710b5f27e7 0b80c14f009758cefeed0edff4f9141957964211 30aa60b3bd12bd79b5324b7b595bd3446ab24b52 5fe9dec0d045437e48f112b8fa705197bd7bc3c0 0118717583cda6f4f36092853ad0345e8150b286 a6d51b68611e98f05042ada662aed5dbe3279c1e
MFC after: 1 week Sponsored by: Mellanox Technologies // NVIDIA Networking
show more ...
|
#
f34f0a65 |
| 16-Nov-2020 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
Report EQE data upon CQ completion in mlx5core.
Report EQE data upon CQ completion to let upper layers use this data.
Linux commit: 4e0e2ea1886afe8c001971ff767f6670312a9b04
MFC after: 1 week Spons
Report EQE data upon CQ completion in mlx5core.
Report EQE data upon CQ completion to let upper layers use this data.
Linux commit: 4e0e2ea1886afe8c001971ff767f6670312a9b04
MFC after: 1 week Sponsored by: Mellanox Technologies // NVIDIA Networking
show more ...
|
Revision tags: release/12.2.0 |
|
#
56fb710f |
| 06-Oct-2020 |
John Baldwin <jhb@FreeBSD.org> |
Store the send tag type in the common send tag header.
Both cxgbe(4) and mlx5(4) wrapped the existing send tag header with their own identical headers that stored the type that the type-specific tag
Store the send tag type in the common send tag header.
Both cxgbe(4) and mlx5(4) wrapped the existing send tag header with their own identical headers that stored the type that the type-specific tag structures inherited from, so in practice it seems drivers need this in the tag anyway. This permits removing these extra header indirections (struct cxgbe_snd_tag and struct mlx5e_snd_tag).
In addition, this permits driver-independent code to query the type of a tag, e.g. to know what type of tag is being queried via if_snd_query.
Reviewed by: gallatin, hselasky, np, kib Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D26689
show more ...
|
Revision tags: release/11.4.0 |
|
#
75dfc66c |
| 27-Feb-2020 |
Dimitry Andric <dim@FreeBSD.org> |
Merge ^/head r358269 through r358399.
|
#
7029da5c |
| 26-Feb-2020 |
Pawel Biernacki <kaktus@FreeBSD.org> |
Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are still not MPSAFE (or already are but aren’t properly mark
Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are still not MPSAFE (or already are but aren’t properly marked). Use it in preparation for a general review of all nodes.
This is non-functional change that adds annotations to SYSCTL_NODE and SYSCTL_PROC nodes using one of the soon-to-be-required flags.
Mark all obvious cases as MPSAFE. All entries that haven't been marked as MPSAFE before are by default marked as NEEDGIANT
Approved by: kib (mentor, blanket) Commented by: kib, gallatin, melifaro Differential Revision: https://reviews.freebsd.org/D23718
show more ...
|
#
7272f9cd |
| 06-Dec-2019 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
Implement hardware TLS via send tags for mlx5en(4), which is supported by ConnectX-6 DX.
Currently TLS v1.2 and v1.3 with AES 128/256 crypto over TCP/IP (v4 and v6) is supported.
A per PCI device U
Implement hardware TLS via send tags for mlx5en(4), which is supported by ConnectX-6 DX.
Currently TLS v1.2 and v1.3 with AES 128/256 crypto over TCP/IP (v4 and v6) is supported.
A per PCI device UMA zone is used to manage the memory of the send tags. To optimize performance some crypto contexts may be cached by the UMA zone, until the UMA zone finishes the memory of the given send tag.
An asynchronous task is used manage setup of the send tags towards the firmware. Most importantly setting the AES 128/256 bit pre-shared keys for the crypto context.
Updating the state of the AES crypto engine and encrypting data, is all done in the fast path. Each send tag tracks the TCP sequence number in order to detect non-contiguous blocks of data, which may require a dump of prior unencrypted data, to restore the crypto state prior to wire transmission.
Statistics counters have been added to count the amount of TLS data transmitted in total, and the amount of TLS data which has been dumped prior to transmission. When non-contiguous TCP sequence numbers are detected, the software needs to dump the beginning of the current TLS record up until the point of retransmission. All TLS counters utilize the counter(9) API.
In order to enable hardware TLS offload the following sysctls must be set: kern.ipc.mb_use_ext_pgs=1 kern.ipc.tls.ifnet.permitted=1 kern.ipc.tls.enable=1
Sponsored by: Mellanox Technologies
show more ...
|
Revision tags: release/12.1.0 |
|
#
8b3bc70a |
| 08-Oct-2019 |
Dimitry Andric <dim@FreeBSD.org> |
Merge ^/head r352764 through r353315.
|
#
6b4040d8 |
| 02-Oct-2019 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
Unify prints in mlx5en(4).
All prints in mlx5en(4) should use on of the macros: mlx5_en_err/dbg/warn
Submitted by: slavash@ MFC after: 3 days Sponsored by: Mellanox Technologies
|
Revision tags: release/11.3.0 |
|
#
0269ae4c |
| 06-Jun-2019 |
Alan Somers <asomers@FreeBSD.org> |
MFHead @348740
Sponsored by: The FreeBSD Foundation
|
#
76a35558 |
| 04-Jun-2019 |
Hans Petter Selasky <hselasky@FreeBSD.org> |
Make sure the DMA tags get freed in mlx5en(4).
MFC after: 3 days Sponsored by: Mellanox Technologies
|
#
fb3bc596 |
| 25-May-2019 |
John Baldwin <jhb@FreeBSD.org> |
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated for a different ifp than the one the packet is being output on), i
Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated for a different ifp than the one the packet is being output on), in ip_output() and ip6_output(). This avoids sending packets with send tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking if_output, we can now try to allocate a new tag before invoking if_output sending the original packet on the new tag if allocation succeeds.
To avoid code duplication for the fragment and unfragmented cases, add ip_output_send() and ip6_output_send() as wrappers around if_output and nd6_output_ifp, respectively. All of the logic for setting send tags and dealing with send tag-related errors is done in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and lagg), wrapper send tags are now allocated so that ip*_output see the wrapper ifp as the ifp in the send tag. The if_transmit routines rewrite the send tags after performing an ifp mismatch check. If an ifp mismatch is detected, the transmit routines fail with EAGAIN.
- To provide clearer life cycle management of send tags, especially in the presence of vlan and lagg wrapper tags, add a reference count to send tags managed via m_snd_tag_ref() and m_snd_tag_rele(). Provide a helper function (m_snd_tag_init()) for use by drivers supporting send tags. m_snd_tag_init() takes care of the if_ref on the ifp meaning that code alloating send tags via if_snd_tag_alloc no longer has to manage that manually. Similarly, m_snd_tag_rele drops the refcount on the ifp after invoking if_snd_tag_free when the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in csum_flags to indicate 'snd_tag' is set (rather than 'rcvif'). Drivers now also check this flag instead of checking snd_tag against NULL. This avoids false positive matches when a forwarded packet has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was detached so that it could kick the firmware to flush any pending work on the flow. This is because the driver doesn't require ACK messages from the firmware for every request, but instead does a kind of manual interrupt coalescing by only setting a flag to request a completion on a subset of requests. If all of the in-flight requests don't have the flag when the tag is detached from the inp, the flow might never return the credits. The current snd_tag_free command issues a flush command to force the credits to return. However, the credit return is what also frees the mbufs, and since those mbufs now hold references on the tag, this meant that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the mbuf is queued in the firmware work queue. This means that once the inp's reference on the tag goes away and all in-flight mbufs have been queued to the firmware, tag's refcount will drop to zero and snd_tag_free will kick in and send the flush request. Note that we need to avoid doing this in the middle of ethofld_tx(), so the driver grabs a temporary reference on the tag around that loop to defer the free to the end of the function in case it sends the last mbuf to the queue after the inp has dropped its reference on the tag.
- mlx5 preallocates send tags and was using the ifp pointer even when the send tag wasn't in use. Explicitly use the ifp from other data structures instead.
- Sprinkle some assertions in various places to assert that received packets don't have a send tag, and that other places that overwrite rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D20117
show more ...
|
#
7648bc9f |
| 13-May-2019 |
Alan Somers <asomers@FreeBSD.org> |
MFHead @347527
Sponsored by: The FreeBSD Foundation
|