#
8eba75ed |
| 03-Dec-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): Stop but don't free netmap queues when netmap is switched off.
It is common for freelists to be starving when a netmap application stops. Mailbox commands to free queues can hang in such
cxgbe(4): Stop but don't free netmap queues when netmap is switched off.
It is common for freelists to be starving when a netmap application stops. Mailbox commands to free queues can hang in such a situation. Avoid that by not freeing the queues when netmap is switched off. Instead, use an alternate method to stop the queues without releasing the context ids. If netmap is enabled again later then the same queue is reinitialized for use. Move alloc_nm_rxq and txq to t4_netmap.c while here.
MFC after: 1 week Sponsored by: Chelsio Communications
show more ...
|
Revision tags: release/12.2.0 |
|
#
b20b25e7 |
| 22-Oct-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): fix the size of the iq/eq maps.
The firmware can allocate ingress and egress context ids anywhere from its configured range. Size the iq/eq maps to match the entire range instead of assum
cxgbe(4): fix the size of the iq/eq maps.
The firmware can allocate ingress and egress context ids anywhere from its configured range. Size the iq/eq maps to match the entire range instead of assuming that the firmware always allocates the first available context id.
Reported by: Baptiste Wicht @ Verisign MFC after: 1 week Sponsored by: Chelsio Communications
show more ...
|
#
56fb710f |
| 06-Oct-2020 |
John Baldwin <jhb@FreeBSD.org> |
Store the send tag type in the common send tag header.
Both cxgbe(4) and mlx5(4) wrapped the existing send tag header with their own identical headers that stored the type that the type-specific tag
Store the send tag type in the common send tag header.
Both cxgbe(4) and mlx5(4) wrapped the existing send tag header with their own identical headers that stored the type that the type-specific tag structures inherited from, so in practice it seems drivers need this in the tag anyway. This permits removing these extra header indirections (struct cxgbe_snd_tag and struct mlx5e_snd_tag).
In addition, this permits driver-independent code to query the type of a tag, e.g. to know what type of tag is being queried via if_snd_query.
Reviewed by: gallatin, hselasky, np, kib Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D26689
show more ...
|
#
8741306b |
| 06-Oct-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4) sysctls do not need Giant.
Sponsored by: Chelsio Communications
|
#
7676c62a |
| 02-Oct-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): validate largest_rx_cluster and safest_rx_cluster.
These tunables can only be set to a valid cluster size (2K, 4K, 9K, or 16K) as documented in the man page. Anything else could lead to a
cxgbe(4): validate largest_rx_cluster and safest_rx_cluster.
These tunables can only be set to a valid cluster size (2K, 4K, 9K, or 16K) as documented in the man page. Anything else could lead to a panic on interface up.
Reported by: mav@ MFC after: 1 week Sponsored by: Chelsio Communications
show more ...
|
#
30e3f2b4 |
| 22-Sep-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): let the PF driver use VM work requests for transmit.
This allows the PF interfaces to communicate with the VF interfaces over the internal switch in the ASIC. Fix the GL limits for VM wor
cxgbe(4): let the PF driver use VM work requests for transmit.
This allows the PF interfaces to communicate with the VF interfaces over the internal switch in the ASIC. Fix the GL limits for VM work requests while here.
MFC after: 3 days Sponsored by: Chelsio Communications
show more ...
|
#
7054f6ec |
| 22-Sep-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): add counters for mbuf pullups and defrags.
MFC after: 3 days Sponsored by: Chelsio Communications
|
#
a4a4ad2d |
| 18-Sep-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): add support for stateless offloads for VXLAN traffic.
Hardware assistance includes checksumming (tx and rx), TSO, and RSS on the inner traffic in a VXLAN tunnel.
Relnotes: Yes Sponsored b
cxgbe(4): add support for stateless offloads for VXLAN traffic.
Hardware assistance includes checksumming (tx and rx), TSO, and RSS on the inner traffic in a VXLAN tunnel.
Relnotes: Yes Sponsored by: Chelsio Communications
show more ...
|
#
565b8fce |
| 01-Sep-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): Check for descriptors before writing a TLS or raw work request.
This fixes a regression in r362905.
Submitted by: jhb@ Sponsored by: Chelsio Communications
|
#
e2515283 |
| 27-Aug-2020 |
Glen Barber <gjb@FreeBSD.org> |
MFH
Sponsored by: Rubicon Communications, LLC (netgate.com)
|
#
6a59b994 |
| 23-Aug-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): Use large clusters for TOE rx queues when TOE+TLS is enabled.
Rx is more efficient within the chip when the receive buffer size matches the TLS PDU size.
MFC after: 3 days Sponsored by: C
cxgbe(4): Use large clusters for TOE rx queues when TOE+TLS is enabled.
Rx is more efficient within the chip when the receive buffer size matches the TLS PDU size.
MFC after: 3 days Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D26127
show more ...
|
#
800535c2 |
| 13-Jul-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbev(4): Compare at most 16 bytes of the Ethernet header when trying to coalesce tx work requests.
Note that Coverity will still treat this as an out-of-bounds access. We do want to compare 16B s
cxgbev(4): Compare at most 16 bytes of the Ethernet header when trying to coalesce tx work requests.
Note that Coverity will still treat this as an out-of-bounds access. We do want to compare 16B starting from ethmacdst but cmp_l2hdr was was going beyond that by 2B.
cmp_l2hdr was introduced in r362905.
Reported by: Coverity (CID 1430284) Sponsored by: Chelsio Communications
show more ...
|
#
3bbb68f0 |
| 05-Jul-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): Fix a bug (introduced in r362905) where some tx traffic wasn't being reported to BPF.
|
#
d735920d |
| 03-Jul-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): changes in the Tx path to help increase tx coalescing.
- Ask the firmware for the number of frames that can be stuffed in one work request.
- Modify mp_ring to increase the likelihood o
cxgbe(4): changes in the Tx path to help increase tx coalescing.
- Ask the firmware for the number of frames that can be stuffed in one work request.
- Modify mp_ring to increase the likelihood of tx coalescing when there are just one or two threads that are doing most of the tx. Add teeth to the abdication mechanism by pushing the consumer lock into mp_ring. This reduces the likelihood that a consumer will get stuck with all the work even though it is above its budget.
- Add support for coalesced tx WR to the VF driver. This, with the changes above, results in a 7x improvement in the tx pps of the VF driver for some common cases. The firmware vets the L2 headers submitted by the VF driver and it's a big win if the checks are performed for a batch of packets and not each one individually.
Reviewed by: jhb@ MFC after: 2 weeks Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D25454
show more ...
|
#
7c228be3 |
| 25-Jun-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): Add a pointer to the adapter softc in vi_info.
There were quite a few places where port_info was being accessed only to get to the adapter.
Reviewed by: jhb@ MFC after: 1 week Sponsored b
cxgbe(4): Add a pointer to the adapter softc in vi_info.
There were quite a few places where port_info was being accessed only to get to the adapter.
Reviewed by: jhb@ MFC after: 1 week Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D25432
show more ...
|
#
0cadedfc |
| 23-Jun-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): Add a tx_len16_to_desc helper.
No functional change.
MFC after: 1 week Sponsored by: Chelsio Communications
|
Revision tags: release/11.4.0 |
|
#
365e8da4 |
| 03-May-2020 |
Gleb Smirnoff <glebius@FreeBSD.org> |
Mechanically rename MBUF_EXT_PGS_ASSERT() to M_ASSERTEXTPG() to match classical M_ASSERTPKTHDR.
Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D24598
|
#
6edfd179 |
| 03-May-2020 |
Gleb Smirnoff <glebius@FreeBSD.org> |
Step 4.1: mechanically rename M_NOMAP to M_EXTPG
Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D24598
|
#
7b6c99d0 |
| 03-May-2020 |
Gleb Smirnoff <glebius@FreeBSD.org> |
Step 3: anonymize struct mbuf_ext_pgs and move all its fields into mbuf within m_epg namespace. All edits except the 'struct mbuf' declaration and mb_dupcl() were done mechanically with sed:
Step 3: anonymize struct mbuf_ext_pgs and move all its fields into mbuf within m_epg namespace. All edits except the 'struct mbuf' declaration and mb_dupcl() were done mechanically with sed:
s/->m_ext_pgs.nrdy/->m_epg_nrdy/g s/->m_ext_pgs.hdr_len/->m_epg_hdrlen/g s/->m_ext_pgs.trail_len/->m_epg_trllen/g s/->m_ext_pgs.first_pg_off/->m_epg_1st_off/g s/->m_ext_pgs.last_pg_len/->m_epg_last_len/g s/->m_ext_pgs.flags/->m_epg_flags/g s/->m_ext_pgs.record_type/->m_epg_record_type/g s/->m_ext_pgs.enc_cnt/->m_epg_enc_cnt/g s/->m_ext_pgs.tls/->m_epg_tls/g s/->m_ext_pgs.so/->m_epg_so/g s/->m_ext_pgs.seqno/->m_epg_seqno/g s/->m_ext_pgs.stailq/->m_epg_stailq/g
Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D24598
show more ...
|
#
c4ee38f8 |
| 03-May-2020 |
Gleb Smirnoff <glebius@FreeBSD.org> |
Step 2.3: Rename mbuf_ext_pg_len() to m_epg_pagelen() that uses mbuf argument.
Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D24598
|
#
49b6b60e |
| 03-May-2020 |
Gleb Smirnoff <glebius@FreeBSD.org> |
Step 2.2: o Shrink sglist(9) functions to work with multipage mbufs down from four functions to two. o Don't use 'struct mbuf_ext_pgs *' as argument, use struct mbuf. o Rename to something matching
Step 2.2: o Shrink sglist(9) functions to work with multipage mbufs down from four functions to two. o Don't use 'struct mbuf_ext_pgs *' as argument, use struct mbuf. o Rename to something matching _epg.
Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D24598
show more ...
|
#
0c103266 |
| 03-May-2020 |
Gleb Smirnoff <glebius@FreeBSD.org> |
Continuation of multi page mbuf redesign from r359919.
The following series of patches addresses three things:
Now that array of pages is embedded into mbuf, we no longer need separate structure to
Continuation of multi page mbuf redesign from r359919.
The following series of patches addresses three things:
Now that array of pages is embedded into mbuf, we no longer need separate structure to pass around, so struct mbuf_ext_pgs is an artifact of the first implementation. And struct mbuf_ext_pgs_data is a crutch to accomodate the main idea r359919 with minimal churn.
Also, M_EXT of type EXT_PGS are just a synonym of M_NOMAP.
The namespace for the newfeature is somewhat inconsistent and sometimes has a lengthy prefixes. In these patches we will gradually bring the namespace to "m_epg" prefix for all mbuf fields and most functions.
Step 1 of 4:
o Anonymize mbuf_ext_pgs_data, embed in m_ext o Embed mbuf_ext_pgs o Start documenting all this entanglement
Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D24598
show more ...
|
#
23feb563 |
| 14-Apr-2020 |
Andrew Gallatin <gallatin@FreeBSD.org> |
KTLS: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself.
While the original implementation of unmapped mbufs was a large step forward in terms of reducing cache misses by enabling mbufs to
KTLS: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself.
While the original implementation of unmapped mbufs was a large step forward in terms of reducing cache misses by enabling mbufs to carry more than a single page for sendfile, they are rather cache unfriendly when accessing the ext_pgs metadata and data. This is because the ext_pgs part of the mbuf is allocated separately, and almost guaranteed to be cold in cache.
This change takes advantage of the fact that unmapped mbufs are never used at the same time as pkthdr mbufs. Given this fact, we can overlap the ext_pgs metadata with the mbuf pkthdr, and carry the ext_pgs meta directly in the mbuf itself. Similarly, we can carry the ext_pgs data (TLS hdr/trailer/array of pages) directly after the existing m_ext.
In order to be able to carry 5 pages (which is the minimum required for a 16K TLS record which is not perfectly aligned) on LP64, I've had to steal ext_arg2. The only user of this in the xmit path is sendfile, and I've adjusted it to use arg1 when using unmapped mbufs.
This change is almost entirely mechanical, except that we change mb_alloc_ext_pgs() to no longer allow allocating pkthdrs, the change to avoid ext_arg2 as mentioned above, and the removal of the ext_pgs zone,
This change saves roughly 2% "raw" CPU (~59% -> 57%), or over 3% "scaled" CPU on a Netflix 100% software kTLS workload at 90+ Gb/s on Broadwell Xeons.
In a follow-on commit, I plan to remove some hacks to avoid access ext_pgs fields of mbufs, since they will now be in cache.
Many thanks to glebius for helping to make this better in the Netflix tree.
Reviewed by: hselasky, jhb, rrs, glebius (early version) Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D24213
show more ...
|
#
aa301e5f |
| 20-Mar-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): Split sge_nm_rxq into three cachelines.
This reduces the lines bouncing around between the driver rx ithread and the netmap rxsync thread. There is no net change in the size of the struct
cxgbe(4): Split sge_nm_rxq into three cachelines.
This reduces the lines bouncing around between the driver rx ithread and the netmap rxsync thread. There is no net change in the size of the struct (it continues to waste a lot of space).
This kind of split was originally proposed in D17869 by Marc De La Gueronniere @ Verisign, Inc.
MFC after: 1 week Sponsored by: Chelsio Communications
show more ...
|
#
2b9010f0 |
| 10-Mar-2020 |
Navdeep Parhar <np@FreeBSD.org> |
cxgbe(4): Do not try to use 0 as an rx buffer address when the driver is already allocating from the safe zone and the allocation fails.
This bug was introduced in r357481.
MFC after: 3 days Sponso
cxgbe(4): Do not try to use 0 as an rx buffer address when the driver is already allocating from the safe zone and the allocation fails.
This bug was introduced in r357481.
MFC after: 3 days Sponsored by: Chelsio Communications
show more ...
|