nvmf_che: NVMe-TCP offload support for Chelsio T7 adapters

This provides an alternative NVMe over TCP transport which uses PDU
offload for TOE connections on a T7.

Similar to iSCSI offload via cxgb

nvmf_che: NVMe-TCP offload support for Chelsio T7 adapters

This provides an alternative NVMe over TCP transport which uses PDU
offload for TOE connections on a T7.

Similar to iSCSI offload via cxgbei.ko, nvmf_che uses DDP when
possible to enable the NIC to DMA received data directly into I/O data
buffers (pages from a struct bio on the host side, pages from a CTL
I/O requests on the controller side) to avoid copying data on the host
CPU. nvmf_che is also able to receive a stream of C2H or H2C PDUs for
a single data transfer when using DDP without processing the header of
each PDU.

Unlike cxgbei, nvmf_che aims to be mostly transparent to end users.
Notably, neither nvmecontrol or ctld have to be explicitly asked to
use an offload. Instead, TCP queue pairs are claimed by this driver
whenever they are eligible (e.g., using TOE).

The main restriction of nvmf_che compared to the software TCP
transport is that Chelsio adapters have a restriction on the largest
PDU that can be sent and received. When sending data, nvmf_che is
able to split large C2H or H2C data requests across multiple PDUs
without affecting nvmf(4) or nvmft(4).

To avoid overly large PDUs when using nvmf(4), nvmf_che reports a data
transfer limit that is honored by nvmf(4). This ensures that the
remote controller's PDUs will never be too large (since the command
transfer size is limited to one PDU) and also ensures that nvmf(4)
will never to try to send a command PDU with ICD that is too large.

For nvmft(4), overly large command PDUs due to ICD are avoided by
clamping the size of the reported IOCCSZ in the controller data.
However, to ensure that H2C PDUs are sufficiently small, nvmf_che will
only claim queue pairs which advertised a suitable MAXH2CDATA
parameter during queue negotiation. For ctld(8), this can be achieved
by setting the MAXH2CDATA option in a transport-group, e.g. for T7:

transport-group tg0 {
discovery-auth-group no-authentication
listen tcp 0.0.0.0
listen tcp [::]
listen discovery-tcp 0.0.0.0
listen discovery-tcp [::]
option MAXH2CDATA 32488
}

Sponsored by: Chelsio Communications

show more ...

cxgbe(4): hw/fw headers and shared code for the Terminator 7 ASIC

This is the first of a series of commits that will add T7 support to
cxgbe. The ASIC is gen5x16 on the PCIe side and has a 400Gbps

cxgbe(4): hw/fw headers and shared code for the Terminator 7 ASIC

This is the first of a series of commits that will add T7 support to
cxgbe. The ASIC is gen5x16 on the PCIe side and has a 400Gbps MAC on
the Ethernet side. NICs using the T7 will come in the following
variants:

* 1 x 400Gbps with QSFP-DD connector
* 2 x 200/100/40Gbps with QSFP56/QSFP28/QSFP+ connectors
* 4 x 50/25/10/1Gbps with SFP28/SFP+/SFP connectors

There are 8 general purpose ARM A72 cores available on select
SmartNIC/DPU boards.

Obtained from: Chelsio Communications
MFC after: 3 days
Sponsored by: Chelsio Communications

show more ...

sys: Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

cxgbe(4): Enable the hardware TCP Offload Module (t4_tom) on aarch64.

MFC after: 3 days
Sponsored by: Chelsio Communications

Merge ^/head r318380 through r318559.

Add a driver for the Chelsio T6 crypto accelerator engine.

The ccr(4) driver supports use of the crypto accelerator engine on
Chelsio T6 NICs in "lookaside" mode via the opencrypto framework.

Curre

Add a driver for the Chelsio T6 crypto accelerator engine.

The ccr(4) driver supports use of the crypto accelerator engine on
Chelsio T6 NICs in "lookaside" mode via the opencrypto framework.

Currently, the driver supports AES-CBC, AES-CTR, AES-GCM, and AES-XTS
cipher algorithms as well as the SHA1-HMAC, SHA2-256-HMAC, SHA2-384-HMAC,
and SHA2-512-HMAC authentication algorithms. The driver also supports
chaining one of AES-CBC, AES-CTR, or AES-XTS with an authentication
algorithm for encrypt-then-authenticate operations.

Note that this driver is still under active development and testing and
may not yet be ready for production use. It does pass the tests in
tests/sys/opencrypto with the exception that the AES-GCM implementation
in the driver does not yet support requests with a zero byte payload.

To use this driver currently, the "uwire" configuration must be used
along with explicitly enabling support for lookaside crypto capabilities
in the cxgbe(4) driver. These can be done by setting the following
tunables before loading the cxgbe(4) driver:

hw.cxgbe.config_file=uwire
hw.cxgbe.cryptocaps_allowed=-1

MFC after: 1 month
Relnotes: yes
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D10763

show more ...

sys/modules: normalize .CURDIR-relative paths to SRCTOP

This simplifies make output/logic

Tested with: `cd sys/modules; make ALL_MODULES=` on amd64
MFC after: 1 month
Sponsored by: Dell EMC Isilon

Merge ^/head r309263 through r309518.

cxgbe(4): Include firmware for T6 cards in the driver. Update all
firmwares to 1.16.12.0.

Obtained from: Chelsio Communications
MFC after: 3 days
Sponsored by: Chelsio Communications

Merge ^/head r305687 through r305890.

cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will
come up as 't6nex' nexus devices with 'cc' ports hanging off them.

The T6 firmware and configuration files will be added as soon

cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will
come up as 't6nex' nexus devices with 'cc' ports hanging off them.

The T6 firmware and configuration files will be added as soon as they
are released. For now the driver will try to work with whatever
firmware and configuration is on the card's flash.

Sponsored by: Chelsio Communications

show more ...

Merge ^/head r305431 through r305622.

Chelsio T4/T5 VF driver.

The cxgbev/cxlv driver supports Virtual Function devices for Chelsio
T4 and T4 adapters. The VF devices share most of their code with the
existing PF4 driver (cxgbe/cxl) an

Chelsio T4/T5 VF driver.

The cxgbev/cxlv driver supports Virtual Function devices for Chelsio
T4 and T4 adapters. The VF devices share most of their code with the
existing PF4 driver (cxgbe/cxl) and as such the VF device driver
currently depends on the PF4 driver.

Similar to the cxgbe/cxl drivers, the VF driver includes a t4vf/t5vf
PCI device driver that attaches to the VF device. It then creates
child cxgbev/cxlv devices representing ports assigned to the VF.
By default, the PF driver assigns a single port to each VF.

t4vf_hw.c contains VF-specific routines from the shared code used to
fetch VF-specific parameters from the firmware.

t4_vf.c contains the VF-specific PCI device driver and includes its
own attach routine.

VF devices are required to use a different firmware request when
transmitting packets (which in turn requires a different CPL message
to encapsulate messages). This alternate firmware request does not
permit chaining multiple packets in a single message, so each packet
results in a firmware request. In addition, the different CPL message
requires more detailed information when enabling hardware checksums,
so parse_pkt() on VF devices must examine L2 and L3 headers for all
packets (not just TSO packets) for VF devices. Finally, L2 checksums
on non-UDP/non-TCP packets do not work reliably (the firmware trashes
the IPv4 fragment field), so IPv4 checksums for such packets are
calculated in software.

Most of the other changes in the non-VF-specific code are to expose
various variables and functions private to the PF driver so that they
can be used by the VF driver.

Note that a limited subset of cxgbetool functions are supported on VF
devices including register dumps, scheduler classes, and clearing of
statistics. In addition, TOE is not supported on VF devices, only for
the PF interfaces.

Reviewed by: np
MFC after: 2 months
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7599

show more ...

MFH r289384-r293170

Sponsored by: The FreeBSD Foundation

Mfh r292839

cxgbei: Hardware accelerated iSCSI target and initiator for TOE capable
cards supported by cxgbe(4).

On the host side this driver interfaces with the storage stack via the
ICL (iSCSI Common Layer) i

cxgbei: Hardware accelerated iSCSI target and initiator for TOE capable
cards supported by cxgbe(4).

On the host side this driver interfaces with the storage stack via the
ICL (iSCSI Common Layer) in the kernel. On the wire the traffic is
standard iSCSI (SCSI over TCP as per RFC 3720/7143 etc.) that
interoperates with all other standards compliant implementations. The
driver is layered on top of the TOE driver (t4_tom) and promotes
connections being handled by t4_tom to iSCSI ULP (Upper Layer Protocol)
mode. Hardware assistance in this mode includes:

- Full TCP processing.
- iSCSI PDU identification and recovery within the TCP stream.
- Header and/or data digest insertion (tx) and verification (rx).
- Zero copy (both tx and rx).

Man page will follow in a separate commit in a couple of weeks.

Relnotes: Yes
Sponsored by: Chelsio Communications

show more ...

Merge sync of head

Improved compliance with style.Makefile(5).

MFH: r278202,r278205-r278590

Sponsored by: The FreeBSD Foundation

Initial drop of the hardare accelerated iSCSI driver.

Submitted by: Sreenivasa Honnur <shonnur at chelsio dot com>
Sponsored by: Chelsio Communications

Merge from HEAD

Merging ^/head r278298 through r278350.

cxgbe(4): Add a minimal if_cxl module that pulls in the real driver as
a dependency. This ensures "ifconfig cxl<n> ..." does the right thing
even when it's run with no driver loaded.

if_cxl.ko is t

cxgbe(4): Add a minimal if_cxl module that pulls in the real driver as
a dependency. This ensures "ifconfig cxl<n> ..." does the right thing
even when it's run with no driver loaded.

if_cxl.ko is the tiniest module in /boot/kernel.

MFC after: 2 weeks

show more ...

Merge from head@274682

IFC @r269962

Submitted by: Anish Gupta (akgupt3@gmail.com)