cxgbe(4): Consolidate all mk_set_tcb_field_ulp in one place.

MFC after: 1 week
Sponsored by: Chelsio Communications

cxgbe tom: Handle a race condition when enabling TLS offload

Use a separate state for when a request to set RX_QUIESCE has been
sent but the resulting TCB reply has not been received. In
particular

cxgbe tom: Handle a race condition when enabling TLS offload

Use a separate state for when a request to set RX_QUIESCE has been
sent but the resulting TCB reply has not been received. In
particular, this correctly handles the case where data has been
received and queued in the receive queue before the quiesce request
takes effect.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44435

show more ...

cxgbe t4_tls: Call t4_rcvd_locked from do_rx_tls_cmp

Similar to dcfddc8dc091e7688abc8488a0307eba425fa7a2, replace the
simpler, inlined version with the full version.

Sponsored by: Chelsio Communica

cxgbe t4_tls: Call t4_rcvd_locked from do_rx_tls_cmp

Similar to dcfddc8dc091e7688abc8488a0307eba425fa7a2, replace the
simpler, inlined version with the full version.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D41690

show more ...

cxgbe t4_tls: Don't bother returning RX credits for a protocol receive error

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D41689

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

cxgbe: Enable TOE TLS RX when an RX key is provided via setsockopt().

Rather than requiring a socket to be created as a TLS socket from the
get go, switch a TOE socket from "plain" TOE to TLS mode w

cxgbe: Enable TOE TLS RX when an RX key is provided via setsockopt().

Rather than requiring a socket to be created as a TLS socket from the
get go, switch a TOE socket from "plain" TOE to TLS mode when a
receive key is added to the socket.

The firmware is only able to switch a "plain" TOE connection to TLS
mode if the head of the pending socket data is the start of a TLS
record, so the connection is migrated to TLS mode as a multi-step
process.

When TOE TLS RX is enabled, the associated connection's receive side
is frozen via a flag in the TCB. The state of the socket buffer is
then examined to determine if the pending data in the socket buffer
ends on a TLS record boundary. If so, the connection is migrated to
TLS mode and unfrozen. Otherwise, the connection is unfrozen
temporarily until more data arrives. Once more data arrives, the
receive queue is frozen again and rechecked. This continues until the
connection is paused at a record boundary. Any records received
before TLS mode is enabled are decrypted as software records.

Note that this removes the 'rx_tls_ports' sysctl. TOE TLS offload for
receive is now enabled automatically on existing TOE connections when
using a KTLS-aware SSL library just as it was previously enabled
automatically for TLS transmit. This also enables TLS offload for TOE
connections which enable TLS after passing initial data in the clear
(e.g. STARTTLS with SMTP).

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37351

show more ...

cxgbe: Various whitespace fixes.

Mostly trailing whitespace and spaces before tabs.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37350

tcp: remove INP_TIMEWAIT flag

Mechanically cleanup INP_TIMEWAIT from the kernel sources. After
0d7445193ab, this commit shall not cause any functional changes.

Note: this flag was very often check

tcp: remove INP_TIMEWAIT flag

Mechanically cleanup INP_TIMEWAIT from the kernel sources. After
0d7445193ab, this commit shall not cause any functional changes.

Note: this flag was very often checked together with INP_DROPPED.
If we modify in_pcblookup*() not to return INP_DROPPED pcbs, we
will be able to remove most of this checks and turn them to
assertions. Some of them can be turned into assertions right now,
but that should be carefully done on a case by case basis.

Differential revision: https://reviews.freebsd.org/D36400

show more ...

cxgbe TOE TLS: Fix handling of unusual record types.

This doesn't matter for real traffic but fixes failures in the KTLS
unit tests that use unusual record types.

Sponsored by: Chelsio Communicatio

cxgbe TOE TLS: Fix handling of unusual record types.

This doesn't matter for real traffic but fixes failures in the KTLS
unit tests that use unusual record types.

Sponsored by: Chelsio Communications

show more ...

cxgbe TOE TLS: Simplify a few routines.

Remove a few more remnants from the old pre-KTLS support and instead
assume that each work request sends a single TLS record.

Sponsored by: Chelsio Communica

cxgbe TOE TLS: Simplify a few routines.

Remove a few more remnants from the old pre-KTLS support and instead
assume that each work request sends a single TLS record.

Sponsored by: Chelsio Communications

show more ...

sockbuf: merge two versions of sbcreatecontrol() into one

No functional change.

sockets: fix build, convert missed sbreserve_locked() calls

Fixes: 4328318445ae

ktls: Zero out TLS_GET_RECORD control messages

Otherwise we end up copying one uninitialized byte into the socket
buffer.

Reported by: KMSAN
Reviewed by: jhb
MFC after: 1 week
Sponsored by: The Fre

ktls: Zero out TLS_GET_RECORD control messages

Otherwise we end up copying one uninitialized byte into the socket
buffer.

Reported by: KMSAN
Reviewed by: jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33953

show more ...

cxgbe(4): Fix "set but not used [-Wunused-but-set-variable]" warnings.

MFC after: 1 week
Sponsored by: Chelsio Communications

cxgbe tom: Remove orphaned function max_imm_tls_space().

Reported by: markj
Fixes: 789f2d4b3f33 cxgbe tom: Remove support for non-KTLS TLS offload.
Sponsored by: Chelsio Communications

cxgbe: De-duplicate some of the code for managing TLS key contexts.

The NIC TLS and TOE TLS modes in cxgbe(4) both work with TLS key
contexts. Previously, TOE TLS supported TLS key contexts created

cxgbe: De-duplicate some of the code for managing TLS key contexts.

The NIC TLS and TOE TLS modes in cxgbe(4) both work with TLS key
contexts. Previously, TOE TLS supported TLS key contexts created by
two different methods, and NIC TLS had a separate bit of code copied
from NIC TLS but specific to KTLS. Now that TOE TLS only supports
KTLS, pull common code for creating TLS key contexts and programming
them into on-card memory into t4_keyctx.c.

Sponsored by: Chelsio Communications

show more ...

cxgbe tom: Remove support for non-KTLS TLS offload.

TOE TLS offload was first supported via a customized OpenSSL developed
by Chelsio with proprietary socket options prior to KTLS being present
eith

cxgbe tom: Remove support for non-KTLS TLS offload.

TOE TLS offload was first supported via a customized OpenSSL developed
by Chelsio with proprietary socket options prior to KTLS being present
either in FreeBSD or upstream OpenSSL. With the addition of KTLS in
both places, cxgbe's TOE driver was extended to support TLS offload
via KTLS as well. This change removes the older interface leaving
only the KTLS bindings for TOE TLS.

Since KTLS was added to TOE TLS second, it was somehat shoe-horned
into the existing code. In addition to removing the non-KTLS TLS
offload, refactor and simplify the code to assume KTLS, e.g. not
copying keys into a helper structure that mimic'ed the non-KTLS mode,
but using the KTLS session object directly when constructing key
contexts.

This also removes some unused code to send TX keys inline in work
requests for TOE TLS. This code was never enabled, and was arguably
sending the wrong thing (it was not sending the raw key context as we
do for NIC TLS when using inline keys).

Sponsored by: Chelsio Communications

show more ...

cxgbe: Make the TOE TLS stats per-queue instead of per-port.

This avoids some atomics by using counter_u64 for TX and relying on
existing single-threading (single ithread per rxq) for RX.

Reviewed

cxgbe: Make the TOE TLS stats per-queue instead of per-port.

This avoids some atomics by using counter_u64 for TX and relying on
existing single-threading (single ithread per rxq) for RX.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29383

show more ...

cxgbe: Add a struct sge_ofld_txq type.

This type mirrors struct sge_ofld_rxq and holds state for TCP offload
transmit queues. Currently it only holds a work queue but will
include additional state

cxgbe: Add a struct sge_ofld_txq type.

This type mirrors struct sge_ofld_rxq and holds state for TCP offload
transmit queues. Currently it only holds a work queue but will
include additional state in future changes.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29382

show more ...

Don't try to adjust a TLS TOE socket that has been closed.

The handshake timer can race with another thread sending a FIN or RST
to close a TOE TLS socket. Just bail from the timer without
reschedu

Don't try to adjust a TLS TOE socket that has been closed.

The handshake timer can race with another thread sending a FIN or RST
to close a TOE TLS socket. Just bail from the timer without
rescheduling if the connection is closed when the timer fires.

Reported by: Sony Arpita Das @ Chelsio QA
Reviewed by: np
Differential Revision: https://reviews.freebsd.org/D27583

show more ...

Clear TLS offload mode if a TLS socket hangs without receiving data.

By default, if a TOE TLS socket stops receiving data for more than 5
seconds, revert the connection back to plain TOE mode. This

Clear TLS offload mode if a TLS socket hangs without receiving data.

By default, if a TOE TLS socket stops receiving data for more than 5
seconds, revert the connection back to plain TOE mode. This provides
a fallback if the userland SSL library does not support KTLS. In
addition, for client TLS 1.3 sockets using connect(), the TOE socket
blocks before the handshake has completed since the socket option is
only invoked for the final handshake.

The timeout defaults to 5 seconds, but can be changed at boot via the
hw.cxgbe.toe.tls_rx_timeout tunable or for an individual interface via
the dev.<nexus>.toe.tls_rx_timeout sysctl.

Reviewed by: np
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27470

show more ...

Clear TLS offload mode for unsupported cipher suites and versions.

If TOE TLS is requested for an unsupported cipher suite or TLS
version, disable TLS processing and fall back to plain TOE. In
addi

Clear TLS offload mode for unsupported cipher suites and versions.

If TOE TLS is requested for an unsupported cipher suite or TLS
version, disable TLS processing and fall back to plain TOE. In
addition, if an error occurs when saving the decryption keys in the
card's memory, disable TLS processing and fall back to plain TOE.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27468

show more ...

Fix downgrading of TOE TLS sockets to plain TOE.

If a TOE TLS socket ends up using an unsupported TLS version or
ciphersuite, it must be downgraded to a "plain" TOE socket with TLS
encryption/decryp

Fix downgrading of TOE TLS sockets to plain TOE.

If a TOE TLS socket ends up using an unsupported TLS version or
ciphersuite, it must be downgraded to a "plain" TOE socket with TLS
encryption/decryption performed on the host. The previous
implementation of this fallback was incomplete and resulted in hung
connections.

Reviewed by: np
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27467

show more ...

Handle CPL_RX_DATA on active TLS sockets.

In certain edge cases, the NIC might have only received a partial TLS
record which it needs to return to the driver. For example, if the
local socket was c

Handle CPL_RX_DATA on active TLS sockets.

In certain edge cases, the NIC might have only received a partial TLS
record which it needs to return to the driver. For example, if the
local socket was closed while data was still in flight, a partial TLS
record might be pending when the connection is closed. Receiving a
RST in the middle of a TLS record is another example. When this
happens, the firmware returns the the partial TLS record as plain TCP
data via CPL_RX_DATA. Handle these requests by returning an error to
OpenSSL (via so_error for KTLS or via an error TLS record header for
the older Chelsio OpenSSL interface).

Reported by: Sony Arpita Das @ Chelsio
Reviewed by: np
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: Revision: https://reviews.freebsd.org/D26800

show more ...