sockets: make pr_aio_queue the default method

Call it directly instead of each time calling pr_aio_queue_notsupp() and
then doing the actual job. The only user of non default method was
cxgbe(4).

sockets: make pr_aio_queue the default method

Call it directly instead of each time calling pr_aio_queue_notsupp() and
then doing the actual job. The only user of non default method was
cxgbe(4). It calls now into the default method in case of own method
failure. This preserves existing behavior.

Reviewed by: markj, jhb
Differential Revision: https://reviews.freebsd.org/D48863

show more ...

cxgbe(4): Allow t4_tom to be unloaded safely.

* Disable IFCAP_TOE automatically on all ifnets on all adapters during
unload. This is user-friendly and avoids panics due to stale ifnet
state aft

cxgbe(4): Allow t4_tom to be unloaded safely.

* Disable IFCAP_TOE automatically on all ifnets on all adapters during
unload. This is user-friendly and avoids panics due to stale ifnet
state after t4_tom is unloaded.
* Do not allow unload if tids are in use by the TOE on any adapter.

Reported by: Bimal Abraham @ Chelsio
MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe/t4_tom: Remove duplicate unlock in t4_tom_deactivate.

Fixes: c1c524852f62 cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
MFC after: 1 week
Sponsored by: Chelsio Communications

cxgbe/t4_tom: Demote STALE L2 table entries to RESOLVING in uld_stop.

The STALE state means the L2T entry is valid in hardware but needs to be
refreshed (ARP/NDP) in software. But stop/suspend wipe

cxgbe/t4_tom: Demote STALE L2 table entries to RESOLVING in uld_stop.

The STALE state means the L2T entry is valid in hardware but needs to be
refreshed (ARP/NDP) in software. But stop/suspend wipes the hardware
L2T and STALE entries need to be updated just like VALID entries to match
actual hardware state.

Fixes: c1c524852f62 cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe/t4_tom: bugfixes in stop/restart.

1. Remove toepcb from the toep_list on active open failure.
2. Purge the wr_list for an L2T entry on an adpater stop.

Fixes: c1c524852f62 cxgbe/t4_tom: Imple

cxgbe/t4_tom: bugfixes in stop/restart.

1. Remove toepcb from the toep_list on active open failure.
2. Purge the wr_list for an L2T entry on an adpater stop.

Fixes: c1c524852f62 cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.
MFC after: 2 weeks
Sponsored by: Chelsio Communications

show more ...

cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.

This allows the adapter to be suspended or reset even when stateful TOE is
active, in some limited configurations.

The LLD has already

cxgbe/t4_tom: Implement uld_stop and uld_restart for ULD_TOM.

This allows the adapter to be suspended or reset even when stateful TOE is
active, in some limited configurations.

The LLD has already stopped the adapter hardware and all its queues by the time
these ULD routines get called. The general approach in t4_tom is to purge the
lookup tables immediately so that they are ready for operation by the time the
adapter resumes, and park all the resources left hanging by the stopped hardware
into separate "stranded" queues that can be dealt with at leisure.

Outstanding active opens, live connections, and synq entries (for connections in
the middle of the 3-way handshake) are all treated as if the hardware had
reported an abrupt error for the tid. The servers/listeners are a bit different
in that no error is reported. They're just noted as non-functional when the
hardware stops and are recreated by the driver during restart.

MFC after: 1 month
Sponsored by: Chelsio Communications

show more ...

cxgbe/t4_tom: Track all synq entries in a per-adapter list.

Live tid entries in tid_tab are either full fledged connections or synq
entries. toep_list tracks the connections already and this change

cxgbe/t4_tom: Track all synq entries in a per-adapter list.

Live tid entries in tid_tab are either full fledged connections or synq
entries. toep_list tracks the connections already and this change adds
a synqe_list to track the synq entries. These two lists can be used to
enumerate and iterate over all live tids.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Changes to ULD list management.

* Convert t4_uld_list to an array. There will be at most 3 items in the
list and it's simpler to track them in an array with a fixed slot for
each ULD.

cxgbe(4): Changes to ULD list management.

* Convert t4_uld_list to an array. There will be at most 3 items in the
list and it's simpler to track them in an array with a fixed slot for
each ULD.
* There is no need to refcount ULDs so stop doing that.
* Add uld_ prefix to all members of uld_info.
* Rename async_event to uld_stop to match its actual purpose. Call it
for all ULDs and not just ULD_IWARP.

Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D46029

show more ...

cxgbe/t4_tom: Free up hardware resources when the final CPL is received.

Final CPL means the tid is done in the hardware and other resources
associated with it can be freed right away. There is no

cxgbe/t4_tom: Free up hardware resources when the final CPL is received.

Final CPL means the tid is done in the hardware and other resources
associated with it can be freed right away. There is no need to wait
for the kernel to detach the toepcb.

Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45991

show more ...

cxgbe/t4_tom: Detach the toep from the tcpcb when entering TIME_WAIT.

The kernel used to call tod_pcb_detach when entering TIME_WAIT but that
seems to have changed, likely with the TIME_WAIT overhau

cxgbe/t4_tom: Detach the toep from the tcpcb when entering TIME_WAIT.

The kernel used to call tod_pcb_detach when entering TIME_WAIT but that
seems to have changed, likely with the TIME_WAIT overhaul in the kernel
some time ago. Catch up by having the driver perform the detach.

The hardware does not handle TIME_WAIT so it's important to detach and
let the kernel arm the 2MSL timer to deal with it.

Reported by: Sony Arpita Das @ Chelsio
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D45990

show more ...

cxgbe(4): Consolidate all mk_set_tcb_field_ulp in one place.

MFC after: 1 week
Sponsored by: Chelsio Communications

cxgbe/tom: Fix the rx channel selection in options2.

This affects TOE operation when multiple rx c-channels are in use for
offload, which is an unusual configuration.

MFC after: 1 week
Sponsored by

cxgbe/tom: Fix the rx channel selection in options2.

This affects TOE operation when multiple rx c-channels are in use for
offload, which is an unusual configuration.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Remove tx_modq lookup table.

The driver always uses the same modulation queue as the channel and the
table is unnecessary.

MFC after: 1 week
Sponsored by: Chelsio Communications

cxgbe: Support TCP_USE_DDP on offloaded TOE connections

When this socket option is enabled, relatively large contiguous
buffers are allocated and used to receive data from the remote
connection. Wh

cxgbe: Support TCP_USE_DDP on offloaded TOE connections

When this socket option is enabled, relatively large contiguous
buffers are allocated and used to receive data from the remote
connection. When data is received a wrapper M_EXT mbuf is queued to
the socket's receive buffer. This reduces the length of the linked
list of received mbufs and allows consumers to consume receive data in
larger chunks.

To minimize reprogramming the page pods in the adapter, receive
buffers for a given connection are recycled. When a buffer has been
fully consumed by the receiver and freed, the buffer is placed on a
per-connection free buffers list.

The size of the receive buffers defaults to 256k and can be set via
the hw.cxgbe.toe.ddp_rcvbuf_len sysctl. The
hw.cxgbe.toe.ddp_rcvbuf_cache sysctl (defaults to 4) determines the
maximum number of free buffers cached per connection. Note that this
limit does not apply to "in-flight" receive buffers that are
associated with mbufs in the socket's receive buffer.

Co-authored-by: Navdeep Parhar <np@FreeBSD.org>
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44001

show more ...

cxgbe tom: Enable ULP_MODE_TCPDDP on demand

Most ULP modes in cxgbe's TOE are enabled on the fly when a protocol
is needed (e.g. ULP_MODE_ISCSI is enabled by cxgbei when offloading a
connection usin

cxgbe tom: Enable ULP_MODE_TCPDDP on demand

Most ULP modes in cxgbe's TOE are enabled on the fly when a protocol
is needed (e.g. ULP_MODE_ISCSI is enabled by cxgbei when offloading a
connection using iSCSI, and ULP_MODE_TLS is enabled when RX TLS keys
are programmed for a TOE connection). The one exception to this is
ULP_MODE_TCPDDP.

Currently the cxgbe driver enables ULP_MODE_TCPDDP when a TOE
connection is first created. However, since DDP connections cannot be
converted to other connection types, this requires some special
handling in the driver. For example, iSCSI daemons use the SO_NO_DDP
socket option to ensure TOE connections use ULP_MODE_NONE so they can
be converted to ULP_MODE_ISCSI. Similarly, using TLS receive offload
(ULP_MODE_TLS) requires disabling TCP DDP for new connections by
default.

This commit changes cxgbe to instead switch a connection from
ULP_MODE_NONE to ULP_MODE_TCPDDP when a connection first attempts to
use TCP DDP via aio_read(2). This permits connections to always start
as ULP_MODE_NONE and switch to a protocol-specific mode as needed.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D43670

show more ...

sys: Use mbufq_empty instead of comparing mbufq_len against 0

Reviewed by: bz, emaste
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D43338

tcp_info: Add and export more FreeBSD-specific fields

This change adds struct tcp_info fields corresponding to the following
struct tcpcb ones:
- snd_una
- snd_max
- rcv_numsacks
- rcv_adv
- dupacks

tcp_info: Add and export more FreeBSD-specific fields

This change adds struct tcp_info fields corresponding to the following
struct tcpcb ones:
- snd_una
- snd_max
- rcv_numsacks
- rcv_adv
- dupacks

Note that while both tcp_fill_info() and fill_tcp_info_from_tcb() are
extended accordingly, no counterpart of rcv_numsacks is available in
the cxgbe(4) TOE PCB, though.

Sponsored by: NetApp, Inc. (originally)

show more ...

tcp_fill_info(): Change lock assertion on INPCB to locked only

This function actually only ever reads from the TCP PCB. Consequently,
also make the pointer to its TCP PCB parameter const.

Sponsored

tcp_fill_info(): Change lock assertion on INPCB to locked only

This function actually only ever reads from the TCP PCB. Consequently,
also make the pointer to its TCP PCB parameter const.

Sponsored by: NetApp, Inc. (originally)

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

IfAPI: Add if_llsoftc member accessors for TOEDEV

Summary:
Keep TOEDEV() macro for backwards compatibility, and add a SETTOEDEV()
macro to complement with the new accessors.

Sponsored by: Juniper N

IfAPI: Add if_llsoftc member accessors for TOEDEV

Summary:
Keep TOEDEV() macro for backwards compatibility, and add a SETTOEDEV()
macro to complement with the new accessors.

Sponsored by: Juniper Networks, Inc.
Reviewed by: glebius
Differential Revision: https://reviews.freebsd.org/D38199

show more ...

cxgbe: Enable TOE TLS RX when an RX key is provided via setsockopt().

Rather than requiring a socket to be created as a TLS socket from the
get go, switch a TOE socket from "plain" TOE to TLS mode w

cxgbe: Enable TOE TLS RX when an RX key is provided via setsockopt().

Rather than requiring a socket to be created as a TLS socket from the
get go, switch a TOE socket from "plain" TOE to TLS mode when a
receive key is added to the socket.

The firmware is only able to switch a "plain" TOE connection to TLS
mode if the head of the pending socket data is the start of a TLS
record, so the connection is migrated to TLS mode as a multi-step
process.

When TOE TLS RX is enabled, the associated connection's receive side
is frozen via a flag in the TCB. The state of the socket buffer is
then examined to determine if the pending data in the socket buffer
ends on a TLS record boundary. If so, the connection is migrated to
TLS mode and unfrozen. Otherwise, the connection is unfrozen
temporarily until more data arrives. Once more data arrives, the
receive queue is frozen again and rechecked. This continues until the
connection is paused at a record boundary. Any records received
before TLS mode is enabled are decrypted as software records.

Note that this removes the 'rx_tls_ports' sysctl. TOE TLS offload for
receive is now enabled automatically on existing TOE connections when
using a KTLS-aware SSL library just as it was previously enabled
automatically for TLS transmit. This also enables TLS offload for TOE
connections which enable TLS after passing initial data in the clear
(e.g. STARTTLS with SMTP).

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37351

show more ...

tcp: provide macros to access inpcb and socket from a tcpcb

There should be no functional changes with this commit.

Reviewed by: rscheff
Differential revision: https://reviews.freebsd.org/D37123

cxgbe: use standard sototcpcb() accessor macro to get socket's tcpcb

Reviewed by: np
Differential revision: https://reviews.freebsd.org/D37041

cxgbe/tom: Fix assertions in the code that maintains TCB history.

The tids used for TOE connections start from tid_base, not 0.

MFC after: 1 week
Sponsored by: Chelsio Communications