mbuf: Don't force all M_EXTPG mbufs to be read-only

Some M_EXTPG mbufs are read-only (e.g. those backing sendfile
requests), but others are not. Add a flags argument to
mb_alloc_ext_pgs that can be

mbuf: Don't force all M_EXTPG mbufs to be read-only

Some M_EXTPG mbufs are read-only (e.g. those backing sendfile
requests), but others are not. Add a flags argument to
mb_alloc_ext_pgs that can be used to set M_RDONLY when needed rather
than setting it unconditionally. Update mb_unmapped_to_ext to
preserve M_RDONLY from the unmapped mbuf.

Reviewed by: gallatin
Differential Revision: https://reviews.freebsd.org/D46783

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

pmap_{un}map_io_transient: Use bool instead of boolean_t.

Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D39920

tcp: remove INP_TIMEWAIT flag

Mechanically cleanup INP_TIMEWAIT from the kernel sources. After
0d7445193ab, this commit shall not cause any functional changes.

Note: this flag was very often check

tcp: remove INP_TIMEWAIT flag

Mechanically cleanup INP_TIMEWAIT from the kernel sources. After
0d7445193ab, this commit shall not cause any functional changes.

Note: this flag was very often checked together with INP_DROPPED.
If we modify in_pcblookup*() not to return INP_DROPPED pcbs, we
will be able to remove most of this checks and turn them to
assertions. Some of them can be turned into assertions right now,
but that should be carefully done on a case by case basis.

Differential revision: https://reviews.freebsd.org/D36400

show more ...

cxgbei: Adjust the calculation for the maximum ISO payload.

Round down the maximim ISO payload by the current MSS. Otherwise the
round up by MSS when calculating the 16-bit maximum payload len pass

cxgbei: Adjust the calculation for the maximum ISO payload.

Round down the maximim ISO payload by the current MSS. Otherwise the
round up by MSS when calculating the 16-bit maximum payload len passed
along in the FLOWC work request can overflow.

Discussed with: np
Sponsored by: Chelsio Communications

show more ...

cxgbei: Return limits for the specific adapter the socket is on.

Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34929

iscsi: Fetch limits based on a socket rather than assuming global limits.

cxgbei needs the ability to return different limits based on the
connection (e.g. if the connection is over a T5 adapter or

iscsi: Fetch limits based on a socket rather than assuming global limits.

cxgbei needs the ability to return different limits based on the
connection (e.g. if the connection is over a T5 adapter or a T6
adapter as well as factoring in the MTU).

This change plumbs through the changes in the ioctls without changing
any of the backends. The limits callback passed to icl_register now
accepts a second socket argument which holds the integer file
descriptor. To support ABI compatiblity for old binaries, the
callback should return "global" values if the socket fd is zero.

The CTL_ISCSI_LIMITS argument used with CTL_ISCSI by ctld(8) now
accepts the socket fd in a field that was previously part of a
reserved spare field. Old binaries zero this request which results in
passing a socket fd of 0 to the limits callback.

The ISCSIDREQUEST ioctl no longer returns limits. Instead, iscsid(8)
invokes a new ISCSIDLIMITS ioctl after establishing the connection via
connect(2). For ABI compat, if the old ISCSIDREQUEST is invoked, the
global limits are still fetched (with a socket fd of 0) and returned.

Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34928

show more ...

cxgbei: Support unmapped I/O requests.

- Add icl_pdu_append_bio and icl_pdu_get_bio methods.

- Add new page pod routines for allocating and writing page pods for
unmapped bio requests. Use these

cxgbei: Support unmapped I/O requests.

- Add icl_pdu_append_bio and icl_pdu_get_bio methods.

- Add new page pod routines for allocating and writing page pods for
unmapped bio requests. Use these new routines for setting up DDP
for iSCSI tasks with a SCSI I/O CCB which uses CAM_DATA_BIO.

- When ICL_NOCOPY is used to append data from an unmapped I/O request
to a PDU, construct unmapped mbufs from the relevant pages backing
the struct bio. This also requires changes in the t4_push_pdus path
to support unmapped mbufs.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34383

show more ...

cxgbei: Replace worker thread pools with per-connection kthreads.

Having a single pool of worker threads adds extra complexity and
overhead. The software backend also uses per-connection kthreads.

cxgbei: Replace worker thread pools with per-connection kthreads.

Having a single pool of worker threads adds extra complexity and
overhead. The software backend also uses per-connection kthreads.

Sponsored by: Chelsio Communications

show more ...

cxgbei: Dispatch sent PDUs to the NIC asynchronously.

Previously the driver was called to send PDUs to the NIC synchronously
from the icl_conn_pdu_queue_cb callback. However, this performed a
fair

cxgbei: Dispatch sent PDUs to the NIC asynchronously.

Previously the driver was called to send PDUs to the NIC synchronously
from the icl_conn_pdu_queue_cb callback. However, this performed a
fair bit of work while holding the icl connection lock. Instead,
change the callback to add sent PDUs to a STAILQ and defer dispatching
of PDUs to the NIC to a helper thread similar to the scheme used in
the TCP iSCSI backend.

- Replace rx_flags int and the sole RXF_ACTIVE flag with a simple
rx_active bool.

- Add a pool of transmit worker threads for cxgbei.

- Fix worker thread exit to depend on the wakeup in kthread_exit()
to fix a race with module unload.

Reported by: mav
Sponsored by: Chelsio Communications

show more ...

cxgbe(4): Fix "set but not used [-Wunused-but-set-variable]" warnings.

MFC after: 1 week
Sponsored by: Chelsio Communications

iscsi: Pass the request PDU to icl_conn_transfer_setup().

This matches icl_conn_task_setup() which passes the PDU and avoids the
need for a layering violation in cxgbei to fetch the request PDU from

iscsi: Pass the request PDU to icl_conn_transfer_setup().

This matches icl_conn_task_setup() which passes the PDU and avoids the
need for a layering violation in cxgbei to fetch the request PDU from
the ctl_io.

Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33746

show more ...

cxgbei: Don't fail task setup if the socket is disconnected.

When the initiator is reconnecting to the target, the connection may
temporarily be marked disconnected or not have an associated socket.

cxgbei: Don't fail task setup if the socket is disconnected.

When the initiator is reconnecting to the target, the connection may
temporarily be marked disconnected or not have an associated socket.
New I/O requests received by the initiator in this state should not
fail with ECONNRESET as that results in an I/O error back to userland.
Instead, they need to still succeed so that CAM can queue the requests
and send them once the connection is re-established.

Setting up DDP for zero-copy receive requires a socket, so just punt
on using DDP for these transfers.

Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications

show more ...

Move the ICL_CONN_*LOCK* macros to <dev/iscsi/icl.h>.

These macros are not backend-specific but reference a
backend-independent field in struct icl_conn.

Reviewed by: mav
Sponsored by: Chelsio Comm

Move the ICL_CONN_*LOCK* macros to <dev/iscsi/icl.h>.

These macros are not backend-specific but reference a
backend-independent field in struct icl_conn.

Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32858

show more ...

cxgbei: Only convert "plain" TCP connections to ISCSI.

Reject attempts to convert a connection using a different ULP
mode: (e.g. DDP or TLS) to ISCSI.

Reported by: Jithesh Arakkan @ Chelsio
Sponsor

cxgbei: Only convert "plain" TCP connections to ISCSI.

Reject attempts to convert a connection using a different ULP
mode: (e.g. DDP or TLS) to ISCSI.

Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications

show more ...

cxgbei: Return early for EBUSY error in icl_cxgbei_conn_handoff.

This permits unindenting almost half of the function.

Sponsored by: Chelsio Communications

cxgbei: Disable ISO for -SO cards without external memory.

Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications

iscsi: Teach the iSCSI stack about "large" received PDUs.

When using iSCSI PDU offload (cxgbei) on T6 adapters, a burst of
received PDUs can be reported via a single message to the driver.

Previous

iscsi: Teach the iSCSI stack about "large" received PDUs.

When using iSCSI PDU offload (cxgbei) on T6 adapters, a burst of
received PDUs can be reported via a single message to the driver.

Previously the driver passed these multi-PDU bursts up to the iSCSI
stack up as a single "large" PDU by rewriting the buffer offset, data
segment length, and DataSN fields in the iSCSI header. The DataSN
field in particular was rewritten so that each of the "large" PDUs
used consecutively increasing values. While this worked, the forged
DataSN values did not match the ExpDataSN value in the subsequent SCSI
Response PDU. The initiator does not currently verify this value, but
the forged DataSN values prevent adding a check.

To avoid this, allow a logical iSCSI PDU (struct icl_pdu) to describe
a burst of PDUs via a new 'ip_additional_pdus' field. Normally this
field is set to zero when 'struct icl_pdu' represents a single PDU.
If logical PDU represents a burst of on-the-wire PDUs, then 'ip_npdus'
contains the count of additional on-the-wire PDUs. The header of this
"large" PDU is still modified, but the DataSN field now contains the
DataSN value of the first on-the-wire PDU in the burst.

Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D31577

show more ...

cxgbei: Restructure how PDU limits are managed.

- Compute data segment limits in read_pdu_limits() rather than PDU
length limits.

- Add back connection-specific PDU overhead lengths to compute PD

cxgbei: Restructure how PDU limits are managed.

- Compute data segment limits in read_pdu_limits() rather than PDU
length limits.

- Add back connection-specific PDU overhead lengths to compute PDU
length limits in icl_cxgbei_conn_handoff().

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D31574

show more ...

cxgbei: Wait for the final CPL to be received in icl_cxgbei_conn_close.

A socket in the FIN_WAIT_1 state is marked disconnected by
do_close_con_rpl() even though there might still receive data pendi

cxgbei: Wait for the final CPL to be received in icl_cxgbei_conn_close.

A socket in the FIN_WAIT_1 state is marked disconnected by
do_close_con_rpl() even though there might still receive data pending.
This is because the socket at that point has set SBS_CANTRCVMORE which
causes the protocol layer to discard any data received before the FIN.
However, icl_cxgbei_conn_close needs to wait until all the data has
been discarded. Replace the wait for SS_ISDISCONNECTED with instead
waiting for final_cpl_received() to be called.

Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications

show more ...

cxgbei: Support for ISO (iSCSI segmentation offload).

ISO can be disabled before establishing a connection by setting
dev.tNnex.N.toe.iso to 0.

Sponsored by: Chelsio Communications
Differential Rev

cxgbei: Support for ISO (iSCSI segmentation offload).

ISO can be disabled before establishing a connection by setting
dev.tNnex.N.toe.iso to 0.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D31223

show more ...

iscsi: Remove icl_soft-only fields from struct icl_conn.

Create a struct icl_soft_conn which extends struct icl_conn and
move fields only used by icl_soft from struct icl_conn to
struct icl_soft_con

iscsi: Remove icl_soft-only fields from struct icl_conn.

Create a struct icl_soft_conn which extends struct icl_conn and
move fields only used by icl_soft from struct icl_conn to
struct icl_soft_conn.

Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D31414

show more ...

cxgbei: Round up the maximum PDU data length by the MSS for TXDATAPLEN_MAX.

Recent firmware versions round down the value passed here by the MSS
and subsequently mishandle transmitted PDUs larger th

cxgbei: Round up the maximum PDU data length by the MSS for TXDATAPLEN_MAX.

Recent firmware versions round down the value passed here by the MSS
and subsequently mishandle transmitted PDUs larger than the rounded
down value.

Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications

show more ...

cxgbei: Wait for socket to close in icl_cxgbei_conn_close.

This ensures the TOE has finished processing any in-flight received
data before returning to the caller. The caller assumes it is safe to

cxgbei: Wait for socket to close in icl_cxgbei_conn_close.

This ensures the TOE has finished processing any in-flight received
data before returning to the caller. The caller assumes it is safe to
free any open tasks or transfers (and associated buffers) after this
function returns.

Previously, data placed directly via DDP could be written to buffers
after the caller had freed the buffers.

Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications

show more ...

cxgbei: Better handle new tasks and transfers when disconnecting.

If the connection is in the process of disconnecting, ic_socket can be
NULL. For icl_cxgbei_conn_transfer_setup(), lock the connect

cxgbei: Better handle new tasks and transfers when disconnecting.

If the connection is in the process of disconnecting, ic_socket can be
NULL. For icl_cxgbei_conn_transfer_setup(), lock the connection and
check ic_socket before using it. For icl_cxgbei_conn_task_setup(),
the caller already holds the connection lock, so assert it and bail
early with ECONNRESET if the connection is disconnecting.

Reported by: Jithesh Arakkan @ Chelsio
Fixes: f949967c8eb3 cxgbei: Fix a race between transfer setup and a peer reset.

show more ...