newbus: globally replace device_add_child(..., -1) with DEVICE_UNIT_ANY

Sponsored by: Netflix

cxgbe(4): Rename rx_c_chan to rx_chan.

It is the equivalent of tx_chan but for receive so rx_chan is a better
name. Initialize both using helper functions and make sure both are
displayed in the sy

cxgbe(4): Rename rx_c_chan to rx_chan.

It is the equivalent of tx_chan but for receive so rx_chan is a better
name. Initialize both using helper functions and make sure both are
displayed in the sysctl MIB.

MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

ccr,ccp: Fix argument order to sglist_append_vmpages.

The offset comes before the byte count.

Reported by: br
Reviewed by: asomers, markj
MFC after: 1 week
Sponsored by: DARPA
Differential Revision

ccr,ccp: Fix argument order to sglist_append_vmpages.

The offset comes before the byte count.

Reported by: br
Reviewed by: asomers, markj
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D38375

show more ...

ccr: Retire ccr_softc member in struct adapter.

Prior to Conrad's changes to replace session integer IDs with a
pointer to the driver-specific state in commit 1b0909d51a8aa, the
driver had to find t

ccr: Retire ccr_softc member in struct adapter.

Prior to Conrad's changes to replace session integer IDs with a
pointer to the driver-specific state in commit 1b0909d51a8aa, the
driver had to find the softc pointer from the adapter before it could
locate the ccr_session structure for a completed request. Since
Conrad's changes, the ccr_session pointer can now be obtained directly
from the crp. Add a backpoint from ccr_session back to ccr_softc and
use this in place of the ccr_softc member in cxgbe's struct adapter.

Sponsored by: Chelsio Communications

show more ...

ccr: Remove unused devclass argument to DRIVER_MODULE.

cxgbe(4): Fix bad races between sysctl and driver detach.

The default sysctl context setup by newbus for a device is eventually
freed by device_sysctl_fini, which runs after the device driver's deta

cxgbe(4): Fix bad races between sysctl and driver detach.

The default sysctl context setup by newbus for a device is eventually
freed by device_sysctl_fini, which runs after the device driver's detach
routine. sysctl nodes associated with this context must not use any
resources (like driver locks, hardware access, counters, etc.) that are
released by driver detach.

There are a lot of sysctl nodes like this in cxgbe(4) and the fix is to
hang them off a context that is explicitly freed by the driver before it
releases any resource that might be used by a sysctl.

This fixes panics when running "sysctl dev.t6nex dev.cc" in a tight loop
and loading/unloading the driver in parallel.

Reported by: Suhas Lokesha
MFC after: 1 week
Sponsored by: Chelsio Communications

show more ...

ccr: Use a software OCF session for requests which fallback to software.

Previously the driver duplicated code from cryptosoft.c to handle
certain edge case AES-CCM and AES-GCM requests. However, t

ccr: Use a software OCF session for requests which fallback to software.

Previously the driver duplicated code from cryptosoft.c to handle
certain edge case AES-CCM and AES-GCM requests. However, this
approach has a few downsides:

1) It only uses "plain" software and not accelerated software since it
uses enc_xform directly.

2) It performs the operation synchronously even though the caller
believes it is invoking an async driver. This was fine for the
original use case of requests with only AAD and no payload that
execute quickly, but is a bit more disingenuous for large requests
which fall back due to exceeding the size of a firmware work
request (e.g. due to large scatter/gather lists).

3) It has required several updates since ccr(4) was added to the tree.

Instead, allocate a software session for AES-CCM and AES-GCM sessions
and dispatch a cloned request asynchronusly to the software session.

Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33608

show more ...

ccr: Replace 'blkcipher' with just 'cipher'.

ccr(4) can handle requests for AES-CTR (a stream cipher), not just
block ciphers, so make the function and structure names more generic.

Sponsored by: C

ccr: Replace 'blkcipher' with just 'cipher'.

ccr(4) can handle requests for AES-CTR (a stream cipher), not just
block ciphers, so make the function and structure names more generic.

Sponsored by: Chelsio Communications

show more ...

crypto: Validate AES-GCM IV length in check_csp().

This centralizes the check for valid nonce lengths for AES-GCM.

While here, remove some duplicate checks for valid AES-GCM tag lengths
from ccp(4)

crypto: Validate AES-GCM IV length in check_csp().

This centralizes the check for valid nonce lengths for AES-GCM.

While here, remove some duplicate checks for valid AES-GCM tag lengths
from ccp(4) and ccr(4).

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33194

show more ...

cryptosoft: Fix support for variable tag lengths in AES-CCM.

The tag length is included as one of the values in the flags byte of
block 0 passed to CBC_MAC, so merely copying the first N bytes is
in

cryptosoft: Fix support for variable tag lengths in AES-CCM.

The tag length is included as one of the values in the flags byte of
block 0 passed to CBC_MAC, so merely copying the first N bytes is
insufficient.

To avoid adding more sideband data to the CBC MAC software context,
pull the generation of block 0, the AAD length, and AAD padding out of
cbc_mac.c and into cryptosoft.c. This matches how GCM/GMAC are
handled where the length block is constructed in cryptosoft.c and
passed as an input to the Update callback. As a result, the CBC MAC
Update() routine is now much simpler and simply performs the
XOR-and-encrypt step on each input block.

While here, avoid a copy to the staging block in the Update routine
when one or more full blocks are passed as input to the Update
callback.

Reviewed by: sef
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32120

show more ...

ccr: Support AES-CCM requests with truncated tags.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32117

ccr: Support multiple nonce lengths for AES-CCM.

Sponsored by: Chelsio Communications, The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32116

cryptosoft, ccr: Use crp_iv directly for AES-CCM and AES-GCM.

Rather than copying crp_iv to a local array on the stack that is then
passed to xform reinit routines, pass crp_iv directly and remove t

cryptosoft, ccr: Use crp_iv directly for AES-CCM and AES-GCM.

Rather than copying crp_iv to a local array on the stack that is then
passed to xform reinit routines, pass crp_iv directly and remove the
local copy.

Reviewed by: markj
Sponsored by: Chelsio Communications, The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32106

show more ...

crypto: Permit variable-sized IVs for ciphers with a reinit hook.

Add a 'len' argument to the reinit hook in 'struct enc_xform' to
permit support for AEAD ciphers such as AES-CCM and Chacha20-Poly13

crypto: Permit variable-sized IVs for ciphers with a reinit hook.

Add a 'len' argument to the reinit hook in 'struct enc_xform' to
permit support for AEAD ciphers such as AES-CCM and Chacha20-Poly1305
which support different nonce lengths.

Reviewed by: markj
Sponsored by: Chelsio Communications, The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32105

show more ...

ccp, ccr: Simplify drivers to assume an AES-GCM IV length of 12.

While here, use crypto_read_iv() in a few more places in ccr(4) that I
missed previously.

Sponsored by: Chelsio Communications
Diffe

ccp, ccr: Simplify drivers to assume an AES-GCM IV length of 12.

While here, use crypto_read_iv() in a few more places in ccr(4) that I
missed previously.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32104

show more ...

crypto: Constify all transform descriptors

No functional change intended.

Reviewed by: ae, jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.

crypto: Constify all transform descriptors

No functional change intended.

Reviewed by: ae, jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31196

show more ...

crypto: Add a new type of crypto buffer for a single mbuf.

This is intended for use in KTLS transmit where each TLS record is
described by a single mbuf that is itself queued in the socket buffer.
U

crypto: Add a new type of crypto buffer for a single mbuf.

This is intended for use in KTLS transmit where each TLS record is
described by a single mbuf that is itself queued in the socket buffer.
Using the existing CRYPTO_BUF_MBUF would result in
bus_dmamap_load_crp() walking additional mbufs in the socket buffer
that are not relevant, but generating a S/G list that potentially
exceeds the limit of the tag (while also wasting CPU cycles).

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30136

show more ...

ccr: Disable requests on port 1 when needed to workaround a firmware bug.

Completions for crypto requests on port 1 can sometimes return a stale
cookie value due to a firmware bug. Disable requests

ccr: Disable requests on port 1 when needed to workaround a firmware bug.

Completions for crypto requests on port 1 can sometimes return a stale
cookie value due to a firmware bug. Disable requests on port 1 by
default on affected firmware.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D26581

show more ...

ccr: Add per-port stats of queued and completed requests.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29176

ccr: Set the RX channel ID correctly in work requests.

These fixes are only relevant for requests on the second port. In
some cases, the crypto completion data, completion message, and
receive desc

ccr: Set the RX channel ID correctly in work requests.

These fixes are only relevant for requests on the second port. In
some cases, the crypto completion data, completion message, and
receive descriptor could be written in the wrong order.

- Add a separate rx_channel_id that is a copy of the port's rx_c_chan
and use it when an RX channel ID is required in crypto requests
instead of using the tx_channel_id.

- Set the correct rx_channel_id in the CPL_RX_PHYS_ADDR used to write
the crypto result.

- Set the FID to the first rx queue ID on the adapter rather than the
queue ID of the first rx queue for the port.

- While here, use tx_chan to set the tx_channel_id though this is
identical to the previous value.

Reviewed by: np
Reported by: Chelsio QA
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29175

show more ...

Fallback to software for more GCM and CCM requests.

ccr(4) uses software to handle GCM and CCM requests not supported by
the crypto engine (e.g. with only AAD and no payload). This change
adds a fa

Fallback to software for more GCM and CCM requests.

ccr(4) uses software to handle GCM and CCM requests not supported by
the crypto engine (e.g. with only AAD and no payload). This change
adds a fallback for a few more requests such as those with more SGL
entries than can fit in a work request (this can happen for GCM when
decrypting a TLS record split across 15 or more packets).

Reported by: Chelsio QA
Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D26582

show more ...

MFH

Sponsored by: Rubicon Communications, LLC (netgate.com)

crypto(9): add CRYPTO_BUF_VMPAGE

crypto(9) functions can now be used on buffers composed of an array of
vm_page_t structures, such as those stored in an unmapped struct bio. It
requires the running

crypto(9): add CRYPTO_BUF_VMPAGE

crypto(9) functions can now be used on buffers composed of an array of
vm_page_t structures, such as those stored in an unmapped struct bio. It
requires the running to kernel to support the direct memory map, so not all
architectures can use it.

Reviewed by: markj, kib, jhb, mjg, mat, bcr (manpages)
MFC after: 1 week
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D25671

show more ...